MicroPython code may rely on the return value of sys.stdout.buffer.write()
to reflect the number of bytes actually written. While in most scenarios a
write() operation is successful, there are cases where it fails, leading to
data loss. This problem arises because, currently, write() merely returns
the number of bytes it was supposed to write, without indication of
failure.
One scenario where write() might fail, is where USB is used and the
receiving end doesn't read quickly enough to empty the receive buffer. In
that case, write() on the MicroPython side can timeout, resulting in the
loss of data without any indication, a behavior observed notably in
communication between a Pi Pico as a client and a Linux host using the ACM
driver.
A complex issue arises with mp_hal_stdout_tx_strn() when it involves
multiple outputs, such as USB, dupterm and hardware UART. The challenge is
in handling cases where writing to one output is successful, but another
fails, either fully or partially. This patch implements the following
solution:
mp_hal_stdout_tx_strn() attempts to write len bytes to all of the possible
destinations for that data, and returns the minimum successful write
length.
The implementation of this is complicated by several factors:
- multiple outputs may be enabled or disabled at compiled time
- multiple outputs may be enabled or disabled at runtime
- mp_os_dupterm_tx_strn() is one such output, optionally containing
multiple additional outputs
- each of these outputs may or may not be able to report success
- each of these outputs may or may not be able to report partial writes
As a result, there's no single strategy that fits all ports, necessitating
unique logic for each instance of mp_hal_stdout_tx_strn().
Note that addressing sys.stdout.write() is more complex due to its data
modification process ("cooked" output), and it remains unchanged in this
patch. Developers who are concerned about accurate return values from
write operations should use sys.stdout.buffer.write().
This patch might disrupt some existing code, but it's also expected to
resolve issues, considering that the peculiar return value behavior of
sys.stdout.buffer.write() is not well-documented and likely not widely
known. Therefore, it's improbable that much existing code relies on the
previous behavior.
Signed-off-by: Maarten van der Schrieck <maarten@thingsconnected.nl>
Necessary to get coverage of the new event functions.
Deletes the case that called usleep(delay) for mp_hal_delay_ms(), it seems
like this wouldn't have ever happened anyhow (MICROPY_EVENT_POOL_HOOK is
always defined for the unix port).
This work was funded through GitHub Sponsors.
Signed-off-by: Angus Gratton <angus@redyak.com.au>
All ports using this common configuration already enable time/date
validation, so this commit is a no-op change.
Signed-off-by: Damien George <damien@micropython.org>
Also move MICROPY_PY_PENDSV_ENTER/REENTER/EXIT to mphalport.h, for ports
where these are not already there.
This helps separate the hardware implementation of these macros from the
MicroPython configuration (eg for renesas-ra and stm32, the IRQ static
inline helper functions can now be moved to irq.h).
Signed-off-by: Damien George <damien@micropython.org>
And use it in all ports. The ports are unchanged, except esp8266 which now
just returns None from this function instead of the time elapsed (to match
other ports), and qemu-arm which gains this function.
Signed-off-by: Damien George <damien@micropython.org>
This is a code factoring to have the dict for the machine module in one
location, and all the ports use that same dict. The machine.soft_reset()
function implementation is also factored because it's the same for all
ports that did already implement it. Eventually more functions/bindings
can be factored.
All ports remain functionally the same, except:
- cc3200 port: gains soft_reset, mem8, mem16, mem32, Signal; loses POWER_ON
(which was a legacy constant, replaced long ago by PWRON_RESET)
- nrf port: gains Signal
- qemu-arm port: gains soft_reset
- unix port: gains soft_reset
- zephyr port: gains soft_reset, mem8, mem16, mem32
Signed-off-by: Damien George <damien@micropython.org>
This fixes the case where e.g.
struct foo_t {
mp_obj_t x;
uint16_t y;
char buf[];
};
will have `sizeof(struct foo_t)==8`, but `offsetof(struct foo_t, buf)==6`.
When computing the size to allocate for `m_new_obj_var` we need to use
offsetof to avoid over-allocating. This is important especially when it
might cause it to spill over into another GC block.
This work was funded through GitHub Sponsors.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
The contents of machine_mem.h, machine_i2c.h and machine_spi.h have been
moved into extmod/modmachine.h.
Signed-off-by: Damien George <damien@micropython.org>
The contents of machine_bitstream.h, machine_pinbase.h, machine_pulse.h and
machine_signal.h have been moved into extmod/modmachine.h.
Signed-off-by: Damien George <damien@micropython.org>
If a non-string buffer was passed to execfile, then it would be passed
as a non-null-terminated char* to mp_lexer_new_from_file.
This changes mp_lexer_new_from_file to take a qstr instead (as in almost
all cases a qstr will be created from this input anyway to set the
`__file__` attribute on the module).
This now makes execfile require a string (not generic buffer) argument,
which is probably a good fix to make anyway.
Fixes issue #12522.
This work was funded through GitHub Sponsors.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
This provides a way to build a non-DEBUG host binary that still has symbols
and debug information.
Document this for the unix port, and update a comment in the unix port
Makefile.
This work was funded through GitHub Sponsors.
Signed-off-by: Angus Gratton <angus@redyak.com.au>
- Superfluous comments in MP_DEFINE_CONST_OBJ_TYPE stop correct macro
expanding.
- MP_ERROR_TEXT now gives mp_rom_error_text_t, but we want plain const
char *.
Signed-off-by: David Yang <mmyangfl@gmail.com>
In CPython, `_thread.start_new_thread()` returns an ID that is the same ID
that is returned by `_thread.get_ident()`. The current MicroPython
implementation of `_thread.start_new_thread()` always returns `None`.
This modifies the required functions to return a value. The native thread
id is returned since this can be used for interop with other functions, for
example, `pthread_kill()` on *nix. `_thread.get_ident()` is also modified
to return the native thread id so that the values match and avoids the need
for a separate `native_id` attribute.
Fixes issue #12153.
Signed-off-by: David Lechner <david@pybricks.com>
The only reason that const had to be disabled was to make the test output
match CPython when const was involved. Instead, this commit fixes the test
to handle the lines where const is used.
Also:
- remove the special handling for MICROPY_PERSISTENT_CODE_SAVE in
unix/mpconfigport.h, and make this automatic.
- move the check for MICROPY_PERSISTENT_CODE_SAVE to where it's used (like
we do for other similar checks) and add a comment explaining it.
This work was funded through GitHub Sponsors.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
A previous commit removed the unix-specific select module implementation
and made unix use the common one.
This commit adds an optimisation so that the system poll function is used
when polling objects that have a file descriptor. With this optimisation
enabled, if code registers both file-descriptor-based objects, and non-
file-descriptor-based objects with select.poll() then the following occurs:
- the system poll is called for all file-descriptor-based objects with a
timeout of 1ms
- then the bare-metal polling implementation is used for remaining objects,
which calls into their ioctl method (which can be in C or Python)
In the case where all objects have file descriptors, the system poll is
called with the full timeout requested by the caller. That makes it as
efficient as possible in the case everything has a file descriptor.
Benefits of this approach:
- all ports use the same select module implementation
- the unix port now supports polling of all objects and matches bare metal
implementations
- it's still efficient for existing cases where only files and sockets are
polled (on unix)
- the bare metal implementation does not change
- polling of SSL objects will now work on unix by calling in to the ioctl
method on SSL objects (this is required for asyncio ssl support)
Note that extmod/vfs_posix_file.c has poll disable when the optimisation is
enabled, because the code is not reachable when the optimisation is used.
Signed-off-by: Damien George <damien@micropython.org>
The unix port has a custom select module which only works with objects that
have a file descriptor, eg files and sockets. On the other hand, bare
metal ports use the common extmod/modselect.c implementation of the select
module that supports polling of arbitrary objects, as long as those objects
provide a MP_STREAM_POLL in their ioctl implementation (which can be done
in C or Python).
This commit removes the unix-specific code and makes unix use the common
one provided by extmod/modselect.c instead. All objects with file
descriptors implement MP_STREAM_POLL so they continue to work.
Signed-off-by: Damien George <damien@micropython.org>
The asyncio module now has much better CPython compatibility and
deserves to be just called "asyncio".
This will avoid people having to write `from uasyncio import asyncio`.
Renames all files, and updates port manifests to use the new path. Also
renames the built-in _uasyncio to _asyncio.
This work was funded through GitHub Sponsors.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
Following how mkrules.cmake works. This makes it easy for a port to enable
frozen code, by defining FROZEN_MANIFEST in its Makefile.
Signed-off-by: Damien George <damien@micropython.org>
This is a MicroPython-specific module that existed to support the old
version of uasyncio. It's undocumented and not enabled on all ports and
takes up code size unnecessarily.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
Previously sys.path could be modified by append/pop or slice assignment.
This allows `sys.path = [...]`, which can be simpler in many cases, but
also improves CPython compatibility.
It also allows sys.path to be set to a tuple which means that you can
clear sys.path (e.g. temporarily) with no allocations.
This also makes sys.path (and sys.argv for consistency) able to be disabled
via mpconfig. The unix port (and upytesthelper) require them, so they
explicitly verify that they're enabled.
This work was funded through GitHub Sponsors.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
Updates any includes, and references from Makefiles/CMake.
This essentially reverts what was done long ago in commit
136b5cbd76
This work was funded through GitHub Sponsors.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
This renames the builtin-modules, such that help('modules') and printing
the module object will show "module" rather than "umodule".
This work was funded through GitHub Sponsors.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
This makes it so that sub-packages are resolved relative to their parent's
`__path__`, rather than re-resolving each parent's filesystem path.
The previous behavior was that `import foo.bar` would first re-search
`sys.path` for `foo`, then use the resulting path to find `bar`.
For already-loaded and u-prefixed modules, because we no longer need to
build the path from level to level, we no longer unnecessarily search
the filesystem. This should improve startup time.
Explicitly makes the resolving process clear:
- Loaded modules are returned immediately without touching the filesystem.
- Exact-match of builtins are also returned immediately.
- Then the filesystem search happens.
- If that fails, then the weak-link handling is applied.
This maintains the existing behavior: if a user writes `import time` they
will get time.py if it exits, otherwise the built-in utime. Whereas `import
utime` will always return the built-in.
This also fixes a regression from a7fa18c203
where we search the filesystem for built-ins. It is now only possible to
override u-prefixed builtins. This will remove a lot of filesystem stats
at startup, as micropython-specific modules (e.g. `pyb`) will no longer
attempt to look at the filesystem.
Added several improvements to the comments and some minor renaming and
refactoring to make it clearer how the import mechanism works. Overall
code size diff is +56 bytes on STM32.
This work was funded through GitHub Sponsors.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
This modifies the unix port Makefile to use += for QSTR_DEFS and
QSTR_GLOBAL_DEPENDENCIES so that variants can add additional files if
needed (similar to stm32 port).
Signed-off-by: David Lechner <david@pybricks.com>
Based on extmod/utime_mphal.c, with:
- a globals dict added
- time.localtime wrapper added
- time.time wrapper added
- time.time_ns function added
New configuration options are added for this module:
- MICROPY_PY_UTIME (enabled at basic features level)
- MICROPY_PY_UTIME_GMTIME_LOCALTIME_MKTIME
- MICROPY_PY_UTIME_TIME_TIME_NS
Signed-off-by: Damien George <damien@micropython.org>
- Use HCI_TRACE macro consistently.
- Use the same colour formatting.
- Add a tool to convert to .pcap for Wireshark.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
This adds a mechanism to track a pending notify/indicate operation that
is deferred due to the send buffer being full. This uses a tracked alloc
that is passed as the content arg to the callback.
This replaces the previous mechanism that did this via the global pending
op queue, shared with client read/write ops.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
This allows:
$ make BOARD_DIR=path/to/board
to infer BOARD=board, rather than the previous behavior that required
additionally setting BOARD explicitly.
Also makes the same change for VARIANT_DIR -> VARIANT on Unix.
This work was funded through GitHub Sponsors.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
Signed-off-by: Damien George <damien@micropython.org>
The assertion that is added here (to gc.c) fails when running this new test
if ALLOC_TABLE_GAP_BYTE is set to 0.
Signed-off-by: Jeff Epler <jepler@gmail.com>
Signed-off-by: Damien George <damien@micropython.org>
Maarten van der Schrieck
Angus Gratton
Damien George
Carlosgg
Jim Mussared
vsfos
David Yang
David Lechner
Brett Cannon
小权一句两句
Juan Francisco Cantero Hurtado
Jeff Epler
iabdalkader