MicroPython code may rely on the return value of sys.stdout.buffer.write()
to reflect the number of bytes actually written. While in most scenarios a
write() operation is successful, there are cases where it fails, leading to
data loss. This problem arises because, currently, write() merely returns
the number of bytes it was supposed to write, without indication of
failure.
One scenario where write() might fail, is where USB is used and the
receiving end doesn't read quickly enough to empty the receive buffer. In
that case, write() on the MicroPython side can timeout, resulting in the
loss of data without any indication, a behavior observed notably in
communication between a Pi Pico as a client and a Linux host using the ACM
driver.
A complex issue arises with mp_hal_stdout_tx_strn() when it involves
multiple outputs, such as USB, dupterm and hardware UART. The challenge is
in handling cases where writing to one output is successful, but another
fails, either fully or partially. This patch implements the following
solution:
mp_hal_stdout_tx_strn() attempts to write len bytes to all of the possible
destinations for that data, and returns the minimum successful write
length.
The implementation of this is complicated by several factors:
- multiple outputs may be enabled or disabled at compiled time
- multiple outputs may be enabled or disabled at runtime
- mp_os_dupterm_tx_strn() is one such output, optionally containing
multiple additional outputs
- each of these outputs may or may not be able to report success
- each of these outputs may or may not be able to report partial writes
As a result, there's no single strategy that fits all ports, necessitating
unique logic for each instance of mp_hal_stdout_tx_strn().
Note that addressing sys.stdout.write() is more complex due to its data
modification process ("cooked" output), and it remains unchanged in this
patch. Developers who are concerned about accurate return values from
write operations should use sys.stdout.buffer.write().
This patch might disrupt some existing code, but it's also expected to
resolve issues, considering that the peculiar return value behavior of
sys.stdout.buffer.write() is not well-documented and likely not widely
known. Therefore, it's improbable that much existing code relies on the
previous behavior.
Signed-off-by: Maarten van der Schrieck <maarten@thingsconnected.nl>
Necessary to get coverage of the new event functions.
Deletes the case that called usleep(delay) for mp_hal_delay_ms(), it seems
like this wouldn't have ever happened anyhow (MICROPY_EVENT_POOL_HOOK is
always defined for the unix port).
This work was funded through GitHub Sponsors.
Signed-off-by: Angus Gratton <angus@redyak.com.au>
This adds #ifdefs around each of the mp_hal_* time functions for the unix
port. This allows variants to override individual functions as needed.
Signed-off-by: David Lechner <david@pybricks.com>
This introduces a new macro to get the main thread and uses it to ensure
that asynchronous exceptions such as KeyboardInterrupt (CTRL+C) are only
scheduled on the main thread. This is more deterministic than being
scheduled on a random thread and is more in line with CPython that only
allow signal handlers to run on the main thread.
Fixes issue #7026.
Signed-off-by: David Lechner <david@pybricks.com>
This moves mp_pending_exception from mp_state_vm_t to mp_state_thread_t.
This allows exceptions to be scheduled on a specific thread.
Signed-off-by: David Lechner <david@pybricks.com>
https://www.python.org/dev/peps/pep-0475/
This implements something similar to PEP 475 on the unix port, and for the
VfsPosix class.
There are a few differences from the CPython implementation:
- Since we call mp_handle_pending() between any ENITR's, additional
functions could be called if MICROPY_ENABLE_SCHEDULER is enabled, not
just signal handlers.
- CPython only handles signal on the main thread, so other threads will
raise InterruptedError instead of retrying. On MicroPython,
mp_handle_pending() will currently raise exceptions on any thread.
A new macro MP_HAL_RETRY_SYSCALL is introduced to reduce duplicated code
and ensure that all instances behave the same. This will also allow other
ports that use POSIX-like system calls (and use, eg, VfsPosix) to provide
their own implementation if needed.
The mp_keyboard_interrupt() function does exactly what is needed here, and
using it gets ctrl-C working when MICROPY_ENABLE_SCHEDULER is enabled on
these ports (and MICROPY_ASYNC_KBD_INTR is disabled).
It is not safe to enable MICROPY_ASYNC_KBD_INTR and MICROPY_PY_THREAD_GIL
at the same time. This will trigger a compiler error to ensure that it
is not possible to make this mistake.
Addition of GIL EXIT/ENTER pairs are:
- modos: release the GIL during system calls. CPython does this as well.
- moduselect: release the GIL during the poll() syscall. This call can be
blocking, so it is important to allow other threads to run at this time.
- modusocket: release the GIL during system calls. Many of these calls can
be blocking, so it is important to allow other threads to run.
- unix_mphal: release the GIL during the read and write syscalls in
mp_hal_stdin_rx_chr and mp_hal_stdout_tx_strn. If we don't do this
threads are blocked when the REPL or the builtin input function are used.
- file, main, mpconfigport.h: release GIL during syscalls in built-in
functions that could block.
The uos.dupterm() signature and behaviour is updated to reflect the latest
enhancements in the docs. It has minor backwards incompatibility in that
it no longer accepts zero arguments.
The dupterm_rx helper function is moved from esp8266 to extmod and
generalised to support multiple dupterm slots.
A port can specify multiple slots by defining the MICROPY_PY_OS_DUPTERM
config macro to an integer, being the number of slots it wants to have;
0 means to disable the dupterm feature altogether.
The unix and esp8266 ports are updated to work with the new interface and
are otherwise unchanged with respect to functionality.
Header files that are considered internal to the py core and should not
normally be included directly are:
py/nlr.h - internal nlr configuration and declarations
py/bc0.h - contains bytecode macro definitions
py/runtime0.h - contains basic runtime enums
Instead, the top-level header files to include are one of:
py/obj.h - includes runtime0.h and defines everything to use the
mp_obj_t type
py/runtime.h - includes mpstate.h and hence nlr.h, obj.h, runtime0.h,
and defines everything to use the general runtime support functions
Additional, specific headers (eg py/objlist.h) can be included if needed.
This is to keep the top-level directory clean, to make it clear what is
core and what is a port, and to allow the repository to grow with new ports
in a sustainable way.
Maarten van der Schrieck
Angus Gratton
Jim Mussared
David Lechner
Damien George
David Lechner
Damien George
Paul m. p. P
Mikhail Zakharov