MicroPython code may rely on the return value of sys.stdout.buffer.write()
to reflect the number of bytes actually written. While in most scenarios a
write() operation is successful, there are cases where it fails, leading to
data loss. This problem arises because, currently, write() merely returns
the number of bytes it was supposed to write, without indication of
failure.
One scenario where write() might fail, is where USB is used and the
receiving end doesn't read quickly enough to empty the receive buffer. In
that case, write() on the MicroPython side can timeout, resulting in the
loss of data without any indication, a behavior observed notably in
communication between a Pi Pico as a client and a Linux host using the ACM
driver.
A complex issue arises with mp_hal_stdout_tx_strn() when it involves
multiple outputs, such as USB, dupterm and hardware UART. The challenge is
in handling cases where writing to one output is successful, but another
fails, either fully or partially. This patch implements the following
solution:
mp_hal_stdout_tx_strn() attempts to write len bytes to all of the possible
destinations for that data, and returns the minimum successful write
length.
The implementation of this is complicated by several factors:
- multiple outputs may be enabled or disabled at compiled time
- multiple outputs may be enabled or disabled at runtime
- mp_os_dupterm_tx_strn() is one such output, optionally containing
multiple additional outputs
- each of these outputs may or may not be able to report success
- each of these outputs may or may not be able to report partial writes
As a result, there's no single strategy that fits all ports, necessitating
unique logic for each instance of mp_hal_stdout_tx_strn().
Note that addressing sys.stdout.write() is more complex due to its data
modification process ("cooked" output), and it remains unchanged in this
patch. Developers who are concerned about accurate return values from
write operations should use sys.stdout.buffer.write().
This patch might disrupt some existing code, but it's also expected to
resolve issues, considering that the peculiar return value behavior of
sys.stdout.buffer.write() is not well-documented and likely not widely
known. Therefore, it's improbable that much existing code relies on the
previous behavior.
Signed-off-by: Maarten van der Schrieck <maarten@thingsconnected.nl>
esp8266 doesn't need ets task because the notify is now scheduled (see
commits 7d57037906 and
c60caf1995 for relevant history).
Signed-off-by: Damien George <damien@micropython.org>
This is necessary to avoid watchdog timeout in long i2c.scan(), as
previously machine_i2c.c would call MICROPY_EVENT_POLL_HOOK if
MICROPY_EVENT_POLL_HOOK_FAST was not available.
Compared to previous implementation, this implementation removes the
ets_event_poll() function and calls the SDK function ets_loop_iter() from
MICROPY_INTERNAL_EVENT_HOOK instead. This allows using the port-agnostic
functions in more places.
There is a small behaviour change, which is that the event loop gets
iterated in a few more places (i.e. anywhere that mp_event_handle_nowait()
is called). However, this looks like maybe only modselect.c - and is
probably good to process Wi-Fi events in that polling loop.
This work was funded through GitHub Sponsors.
Signed-off-by: Angus Gratton <angus@redyak.com.au>
This header has no include guards and is apparently only supposed to be
included from py/mphal.h.
This work was funded through GitHub Sponsors.
Signed-off-by: Angus Gratton <angus@redyak.com.au>
This is a best-effort implementation of write polling. It's difficult to
do correctly because if there are multiple output streams (eg UART and USB
CDC) then some may not be writeable while others are. A full solution
should also have a return value from mp_hal_stdout_tx_strn(), returning the
number of bytes written to the stream(s). That's also hard to define.
The renesas-ra and stm32 ports already implement a similar best-effort
mechanism for write polling.
Fixes issue #11026.
Signed-off-by: Damien George <damien@micropython.org>
For time-based functions that work with absolute time there is the need for
an Epoch, to set the zero-point at which the absolute time starts counting.
Such functions include time.time() and filesystem stat return values. And
different ports may use a different Epoch.
To make it clearer what functions use the Epoch (whatever it may be), and
make the ports more consistent with their use of the Epoch, this commit
renames all Epoch related functions to include the word "epoch" in their
name (and remove references to "2000").
Along with this rename, the following things have changed:
- mp_hal_time_ns() is now specified to return the number of nanoseconds
since the Epoch, rather than since 1970 (but since this is an internal
function it doesn't change anything for the user).
- littlefs timestamps on the esp8266 have been fixed (they were previously
off by 30 years in nanoseconds).
Otherwise, there is no functional change made by this commit.
Signed-off-by: Damien George <damien@micropython.org>
The ones that are moved out of iRAM should not need to be there, because
either they call functions in iROM (eg mp_hal_stdout_tx_str), or they are
only ever called from a function in iROM and not from an interrupt (eg
ets_esf_free_bufs).
This frees up about 800 bytes of iRAM.
This change is made for two reasons:
1. A 3rd-party library (eg berkeley-db-1.xx, axtls) may use the system
provided errno for certain errors, and yet MicroPython stream objects
that it calls will be using the internal mp_stream_errno. So if the
library returns an error it is not known whether the corresponding errno
code is stored in the system errno or mp_stream_errno. Using the system
errno in all cases (eg in the mp_stream_posix_XXX wrappers) fixes this
ambiguity.
2. For systems that have threading the system-provided errno should always
be used because the errno value is thread-local.
For systems that do not have an errno, the new lib/embed/__errno.c file is
provided.
Note: the uncrustify configuration is explicitly set to 'add' instead of
'force' in order not to alter the comments which use extra spaces after //
as a means of indenting text for clarity.
Previous behaviour is when this argument is set to "true", in which case
the function will raise any pending exception. Setting it to "false" will
cancel any pending exception.
This patch makes it so that UART(0) can by dynamically attached to and
detached from the REPL by using the uos.dupterm function. Since WebREPL
uses dupterm slot 0 the UART uses dupterm slot 1 (a slot which is newly
introduced by this patch). UART(0) must now be attached manually in
boot.py (or otherwise) and inisetup.py is changed to provide code to do
this. For example, to attach use:
import uos, machine
uart = machine.UART(0, 115200)
uos.dupterm(uart, 1)
and to detach use:
uos.dupterm(None, 1)
When attached, all incoming chars on UART(0) go straight to stdin so
uart.read() will always return None. Use sys.stdin.read() if it's needed
to read characters from the UART(0) while it's also used for the REPL (or
detach, read, then reattach). When detached the UART(0) can be used for
other purposes.
If there are no objects in any of the dupterm slots when the REPL is
started (on hard or soft reset) then UART(0) is automatically attached.
Without this, the only way to recover a board without a REPL would be to
completely erase and reflash (which would install the default boot.py which
attaches the REPL).
The uos.dupterm() signature and behaviour is updated to reflect the latest
enhancements in the docs. It has minor backwards incompatibility in that
it no longer accepts zero arguments.
The dupterm_rx helper function is moved from esp8266 to extmod and
generalised to support multiple dupterm slots.
A port can specify multiple slots by defining the MICROPY_PY_OS_DUPTERM
config macro to an integer, being the number of slots it wants to have;
0 means to disable the dupterm feature altogether.
The unix and esp8266 ports are updated to work with the new interface and
are otherwise unchanged with respect to functionality.
Header files that are considered internal to the py core and should not
normally be included directly are:
py/nlr.h - internal nlr configuration and declarations
py/bc0.h - contains bytecode macro definitions
py/runtime0.h - contains basic runtime enums
Instead, the top-level header files to include are one of:
py/obj.h - includes runtime0.h and defines everything to use the
mp_obj_t type
py/runtime.h - includes mpstate.h and hence nlr.h, obj.h, runtime0.h,
and defines everything to use the general runtime support functions
Additional, specific headers (eg py/objlist.h) can be included if needed.
This is to keep the top-level directory clean, to make it clear what is
core and what is a port, and to allow the repository to grow with new ports
in a sustainable way.
Maarten van der Schrieck
Damien George
Angus Gratton
Jim Mussared
Damien George
stijn