MicroPython code may rely on the return value of sys.stdout.buffer.write()
to reflect the number of bytes actually written. While in most scenarios a
write() operation is successful, there are cases where it fails, leading to
data loss. This problem arises because, currently, write() merely returns
the number of bytes it was supposed to write, without indication of
failure.
One scenario where write() might fail, is where USB is used and the
receiving end doesn't read quickly enough to empty the receive buffer. In
that case, write() on the MicroPython side can timeout, resulting in the
loss of data without any indication, a behavior observed notably in
communication between a Pi Pico as a client and a Linux host using the ACM
driver.
A complex issue arises with mp_hal_stdout_tx_strn() when it involves
multiple outputs, such as USB, dupterm and hardware UART. The challenge is
in handling cases where writing to one output is successful, but another
fails, either fully or partially. This patch implements the following
solution:
mp_hal_stdout_tx_strn() attempts to write len bytes to all of the possible
destinations for that data, and returns the minimum successful write
length.
The implementation of this is complicated by several factors:
- multiple outputs may be enabled or disabled at compiled time
- multiple outputs may be enabled or disabled at runtime
- mp_os_dupterm_tx_strn() is one such output, optionally containing
multiple additional outputs
- each of these outputs may or may not be able to report success
- each of these outputs may or may not be able to report partial writes
As a result, there's no single strategy that fits all ports, necessitating
unique logic for each instance of mp_hal_stdout_tx_strn().
Note that addressing sys.stdout.write() is more complex due to its data
modification process ("cooked" output), and it remains unchanged in this
patch. Developers who are concerned about accurate return values from
write operations should use sys.stdout.buffer.write().
This patch might disrupt some existing code, but it's also expected to
resolve issues, considering that the peculiar return value behavior of
sys.stdout.buffer.write() is not well-documented and likely not widely
known. Therefore, it's improbable that much existing code relies on the
previous behavior.
Signed-off-by: Maarten van der Schrieck <maarten@thingsconnected.nl>
If a port defines MICROPY_SOFT_TIMER_TICKS_MS then soft_timer assumes a
SysTick back end, and provides a soft_timer_next variable that sets when
the next call to soft_timer_handler() should occur.
Otherwise, a port should provide soft_timer_get_ms() and
soft_timer_schedule_at_ms() with appropriate semantics (see comments).
Existing users of soft_timer should continue to work as they did.
Signed-off-by: Damien George <damien@micropython.org>
Change the rp2 and renesas-ra ports to use the helper function.
Saves copy-pasta, at the small cost of one more function call in the
firmware (if not using LTO).
This work was funded through GitHub Sponsors.
Signed-off-by: Angus Gratton <angus@redyak.com.au>
dcd_event_handler() is called from the IRQ when a new DCD event is queued
for processing by the TinyUSB thread mode task. This lets us queue the
handler to run immediately when MicroPython resumes.
Currently this relies on a linker --wrap hack to work, but a PR has been
submitted to TinyUSB to allow the function to be called inline from
dcd_event_handler() itself.
This work was funded through GitHub Sponsors.
Signed-off-by: Angus Gratton <angus@redyak.com.au>
C99 says that strncmp has UB for either string being NULL, so the
current behavior is technically correct, but it's an easy fix to handle
this case correctly.
7.1.4: "unless explicitly stated otherwise in the detailed
description... if an argument to a function has ...null pointer.. the
behavior is undefined".
7.21.1: "Unless explicitly stated otherwise in the description of a
particular function in this subclause, pointer arguments on such a call
shall still have valid values, as described in 7.1.4".
Also make the same change for the minimal version in bare-arm/lib.c.
This work was funded through GitHub Sponsors.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
This prevents each port Makefile from having to add an explicit rule for
`build-BOARD/pins_BOARD.c`.
This work was funded through GitHub Sponsors.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
This updates a small number of files that change with ruff-format's (vs
black's) rules.
This work was funded through GitHub Sponsors.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
If a non-string buffer was passed to execfile, then it would be passed
as a non-null-terminated char* to mp_lexer_new_from_file.
This changes mp_lexer_new_from_file to take a qstr instead (as in almost
all cases a qstr will be created from this input anyway to set the
`__file__` attribute on the module).
This now makes execfile require a string (not generic buffer) argument,
which is probably a good fix to make anyway.
Fixes issue #12522.
This work was funded through GitHub Sponsors.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
This was previously hard-coded to "Micropy" / "Mass Storage" / "1.0".
Now allow it to be overridden by a board.
Also change "Micropy" to "MicroPy" and "1.0" to "1.00" to match stm32.
This work was funded through GitHub Sponsors.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
The DHCP server broadcasts messages. They are being sent via the default
netif which might be completely the wrong network. We want to send
messages to the netif we got the original message from.
Original author: [Peter Harper](https://github.com/peterharperuk)
Source: https://github.com/raspberrypi/pico-examples/pull/392
Signed-off-by: Samveen <samveen@samveen.in>
There are enough places that implement __exit__ by forwarding directly to
mp_stream_close that this saves code size.
For the cases where __exit__ is a no-op, additionally make their
MP_STREAM_CLOSE ioctl handled as a no-op.
This work was funded through GitHub Sponsors.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
On targets that provide a reference TinyUSB implementation, like ESP32,
the SDK already defines and implements standard callback functions such
as tud_cdc_line_state_cb(). This causes a symbol clash when enabling
shared implementations like the MicroPython 1200 touch functionality.
To avoid this symbol clash, add an optional macro to allow ports to
use a different function name in the shared implementation.
Signed-off-by: Luca Burelli <l.burelli@arduino.cc>
Previously sys.path could be modified by append/pop or slice assignment.
This allows `sys.path = [...]`, which can be simpler in many cases, but
also improves CPython compatibility.
It also allows sys.path to be set to a tuple which means that you can
clear sys.path (e.g. temporarily) with no allocations.
This also makes sys.path (and sys.argv for consistency) able to be disabled
via mpconfig. The unix port (and upytesthelper) require them, so they
explicitly verify that they're enabled.
This work was funded through GitHub Sponsors.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
This fixes:
- type-comparison (E721): do not compare types, use isinstance().
- string-dot-format-missing-arguments (F524): .format call is missing
argument(s) for placeholder(s): {message}.
- f-string-missing-placeholders (F541).
- is-literal (F632): Use != to compare constant literals.
The last one is fixed by just comparing for truthfulness of `state`.
This reverts commit 0613d3e356.
The value of CFG_TUD_CDC_EP_BUFSIZE should match the endpoint size,
otherwise data may stay in the lower layers of the USB stack (and not
passed up to the higher layers) until a total of CFG_TUD_CDC_EP_BUFSIZE
bytes have been received, which may not happen for a long time.
Fixes issue #11253.
Signed-off-by: Damien George <damien@micropython.org>
Helps prevent the filesystem from getting formatted by mistake, among other
things. For example, on a Pico board, entering Ctrl+D and Ctrl+C fast many
times will eventually wipe the filesystem (without warning or notice).
Further rationale: Ctrl+C is used a lot by automation scripts (eg mpremote)
and UI's (eg Mu, Thonny) to get the board into a known state. If the board
is not responding for a short time then it's not possible to know if it's
just a slow start up (eg in _boot.py), or an infinite loop in the main
application. The former should not be interrupted, but the latter should.
The only way to distinguish these two cases would be to wait "long enough",
and if there's nothing on the serial after "long enough" then assume it's
running the application and Ctrl+C should break out of it. But defining
"long enough" is impossible for all the different boards and their possible
behaviour. The solution in this commit is to make it so that frozen
start-up code cannot be interrupted by Ctrl+C. That code then effectively
acts like normal C start-up code, which also cannot be interrupted.
Note: on the stm32 port this was never seen as an issue because all
start-up code is in C. But now other ports start to put more things in
_boot.py and so this problem crops up.
Signed-off-by: David Grayson <davidegrayson@gmail.com>
Previously, setting MICROPY_HW_ENABLE_USBDEV to 0 caused build errors. The
change affects the nrf and samd ports as well, so MICROPY_HW_ENABLE_USBDEV
had to be explicitly enabled there.
The configuration options MICROPY_HW_ENABLE_USBDEV and
MICROPY_HW_ENABLE_UART_REPL are independent, and can be enabled or disabled
by a board.
Signed-off-by: Damien George <damien@micropython.org>
The C-level printf is usually used for internal debugging prints, and a
port/board may want to redirect this somewhere other than stdout.
Signed-off-by: Damien George <damien@micropython.org>
The previous computation incorrectly assumed that the uint32_t ticks
counter MICROPY_SOFT_TIMER_TICKS_MS was in the range [0,0x80000000) where
its actually [0,0xffffffff]. This means the diff calculation can be
simplified compared to the original implementation copied from
utime_mphal.c, which has to deal with a ticks range constrained by the
small int range.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
Before, both uwTick and mp_hal_ticks_ms() were used as clock source. That
assumes, that these two are synchronous and start with the same value,
which may be not the case for all ports. If the lag between uwTick and
mp_hal_ticks_ms() is larger than the timer interval, the timer would either
rush up until the times are synchronous, or not start until uwTick wraps
over.
As suggested by @dpgeorge, MICROPY_SOFT_TIMER_TICKS_MS is now used in
softtimer.c, which has to be defined in a port's mpconfigport.h with
the variable that holds the SysTick counter.
Note that it's not possible to switch everything in softtimer.c to use
mp_hal_ticks_ms() because the logic in SysTick_Handler that schedules
soft_timer_handler() uses (eg on mimxrt) the uwTick variable directly
(named systick_ms there), and mp_hal_ticks_ms() uses a different source
timer. Thus it is made fully configurable.
This drops the `.cpu` directive from the ARM gchelper_*.s files. Having
this directive breaks the linker when targeting older CPUs (e.g. `-mthumb
-mthumb-interwork` for `-mcpu=arm7tdmi`). The actual target CPU should be
determined by the compiler options.
The exact CPU doesn't actually matter, but rather the supported assembly
instruction set. So the files are renamed to *_thumb1.s and *thumb2.s to
indicate the instruction set support instead of the CPU support.
Signed-off-by: David Lechner <david@pybricks.com>
App the mp_ prefix to usbd_ symbols and files which are defined here and
not in TinyUSB.
rp2 only for now. This includes some groundwork for dynamic USB devices
(defined in Python).
This work was funded through GitHub Sponsors.
Signed-off-by: Angus Gratton <angus@redyak.com.au>
Instead of being an explicit field, it's now a slot like all the other
methods.
This is a marginal code size improvement because most types have a make_new
(100/138 on PYBV11), however it improves consistency in how types are
declared, removing the special case for make_new.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
The goal here is to remove a slot (making way to turn make_new into a slot)
as well as reduce code size by the ~40 references to mp_identity_getiter
and mp_stream_unbuffered_iter.
This introduces two new type flags:
- MP_TYPE_FLAG_ITER_IS_ITERNEXT: This means that the "iter" slot in the
type is "iternext", and should use the identity getiter.
- MP_TYPE_FLAG_ITER_IS_CUSTOM: This means that the "iter" slot is a pointer
to a mp_getiter_iternext_custom_t instance, which then defines both
getiter and iternext.
And a third flag that is the OR of both, MP_TYPE_FLAG_ITER_IS_STREAM: This
means that the type should use the identity getiter, and
mp_stream_unbuffered_iter as iternext.
Finally, MP_TYPE_FLAG_ITER_IS_GETITER is defined as a no-op flag to give
the default case where "iter" is "getiter".
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
stdio_obj_print is private to this file so can be made static. The __del__
method does nothing so can be removed (it's only called by the GC if it
exists, so if it doesn't exist it won't be called). And FileIO doesn't
support a constructor in MicroPython at this stage.
Signed-off-by: Damien George <damien@micropython.org>
This uses MP_REGISTER_ROOT_POINTER() to register repl_line
instead of using a conditional inside of mp_state_vm_t.
Signed-off-by: David Lechner <david@pybricks.com>
This uses MP_REGISTER_ROOT_POINTER() to register the readline_history root
pointer array used by shared/readline.c and removes the registration from
all mpconfigport.h files.
This also required adding a new MICROPY_READLINE_HISTORY_SIZE config option
since not all ports used the same sized array.
Signed-off-by: David Lechner <david@pybricks.com>
Change the default DNS to match the gateway IP of a board running in access
point mode (or otherwise acting as a "server").
This fixes the rather meaningless use of "8.8.8.8" as the default DNS
server address offered up to access point clients via the DHCP server.
Since most devices wont be able to proxy access to the real "8.8.8.8".
It allows for a DNS responder to run and provide a catchall response for
captive portal functionality, or just a quality-of-life response to a
friendly URL for access-point based configuration and other applications.
Signed-off-by: Phil Howard <phil@gadgetoid.com>
This contains a string useful for identifying the underlying machine. This
string is kept consistent with the second part of the REPL banner via the
new config option MICROPY_BANNER_MACHINE.
This makes os.uname() more or less redundant, as all the information in
os.uname() is now available in the sys module.
Signed-off-by: Damien George <damien@micropython.org>
This commit adds the git hash and build date to sys.version. This is
allowed according to CPython docs, and is what PyPy does. The docs state:
A string containing the version number of the Python interpreter plus
additional information on the build number and compiler used.
Eg on CPython:
Python 3.10.4 (main, Mar 23 2022, 23:05:40) [GCC 11.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> sys.version
'3.10.4 (main, Mar 23 2022, 23:05:40) [GCC 11.2.0]'
and PyPy:
Python 2.7.12 (5.6.0+dfsg-4, Nov 20 2016, 10:43:30)
[PyPy 5.6.0 with GCC 6.2.0 20161109] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>>> import sys
>>>> sys.version
'2.7.12 (5.6.0+dfsg-4, Nov 20 2016, 10:43:30)\n[PyPy 5.6.0 with GCC ...
With this commit on MicroPython we now have:
MicroPython v1.18-371-g9d08eb024 on 2022-04-28; linux [GCC 11.2.0] v...
Use Ctrl-D to exit, Ctrl-E for paste mode
>>> import sys
>>> sys.version
'3.4.0; MicroPython v1.18-371-g9d08eb024 on 2022-04-28'
Note that the start of the banner is the same as the end of sys.version.
This helps to keep code size under control because the string can be reused
by the compiler.
Signed-off-by: Damien George <damien@micropython.org>
Auto-indent still works as the default behaviour, but it is now undone and
disabled if there is a space/tab immediately after an automatically-added
indent. This makes the REPL behaviour closer to CPython, and in particular
allows text to be pasted at the normal REPL.
Addresses issue #7925.
Signed-off-by: Damien George <damien@micropython.org>
Maarten van der Schrieck
Damien George
Angus Gratton
Jim Mussared
Peter Harper
iabdalkader
Luca Burelli
David Lechner
Christian Clauss
David Grayson
robert-hh
Blake Felt
Simon Arlott
Phil Howard