There are currently 7 ports that implement machine.WDT and a lot of code is
duplicated across these implementations. This commit factors the common
parts of all these implementations to a single location in
extmod/machine_wdt.c. This common code provides the top-level Python
bindings (class and method wrappers), and then each port implements the
back end specific to that port.
With this refactor the ports remain functionally the same except for:
- The esp8266 WDT constructor now takes keyword arguments, and accepts the
"timeout" argument but raises an exception if it's not the default value
(this port doesn't support changing the timeout).
- The mimxrt and samd ports now interpret the argument to WDT.timeout_ms()
as signed and if it's negative truncate it to the minimum timeout (rather
than it being unsigned and a negative value truncating to the maximum
timeout).
Signed-off-by: Damien George <damien@micropython.org>
The unwritten API contract expected of a VFS.getcwd() by mp_vfs_getcwd()
is that its return value should be either "" or "/" when the CWD is at
the root of the VFS and otherwise start with a slash and not end with a
slash. This was not correctly implemented in VfsPosix for instances with
a non-empty root - the required leading slash, if any, was cut off
because the root length includes a trailing slash. This would result in
missing slashes in the middle of the return value of os.getcwd() or in
uninitialized garbage from beyond a string's null terminator when the
CWD was at the VFS root.
Signed-off-by: Christian Walther <cwalther@gmx.ch>
The unwritten API contract expected of a VFS by mp_vfs_lookup_path() is
that paths passed in are relative to the root of the VFS if they start
with '/' and relative to the current directory of the VFS otherwise.
This was not correctly implemented in VfsPosix for instances with a
non-empty root - all paths were interpreted relative to the root. Fix
that. Since VfsPosix tracks its CWD using the "external" CWD of the Unix
process, the correct handling for relative paths is to pass them through
unmodified.
Also, when concatenating absolute paths, fix an off-by-one resulting in
a harmless double slash (the root path already has a trailing slash).
Signed-off-by: Christian Walther <cwalther@gmx.ch>
A VfsPosix created with a relative root path would get confused when
chdir() was called on it and become unable to properly resolve absolute
paths, because changing directories effectively shifted its root. The
simplest fix for that would be to say "don't do that", but since the
unit tests themselves do it, fix it by making a relative path absolute
before storing it.
Signed-off-by: Christian Walther <cwalther@gmx.ch>
This wasn't correctly accounting for the bits-per-pixel and was returning a
bufinfo struct with the incorrect length. Instead, just forward directly
to the underlying buffer object.
Fixes issue #12563.
This work was funded through GitHub Sponsors.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
This ensures that the buffer is large enough for the specified width,
height, bits-per-pixel, and stride.
Also makes the legacy FrameBuffer1 constructor re-use the FrameBuffer
make_new to save some code size.
Fixes issue #12562.
This work was funded through GitHub Sponsors.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
Sometimes these are different file descriptors, not to mention the Unix
port, so use stderr to distinguish these error messages.
CPython prints to stdout, but it does it via a call to the logging module.
We don't want to introduce a dependency on logging, so printing to stderr
is a good alternative. One can override default_exception_handler() if
needed.
If a non-string buffer was passed to execfile, then it would be passed
as a non-null-terminated char* to mp_lexer_new_from_file.
This changes mp_lexer_new_from_file to take a qstr instead (as in almost
all cases a qstr will be created from this input anyway to set the
`__file__` attribute on the module).
This now makes execfile require a string (not generic buffer) argument,
which is probably a good fix to make anyway.
Fixes issue #12522.
This work was funded through GitHub Sponsors.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
Flushing console output in msvc builds always fails because that
output is not buffered so don't propagate that as an error (in a
simlar way as was done in 1c047742 for macOS).
Signed-off-by: stijn <stijn@ignitron.net>
This removes the duplicate code in cyw43, esp32, esp8266 that implements
the same logic as network.hostname.
Renames the `mod_network_hostname` (where we store the hostname value in
`.data`) to `mod_network_hostname_data` to make way for calling the shared
function `mod_network_hostname`.
And uses memcpy for mod_network_hostname_data, because the length of source
is already known and removes reliance on string data being null-terminated.
This work was funded through GitHub Sponsors.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
This changes from the previous limit of 15 characters. Although DHCP and
mDNS allow for up to 63, ESP32 and ESP8266 only allow 32, so this seems
like a reasonable limit to enforce across all ports (and avoids wasting the
additional memory).
Also clarifies that `MICROPY_PY_NETWORK_HOSTNAME_MAX_LEN` does not include
the null terminator (which was unclear before).
This work was funded through GitHub Sponsors.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
The following code:
server = await asyncio.start_server(...)
async with server:
... code that raises ...
would lose the original exception because the server's task would not have
had a chance to be scheduled yet, and so awaiting the task in wait_closed
would raise the cancellation instead of the original exception.
Additionally, ensures that explicitly cancelling the parent task delivers
the cancellation correctly (previously was masked by the server loop), now
this only happens if the server was closed, not when the task was
cancelled.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
Prior to this commit, BTstack would only be notified of sent UART data when
the mp_bluetooth_hci_poll() function was called for some other reason, eg
because of incoming data over UART. This is highly suboptimal.
With this commit, BTstack is now notified immediately after UART data has
been sent out. This improves the multi_bluetooth/perf_gatt_char_write.py
performance test by about a factor of 10x for write-without-response, and
about 4x for write-with-response (tested on LEGO_HUB_NO6 as instance1).
Signed-off-by: Damien George <damien@micropython.org>
iobase_ioctl expects that an ioctl method must return an integer, and will
raise otherwise.
This was tripping up aioble on Unix, where the new hybrid modselect.c
implementation will attempt to extract a file descriptor from the pollable
via ioctl(MP_STREAM_GET_FILENO).
However, ThreadSafeFlag's ioctl only supported MP_STREAM_POLL, and returned
None otherwise.
This makes it return `-1` (to match tests/extmod/select_poll_custom.py). It
should probably be `-22` (corresponding to MP_EINVAL), but the value is
never checked, and MP_EINVAL can be a different value on different ports.
This work was funded through GitHub Sponsors.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
This is a host driver for ESP32 chips running the esp-hosted firmware,
which turns ESP32s into a WLAN/BT co-processor.
Signed-off-by: iabdalkader <i.abdalkader@gmail.com>
It appears a new session ticket being issued by the server right after
completed handshake is not uncommon and shouldn't be treated as fatal.
mbedtls itself states "This error code is experimental and may be changed
or removed without notice."
Signed-off-by: Mirko Vogt <mirko-dev|mpy@nanl.de>
Whenever the PSA interface is used (if MBEDTLS_PSA_CRYPTO is defined),
psa_crypto_init() needs to be called to initialise the global PSA data
struct, before any PSA related operations.
TLSv1.3 depends on the PSA interface, TLSv1.2 only uses the PSA stack if
MBEDTLS_USE_PSA_CRYPTO is defined.
Without psa_crypto_init() every PSA related call will result in
-0x6C00/-27648 which translates to "SSL - Internal error (eg, unexpected
failure in lower-level module)".
The error is misleading, especially since mbedtls in its docs itself
advices "to return #PSA_ERROR_BAD_STATE or some other applicable error.".
Signed-off-by: Mirko Vogt <mirko-dev|mpy@nanl.de>
Add the buffer attribute to sys.stdin, sys.stdout and sys.stderr. This
provides raw access to underlying stdio streams for the unix port (and
others that use VfsPosix).
Signed-off-by: stephanelsmith <stephane.smith@titansensor.com>
On macOS, if running micropython from subprocess.check_output, then a
stdout.flush() raises error 45.
Here's a test case. This will run fine on linux, but crashes on macOS with
error 45.
import sys
import subprocess
import tempfile
with tempfile.NamedTemporaryFile('w') as fp:
fp.write('''
import sys
sys.stdout.write('hello world')
sys.stdout.flush()
print('')
''')
fp.flush()
print('py3')
o = subprocess.check_output(f'python3 {fp.name}'.split())
print(o)
print('upy')
o = subprocess.check_output(f'micropython {fp.name}'.split())
print(o)
On macOS:
py3
b'hello world\n'
upy
Traceback (most recent call last):
File "...", line 4, in <module>
OSError: 45
On unix:
py3
b'hello world\n'
upy
b'hello world\n'
Signed-off-by: stephanelsmith <stephane.smith@titansensor.com>
The primary purpose of this commit is to make decompress default to
wbits=15 when the format is gzip (or auto format with gzip detected). The
idea is that someone decompressing a gzip stream should be able to use the
default `deflate.DeflateIO(f)` and it will "just work" for any input
stream, even though it uses a lot of memory.
This is done by making uzlib report gzip files as having wbits set to 15
in their header (where it previously only set the wbits out parameter for
zlib files), and then fixing up the logic in `deflateio_init_read`.
Updates the documentation to match.
This work was funded through GitHub Sponsors.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
Otherwise if/when the finaliser runs for this newly created SSLSocket the
mbedtls state will be freed again.
Signed-off-by: Damien George <damien@micropython.org>
This is difficult to implement on cmake-based ports, and having the list
of variants in mpconfigboard.{cmake,mk} duplicates information that's
already in board.json.
This removes the existing query-variants make target from stm32 & rp2
and the definition of BOARD_VARIANTS from the various board files.
Also renames the cmake variable to MICROPY_BOARD_VARIANT to match other
variables such as MICROPY_BOARD. The make variable stays as
BOARD_VARIANT.
This work was funded through GitHub Sponsors.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
An SSL stream can only handle CLOSE and POLL ioctls. Other ones do not
make sense, or at least it doesn't make sense to pass the ioctl request
directly down to the underlying stream.
In particular MP_STREAM_GET_FILENO should not be passed to the underlying
stream because the SSL stream is not directly related to a file descriptor,
and the SSL stream must handle the polling itself.
Signed-off-by: Damien George <damien@micropython.org>
The signature of this method was poller.poll(timeout=-1, flags=0, /) but
the flags argument was not documented and is not CPython compatible. So
it's removed in this commit.
(The optional flags remains for the ipoll() method, which is documented.)
Signed-off-by: Damien George <damien@micropython.org>
A previous commit removed the unix-specific select module implementation
and made unix use the common one.
This commit adds an optimisation so that the system poll function is used
when polling objects that have a file descriptor. With this optimisation
enabled, if code registers both file-descriptor-based objects, and non-
file-descriptor-based objects with select.poll() then the following occurs:
- the system poll is called for all file-descriptor-based objects with a
timeout of 1ms
- then the bare-metal polling implementation is used for remaining objects,
which calls into their ioctl method (which can be in C or Python)
In the case where all objects have file descriptors, the system poll is
called with the full timeout requested by the caller. That makes it as
efficient as possible in the case everything has a file descriptor.
Benefits of this approach:
- all ports use the same select module implementation
- the unix port now supports polling of all objects and matches bare metal
implementations
- it's still efficient for existing cases where only files and sockets are
polled (on unix)
- the bare metal implementation does not change
- polling of SSL objects will now work on unix by calling in to the ioctl
method on SSL objects (this is required for asyncio ssl support)
Note that extmod/vfs_posix_file.c has poll disable when the optimisation is
enabled, because the code is not reachable when the optimisation is used.
Signed-off-by: Damien George <damien@micropython.org>
This provides similar functionality to the former zlib.DecompIO and
especially CPython's gzip.GzipFile for both compression and decompression.
This class can be used directly, and also can be used from Python to
implement (via io.BytesIO) zlib.decompress and zlib.compress, as well as
gzip.GzipFile.
Enable/disable this on all ports/boards that zlib was previously configured
for.
This work was funded through GitHub Sponsors.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
There are enough places that implement __exit__ by forwarding directly to
mp_stream_close that this saves code size.
For the cases where __exit__ is a no-op, additionally make their
MP_STREAM_CLOSE ioctl handled as a no-op.
This work was funded through GitHub Sponsors.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
This will be replaced with a new deflate module providing the same
functionality, with an optional frozen Python wrapper providing a
replacement zlib module.
binascii.crc32 is temporarily disabled.
This work was funded through GitHub Sponsors.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
This commit adds the SSLContext class to the ssl module, and retains the
existing ssl.wrap_socket() function to maintain backwards compatibility.
CPython deprecated the ssl.wrap_socket() function since CPython 3.7 and
instead one should use ssl.SSLContext().wrap_socket(). This commit makes
that possible.
For the axtls implementation:
- ssl.SSLContext is added, although it doesn't hold much state because
axtls requires calling ssl_ctx_new() for each new socket
- ssl.SSLContext.wrap_socket() is added
- ssl.PROTOCOL_TLS_CLIENT and ssl.PROTOCOL_TLS_SERVER are added
For the mbedtls implementation:
- ssl.SSLContext is added, and holds most of the mbedtls state
- ssl.verify_mode is added (getter and setter)
- ssl.SSLContext.wrap_socket() is added
- ssl.PROTOCOL_TLS_CLIENT and ssl.PROTOCOL_TLS_SERVER are added
The signatures match CPython:
- SSLContext(protocol)
- SSLContext.wrap_socket(sock, *, server_side=False,
do_handshake_on_connect=True, server_hostname=None)
The existing ssl.wrap_socket() functions retain their existing signature.
Signed-off-by: Damien George <damien@micropython.org>
The mod_binascii_a2b_base64() function allocates a buffer which may be
too small. It needs to be no less than three-quarters of the input
length, but is calculated as (<length> / 4) * 3 + 1, which may be less
due to integer division. Changed to (<length> * 3) / 4 + 1.
Signed-off-by: Duncan Lowther <Duncan.Lowther@glasgow.ac.uk>
This allows existing code that does `import uasyncio` or
`import uasyncio as asyncio` to continue working.
It uses the same lazy-loading as asyncio to prevent loading of unused
features.
This work was funded through GitHub Sponsors.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
The asyncio module now has much better CPython compatibility and
deserves to be just called "asyncio".
This will avoid people having to write `from uasyncio import asyncio`.
Renames all files, and updates port manifests to use the new path. Also
renames the built-in _uasyncio to _asyncio.
This work was funded through GitHub Sponsors.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
This is a MicroPython-specific module that existed to support the old
version of uasyncio. It's undocumented and not enabled on all ports and
takes up code size unnecessarily.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
Applies to drivers/examples/extmod/port-modules/tools.
This work was funded through GitHub Sponsors.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
Updates any includes, and references from Makefiles/CMake.
This essentially reverts what was done long ago in commit
136b5cbd76
This work was funded through GitHub Sponsors.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
This renames the builtin-modules, such that help('modules') and printing
the module object will show "module" rather than "umodule".
This work was funded through GitHub Sponsors.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
To be consistent with MP_UNARY_OP_INT_FLOAT and MP_UNARY_OP_INT_COMPLEX,
and allow int() to first check if a type supports __int__ before trying
other things (as per CPython).
Signed-off-by: Damien George <damien@micropython.org>
Avoiding code duplication. To enable it, set MICROPY_PY_UOS_SYNC in the
port's mpconfigport.h. It is operational only for FAT file system. For
other filesystems it's a no-op.
Signed-off-by: robert-hh <robert@hammelrath.com>
All ports that enable MICROPY_PY_MACHINE_PWM now enable these two
sub-options, so remove these sub-options altogether to force consistency in
new ports that implement machine.PWM.
Signed-off-by: Damien George <damien@micropython.org>
This fixes:
- type-comparison (E721): do not compare types, use isinstance().
- string-dot-format-missing-arguments (F524): .format call is missing
argument(s) for placeholder(s): {message}.
- f-string-missing-placeholders (F541).
- is-literal (F632): Use != to compare constant literals.
The last one is fixed by just comparing for truthfulness of `state`.
Based on extmod/utime_mphal.c, with:
- a globals dict added
- time.localtime wrapper added
- time.time wrapper added
- time.time_ns function added
New configuration options are added for this module:
- MICROPY_PY_UTIME (enabled at basic features level)
- MICROPY_PY_UTIME_GMTIME_LOCALTIME_MKTIME
- MICROPY_PY_UTIME_TIME_TIME_NS
Signed-off-by: Damien George <damien@micropython.org>
- Use HCI_TRACE macro consistently.
- Use the same colour formatting.
- Add a tool to convert to .pcap for Wireshark.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
The att_flags for descriptors does not use the same bitfield as for
characteristics. This was leading to NimBLE descriptors getting the wrong
flags set.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
This allows gatts_write(..., send_update=True) to work, which will send
notifications/indications to subscribed clients.
btstack already created the CCCD but writes to it were ignored.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
There was no event handler for central-initiated MTU exchange.
Fix truncation of notify/indicate to match NimBLE.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
This replaces the previous pending operation queue (that used to also be
shared with pending server notify/indicate ops) with a single pending
operation per connection. This allows the value handle to be correctly
passed to the Python-level events.
Also re-structure GATT client event handling to simplify the packet handler
functions.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
This adds a mechanism to track a pending notify/indicate operation that
is deferred due to the send buffer being full. This uses a tracked alloc
that is passed as the content arg to the callback.
This replaces the previous mechanism that did this via the global pending
op queue, shared with client read/write ops.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
Makes gatts_notify and gatts_indicate work in the same way: by default they
send the DB value, but you can manually override the payload.
In other words, makes gatts_indicate work the same as gatts_notify.
Note: This removes support for queuing notifications and indications on
btstack when the ACL buffer is full. This functionality will be
reimplemented in a future commit.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
This enables the use of WLAN(0).status('rssi') to get current RSSI of the
AP that the STA is connected to.
Signed-off-by: Damien George <damien@micropython.org>
The NINA socket types have the same values as modnetwork, but that may
change in the future. So check the socket types passed to socket() and
convert them (if needed) to their respective Nina socket types.
Also remove the unnecessary socket type check code from bind(), as pointed
out by @robert-hh.
When iterating over os.ilistdir(), the special directories '.' and '..'
are filtered from the results. But the code inadvertently also filtered
any file/directory which happened to match '..*'. This change fixes the
filter.
Fixes issue #11032.
Signed-off-by: Jeremy Rand <jeremy@rand-family.com>
This removes the previous WiFi driver from drivers/cyw43 (but leaves behind
the BT driver), and makes the stm32 port (i.e. PYBD and Portenta) use the
new "lib/cyw43-driver" open-source driver already in use by the rp2 port.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
This allows for a port (e.g. esp8266/esp32) to use extmod/modnetwork.c
and provide the globals dict, rather than just a list of interfaces.
When this is used, the default implementation of `network.route` and the
NIC list is not enabled.
Also splits out the LWIP-specific helpers from modnetwork.c into
network_lwip.c.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
This provides a standard interface to setting the global networking config
for all interfaces and interface types.
For ports that already use either a static hostname (mimxrt, rp2) they will
now use the configured value. The default is configured by the port
(or optionally the board).
For interfaces that previously supported .config(hostname), this is still
supported but now implemented using the global network.hostname.
Similarly, pyb.country and rp2.country are now deprecated, but the methods
still exist (and forward to network.hostname).
Because ESP32/ESP8266 do not use extmod/modnetwork.c they are not affected
by this commit.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
Since commit d6d8722558, modbtree.c is
included unconditionally in the build (if SRC_EXTMOD_C is used). So guard
the includes of system headers files in case a target doesn't have them.
Signed-off-by: Damien George <damien@micropython.org>
MicroPython overrides the axTLS port configuration file, but fails to
include <arpa/inet.h> (needed for htonl) and <sys/time.h> (needed for
gettimeofday). This results in build failures with compilers which do not
support implicit function declarations (which were removed from C in 1999).
This commit adds back the needed headers that were removed in this commit:
bd08017309
Signed-off-by: Damien George <damien@micropython.org>
Prior to this commit, the default security=-1 would be passed directly
through to the cyw43 driver to auto-detect the security type, but that
driver did not correctly handle the case of open security.
The cyw43 driver has now been changed to no longer support auto-detection,
rather it is up to the caller to always select the security type. The
defaults are now implemented in the Python bindings and are:
- if no key is given then it selects open security
- if a key is given then it selects WPA2_MIXED_PSK
Calling `wlan.connect(<ssid>)` will now connect to an open network, on
both rp2 and stm32 ports. The form `wlan.connect(<ssid>, <key>)` will
connect to a WPA2 network.
Fixes issue #9016.
Signed-off-by: Damien George <damien@micropython.org>
During the initial handshake or subsequent renegotiation, the protocol
might need to read in order to write (or conversely to write in order
to read). It might be blocked from doing so by the state of the
underlying socket (i.e. there is no data to read, or there is no space
to write).
The library indicates this condition by returning one of the errors
`MBEDTLS_ERR_SSL_WANT_READ` or `MBEDTLS_ERR_SSL_WANT_WRITE`. When that
happens, we need to enforce that the next poll operation only considers
the direction that the library indicated.
In addition, mbedtls does its own read buffering that we need to take
into account while polling, and we need to save the last error between
read()/write() and ioctl().
This was previously implemented by adding additional members to the
mp_obj_type_t defined for each NIC, which is difficult to do cleanly with
the new object type slots mechanism. The way this works is also not
supported on GCC 8.x and below.
Instead replace it with the type protocol, which is a much simpler way of
achieving the same thing.
This affects the WizNet (in non-LWIP mode) and Nina NIC drivers.
This work was funded through GitHub Sponsors.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
The compiler is not picky right now, but these are actually all syntax
errors:
- await is only valid in an async function
- async functions that use yield are actually async generators (a construct
not supported by the compiler right now)
`_WIN64` is defined for all 64-bit targets, including Arm, so it doesn't
work for detecting `x86_64`. We can use `_M_X64` instead.
Signed-off-by: David Lechner <david@pybricks.com>
Damien George
iabdalkader
Christian Walther
Jim Mussared
Matthias Urlichs
stijn
Mirko Vogt
Wang Xuancong
stephanelsmith
Duncan Lowther
Peter Harper
robert-hh
Chris Wilson
Mingjie Shen
Christian Clauss
Carlosgg
Oliver Joos
Jeremy Rand
Florian Weimer
Damien Tournoud
TPReal
David Lechner