Commit Graph

1141 Commits

Author SHA1 Message Date
Jim Mussared 0e7bfc88c6 all: Use mp_obj_malloc everywhere it's applicable.
This replaces occurences of

    foo_t *foo = m_new_obj(foo_t);
    foo->base.type = &foo_type;

with

    foo_t *foo = mp_obj_malloc(foo_t, &foo_type);

Excludes any places where base is a sub-field or when new0/memset is used.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-05-03 22:28:14 +10:00
Damien George caaff940a2 extmod/uasyncio: Rename and merge TaskQueue push/pop methods.
These are internal names and can be safely renamed without affecting user
code.  push_sorted() and push_head() are merged into a single push()
method, which is already how the C version is implemented.  pop_head() is
simply renamed to pop().

The changes are:
- q.push_sorted(task, t) -> q.push(task, t)
- q.push_head(task) -> q.push(task)
- q.pop_head() -> q.pop()

The shorter names and removal of push_head() leads to a code size reduction
of between 40 and 64 bytes on bare-metal targets.

Signed-off-by: Damien George <damien@micropython.org>
2022-04-22 16:37:02 +10:00
Damien George 28e7e15c0a extmod/uasyncio: Fix bug with task ending just after gather is cancel'd.
This fixes a bug where the gather is cancelled externally and then one of
its sub-tasks (that the gather was waiting on) finishes right between the
cancellation being queued and being executed.

Signed-off-by: Damien George <damien@micropython.org>
2022-04-21 14:25:17 +10:00
Damien George f7454f850f extmod/uasyncio: Make Python Task match C version with use of asserts.
This helps to catch bugs when a Task is put on more than one pairing heap.

Signed-off-by: Damien George <damien@micropython.org>
2022-04-21 14:25:17 +10:00
Damien George aab005c75b extmod/modusocket: Provide config macro for socket.listen backlog deflt.
To make it possible to change this value for any given port or board.

Signed-off-by: Damien George <damien@micropython.org>
2022-04-11 15:28:56 +10:00
Jon Bjarni Bjarnason 919f696ad2 extmod/modusocket: Implement optional socket.listen backlog argument.
This follows the CPython change: https://bugs.python.org/issue21455

Socket listen backlog defaults to 2 if not given, based on most bare metal
targets not having many resources for a large backlog.  On UNIX it defaults
to SOMAXCONN or 128, whichever is less.
2022-04-11 15:26:47 +10:00
Damien George 11ab90391d extmod/extmod.cmake: Add micropy_lib_mbedtls component.
Signed-off-by: Damien George <damien@micropython.org>
2022-04-04 23:15:01 +10:00
Damien George 90aaf2dbef extmod/uasyncio: Fix gather cancelling and handling of exceptions.
The following fixes are made:
- cancelling a gather now cancels all sub-tasks of the gather (previously
  it would only cancel the first)
- if any sub-task of a gather raises an exception then the gather finishes
  (previously it would only finish if the first sub-task raised)

Fixes issues #5798, #7807, #7901.

Signed-off-by: Damien George <damien@micropython.org>
2022-03-30 16:07:44 +11:00
Damien George 335002a4c0 extmod/uasyncio: Allow task state to be a callable.
This implements a form of CPython's "add_done_callback()", but at this
stage it is a hidden feature and only intended to be used internally.

Signed-off-by: Damien George <damien@micropython.org>
2022-03-30 16:07:44 +11:00
Damien George 63f0e700f4 extmod/modure: Set subject begin_line so ^ doesn't match interior.
Fixes issue #8402.

Signed-off-by: Damien George <damien@micropython.org>
2022-03-16 12:21:00 +11:00
Damien George 0149cd6b8b windows: Switch to VFS subsystem and use VfsPosix.
Following the unix port.

Signed-off-by: Damien George <damien@micropython.org>
2022-03-10 00:26:36 +11:00
Damien George 2b409ef8a4 unix/moduos: Convert module to use extmod version.
All variants now use extmod/moduos.c as their uos module implementation.
In particular this means they all have MICROPY_VFS enabled and use VfsPosix
for their filesystem.

As part of this, the available functions in uos become more consistent with
other ports:
- coverage variant gets uos.urandom
- minimal and standard variant get: unlink, chdir, getcwd, listdir

Signed-off-by: Damien George <damien@micropython.org>
2022-03-09 21:13:57 +11:00
Damien George ade2720e55 esp8266/moduos: Convert module to use extmod version.
Signed-off-by: Damien George <damien@micropython.org>
2022-03-09 10:03:23 +11:00
Damien George 11b77263ef stm32/moduos: Convert module to use extmod version.
Signed-off-by: Damien George <damien@micropython.org>
2022-03-09 10:03:23 +11:00
Damien George 818be10bb5 zephyr/moduos: Convert module to use extmod version.
This also adds uos.unlink(), for all ports that use extmod/moduos.c.

Signed-off-by: Damien George <damien@micropython.org>
2022-03-09 10:03:23 +11:00
Damien George 1c53d85162 esp32/moduos: Convert module to use extmod version.
Signed-off-by: Damien George <damien@micropython.org>
2022-03-09 10:03:23 +11:00
Damien George 926b554daf extmod/moduos: Create general uos module to be used by all ports.
Based on the rp2 port version, with the rp2 port converted to use this
module.

Signed-off-by: Damien George <damien@micropython.org>
2022-03-09 10:03:23 +11:00
Daniël van de Giessen e2513bfe8d extmod/moduzlib: Fix parsing zlib header dict size.
From RFC 1950 section 2.2: "CINFO is the base-2 logarithm of the LZ77
window size, minus eight (CINFO=7 indicates a 32K window size)"

Signed-off-by: Daniël van de Giessen <daniel@dvdgiessen.nl>
2022-03-08 23:16:14 +11:00
stijn 49934fcf8b extmod/moduplatform: Move platform PP definitions into a header.
These are more generally useful than just for the module so make them
globally available, prefixed consistently with MICROPY_PLATFORM_.
2022-02-22 00:59:14 +11:00
iabdalkader 465b74e78d drivers/ninaw10: Add NIC-level ioctl function.
This commit adds support in the driver for irregular commands.  It
currently supports setting GPIO pin mode, and GPIO pin read/write value.
2022-02-18 14:35:26 +11:00
Christian Decker 2e3a2785cd extmod/modubinascii: Add newline keyword to b2a_base64 function.
This allows encoding things (eg a Basic-Auth header for a request) without
slicing the \n from the string, which allocates additional memory.

Co-authored-by: David Lechner <david@lechnology.com>
2022-01-23 10:18:01 +11:00
Damien George 7d71ae25ed extmod/machine_i2c: Increase default SoftI2C timeout to 50ms.
Some devices, eg BNO055, can stretch SCL for a long time, so make the
default large to accommodate them.  50ms matches the current default for
stm32 hardware I2C .

Signed-off-by: Damien George <damien@micropython.org>
2022-01-21 15:10:30 +11:00
Damien George 9438fb7321 extmod/modusocket: Support additional args to getaddrinfo.
Signed-off-by: Damien George <damien@micropython.org>
2022-01-21 13:40:06 +11:00
iabdalkader 1aac151d68 drivers/ninaw10: Return standard error numbers. 2022-01-21 13:35:05 +11:00
iabdalkader a63875d5ad extmod/modusocket: Create new sockets in blocking mode.
To conform with CPython and other MicroPython ports.
2022-01-21 13:34:56 +11:00
iabdalkader 155eb1361e extmod/modusocket: Add makefile() method and common socket options. 2022-01-21 13:34:06 +11:00
iabdalkader b23178a9c0 extmod/modusocket: Make setsockopt return if NIC is not connected. 2022-01-21 13:32:09 +11:00
iabdalkader e401ff8935 drivers/ninaw10: Fix timeout handling to match modusocket. 2022-01-21 13:31:41 +11:00
iabdalkader 9a61bc3aa7 extmod/network_ninaw10: Implement MP_STREAM_POLL in ioctl.
There is currently no function to query if the socket is writable.
2022-01-21 13:30:48 +11:00
Damien George e7fff736b5 extmod/modbluetooth: Put declaration of connect_cancel in correct place.
This fixes a bug introduced in 851ecb2da1

Signed-off-by: Damien George <damien@micropython.org>
2022-01-13 13:45:43 +11:00
Damien George 889dee8076 extmod/modbluetooth: Fix conditional compilation of ringbuf_put_uuid.
This fixes a bug introduced in a76604afba

Signed-off-by: Damien George <damien@micropython.org>
2022-01-13 13:45:07 +11:00
iabdalkader f2ccf87e0b extmod/network_ninaw10: Use socket timeout preset in modusocket. 2022-01-12 14:37:40 +11:00
iabdalkader 842da93011 extmod/modusocket: Initialise accepted socket state. 2022-01-12 14:37:32 +11:00
iabdalkader 67420de4f4 extmod/modusocket: Allow setting timeout on unbound sockets.
For an extended state socket, if settimeout() is called before a NIC is
bound, save the timeout until the NIC is bound.
2022-01-12 14:36:55 +11:00
Damien George 3243abfda2 extmod/moduplatform: Detect xtensa arch.
Signed-off-by: Damien George <damien@micropython.org>
2022-01-06 18:24:52 +11:00
iabdalkader 5a86031223 extmod/network_ninaw10: Make recv/recvfrom interchangeable. 2022-01-06 14:36:57 +11:00
iabdalkader 73a6b53dbe extmod/network_ninaw10: Return -1 on timeout from recv/send. 2022-01-06 14:36:55 +11:00
iabdalkader 544c232eb7 extmod/network_ninaw10: Make NIC state persistent. 2022-01-06 14:36:51 +11:00
iabdalkader 0f25e0387c extmod/network_ninaw10: Disable active connections before connecting. 2022-01-06 14:36:44 +11:00
iabdalkader 1b7eee24eb extmod/network_ninaw10: Fix config of AP mode.
* Fix missing call to connect to configure module in AP mode.
* Use enum for config/connect args indices.
2021-12-14 15:14:58 +11:00
Damien George efde4b2c75 extmod/modure: Redirect regex debug printing to mp_printf.
Signed-off-by: Damien George <damien@micropython.org>
2021-12-09 12:53:04 +11:00
Jim Mussared 3770fab334 all: Update Python formatting to latest Black version 21.12b0.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2021-12-09 12:09:40 +11:00
Jonathan Hogg 851ecb2da1 extmod/modbluetooth: Support gap_connect(None) to cancel a connection.
Allow cancellation of in-progress peripheral connections.
2021-12-01 11:56:37 +11:00
iabdalkader f7a0c98e00 extmod/network_ninaw10: Fix scan list order to match other NICs. 2021-11-19 15:41:26 +11:00
stijn 5900257dd6 extmod/uplatform: Use generic custom platform string.
Don't force the 'HAL' string to be part of the platform string because
it doesn't have a sensible meaning for all possible platforms, and
swap it with the PLATFORM_ARCH string so the strings which most platforms
have come first.
2021-11-18 10:46:14 +11:00
stijn 1e5875557a extmod/uplatform: Remove unused definitions. 2021-11-18 10:46:14 +11:00
Damien George 43d08688c3 extmod/uasyncio: Fix gather returning exceptions from a cancelled task.
Fixes issue #5882.
2021-11-17 14:11:31 +11:00
iabdalkader 43079aaf86 drivers/ninaw10: Add ublox Nina-W10 WiFi/BT module driver.
- Add WiFi/BT drivers for ublox Nina-W10 (esp32 based) module.
- Add ublox Nina-W10 Python module in extmod.
2021-11-13 23:01:03 +11:00
Jim Mussared 9519484c56 extmod/nimble: Remove workaround for OS_ENOMEM.
This was fixed in NimBLE 1.4.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2021-10-26 16:43:18 +11:00
Jim Mussared 948e3289bf extmod/nimble: Update to NimBLE v1.4.
We're using the MicroPython fork of NimBLE, which on the
`micropython_1_4_0` branch re-adds support for 64-bit targets and fixes
initialisation of g_msys_pool_list.

Also updates modbluetooth_nimble.c to suit v1.4.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2021-10-26 16:43:18 +11:00
Jim Mussared 43467b9c71 extmod/modbluetooth: Add connection interval to gap_connect.
This forwards through directly to the NimBLE and BTStack connect functions.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2021-10-26 00:16:53 +11:00
Andrew Leech 2ceeabf180 extmod/vfs_posix_file: Support MP_STREAM_POLL in vfs_posix_file_ioctl.
Allows asyncio reading of sys.stdin when MICROPY_PY_USELECT is used in the
build configuration.
2021-10-19 22:47:18 +11:00
stijn d42cba0d22 extmod/moduplatform: Improve implementation for PC ports.
Fix identification of 32/64 bit and of the Windows platform and add a
platform string mimicking CPython for the latter.
2021-09-24 13:51:39 +10:00
iabdalkader 2c5e9bbdfa extmod: Add platform module.
It contains the compiler version, and underlying system HAL/SDK version.
2021-09-19 23:35:10 +10:00
iabdalkader 38f8e852e0 rp2: Add framework for networking.
MICROPY_PY_NETWORK and MICROPY_PY_USOCKET need to be enabled by a board to
get networking.  No NICs have yet been defined.
2021-09-19 23:20:13 +10:00
iabdalkader 8064c3bf9c extmod/nimble: Add nimble CMake fragment file. 2021-09-19 23:02:16 +10:00
iabdalkader 80f2c794e6 extmod/mpbthci.h: Add mp_bluetooth_hci_uart_any prototype.
This allows drivers to use mpbthciport functions to read/write/poll UART.
2021-09-19 23:00:39 +10:00
iabdalkader d9749f90ad extmod/modnetwork: Remove modnetwork socket u_state member.
To simplify the socket state.

The CC3K driver (see drivers/cc3000/inc/socket.h and src/socket.c) has
socket() returning an INT16 so there is now enough room to store it
directly in the fileno member.
2021-09-15 11:29:02 +10:00
iabdalkader f9d573a4ac extmod/modnetwork: Remove STM32 references. 2021-09-15 11:27:38 +10:00
Damien George a34d43b2b7 extmod/network_cyw43: Make consistent use of STA and AP constants.
The network.STA_IF and network.AP_IF constants are now independent to the
CYW43_ITF_STA and CYW43_ITF_AP constants.

Signed-off-by: Damien George <damien@micropython.org>
2021-09-15 01:37:27 +10:00
iabdalkader 4dba04a50f extmod/modnetwork: Define network interfaces in port config files.
So this network implementation becomes more generic.
2021-09-15 01:29:26 +10:00
iabdalkader c13e25c329 extmod/modusocket: Add read/write stream methods to socket object.
Following other socket implementations.
2021-09-15 01:28:37 +10:00
iabdalkader e7429389a6 extmod/modnetwork: Add extended socket state. 2021-09-15 01:26:23 +10:00
iabdalkader d889f672da extmod/modnetwork: Add STA_IF and AP_IF constants. 2021-09-15 01:25:42 +10:00
iabdalkader 7aab0dc5d8 extmod: Move modnetwork and modusocket from stm32 to extmod.
So they can be used by other ports.
2021-09-15 01:25:12 +10:00
Damien George af64c2ddbd extmod/machine_pwm: Factor out machine.PWM bindings to common code.
This commit refactors machine.PWM and creates extmod/machine_pwm.c.  The
esp8266, esp32 and rp2 ports all use this and provide implementations of
the required PWM functionality.  This helps to reduce code duplication and
keep the same Python API across ports.

This commit does not make any functional changes.

Signed-off-by: Damien George <damien@micropython.org>
2021-09-04 16:31:17 +10:00
Damien George d41f6dde56 extmod/modonewire: Make _onewire module configurable via macro option.
Signed-off-by: Damien George <damien@micropython.org>
2021-09-02 13:11:23 +10:00
Damien George afe0634c98 extmod/machine_spi: Make SoftSPI configurable via macro option.
Signed-off-by: Damien George <damien@micropython.org>
2021-09-02 13:11:23 +10:00
Damien George 122d901ef1 extmod/machine_i2c: Make SoftI2C configurable via macro option.
The zephyr port doesn't support SoftI2C so it's not enabled, and the legacy
I2C constructor check can be removed.

Signed-off-by: Damien George <damien@micropython.org>
2021-09-02 13:11:23 +10:00
Damien George 30691ed2a1 drivers/cyw43: Make wifi join fail if interface is not active.
Otherwise the Python network object continues to report that it is
attempting to connect.

Also make the return error code consistent with wifi scan.

Signed-off-by: Damien George <damien@micropython.org>
2021-08-31 13:00:11 +10:00
Peter Hinch 2296df0a32 extmod/modframebuf: Enable blit between different formats via a palette.
This achieves a substantial performance improvement when rendering glyphs
to color displays, the benefit increasing proportional to the number of
pixels in the glyph.
2021-08-25 15:31:23 +10:00
Jim Mussared 870000f35b extmod: Add machine.bitstream.
This is a generic API for synchronously bit-banging data on a pin.

Initially this adds a single supported encoding, which supports controlling
WS2812 LEDs.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2021-08-19 22:50:11 +10:00
Jim Mussared 1d9e489af3 extmod/modbluetooth: Add send_update arg to gatts_write.
This allows the write to trigger a notification or indication, but only to
subscribed clients. This is different to gatts_notify/gatts_indicate,
which will unconditionally notify/indicate.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2021-08-14 22:21:55 +10:00
Damien George 8fcdb5490c extmod/modlwip: Fix close and clean up of UDP and raw sockets.
The correct callback-deregister functions must be called dependent on the
socket type, otherwise resources may not be freed correctly.

Signed-off-by: Damien George <damien@micropython.org>
2021-08-13 23:46:11 +10:00
Peter Züger ffc854f17f extmod/modujson: Add support for dump/dumps separators keyword-argument.
Optionally enabled via MICROPY_PY_UJSON_SEPARATORS.  Enabled by default.

For dump, make sure mp_get_stream_raise is called after
mod_ujson_separators since CPython does it in this order (if both
separators and stream are invalid, separators will raise an exception
first).

Add separators argument in the docs as well.

Signed-off-by: Peter Züger <zueger.peter@icloud.com>
Signed-off-by: Damien George <damien@micropython.org>
2021-08-07 13:52:16 +10:00
oclyke e29259d171 extmod/uasyncio: In open_connection use address info in socket creation.
Rudimentary support for various address families.

Signed-off-by: oclyke <oclyke@gmail.com>
2021-07-31 15:33:48 +10:00
Jim Mussared 341158c251 extmod/nimble: Add "memory stalling" mechanism for l2cap_send.
When l2cap_send detects that the sys mempool is running low (used to store
the outgoing HCI payloads), it will report stalled back to the application,
and then only unstall once these HCI payloads have been sent.

This prevents a situation where a remote receiver with very large MTU can
cause NimBLE to queue up more than MYNEWT_VAL_MSYS_1_BLOCK_COUNT (i.e. 12)
payloads, causing further attempts to send to fail with ENOMEM (even though
the channel is not stalled and we have room in the channel mbufs). The
regular credit/stall flow control is not effective here because the
receiver's MTU is large enough that it will not activate (i.e. there are
lots of credits available).

Thresholds of 1/2 (stall) and 1/4 (unstall) chosen to allow headroom for
other payloads (e.g. notifications) and that when a regular stall occurs it
might keep sending (and creating more payloads) in the background.
2021-07-23 21:58:10 +10:00
Jim Mussared edfb5d56c8 extmod/nimble: Allow modbluetooth binding to hook "sent HCI packet". 2021-07-23 21:58:04 +10:00
Jim Mussared 53dfb279da extmod/modbluetooth: Clamp MTU values to 32->UINT16_MAX. 2021-07-23 21:57:58 +10:00
Josh Lloyd db6d60b079 extmod/utime: Always invoke mp_hal_delay_ms when >= to 0ms.
This makes sleep_ms(0) useful as a "yield" so event processing and thread
switching can take place.

Fixes issue #5345.
2021-07-22 00:05:17 +10:00
Damien George 74db526cf0 extmod/btstack/btstack.mk: Use -Wno-implicit-fallthrough, not =0.
In 2ae3c890bd, -Wimplicit-fallthrough=0 was
added to get the build to pass.  This option is equivalent to
-Wno-implicit-fallthrough, and the latter is compatible with clang (while
the former is not).

Fixes issue #7546.

Signed-off-by: Damien George <damien@micropython.org>
2021-07-17 23:55:25 +10:00
Jim Mussared 12e3fcc785 extmod/nimble: Fix leak in l2cap_send if send-while-stalled.
A correctly-behaved application shouldn't do this, but in the
case where the channel is stalled, there's still enough room
in the mbuf pool, then we'll fail the send (BLE_HS_EBUSY) so
the mbuf needs to be freed.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2021-07-17 23:38:39 +10:00
David Lechner 8758504f0f extmod/moduselect: Conditionally compile select().
This adds #if MICROPY_PY_USELECT_SELECT around the uselect.select()
function. According to the docs, this function is only for CPython
compatibility and should not normally be used. So we can disable it
and save a few bytes of flash space where possible.

Signed-off-by: David Lechner <david@pybricks.com>
2021-07-17 23:32:39 +10:00
Damien George 38a204ed96 py: Introduce and use mp_raise_type_arg helper.
To reduce code size.

Signed-off-by: Damien George <damien@micropython.org>
2021-07-15 00:12:41 +10:00
Damien George 136369d72f all: Update to point to files in new shared/ directory.
Signed-off-by: Damien George <damien@micropython.org>
2021-07-12 17:08:10 +10:00
Damien George 94dfaff18b extmod: Update for move of crypto-algorithms, re1.5, uzlib to lib.
Signed-off-by: Damien George <damien@micropython.org>
2021-07-12 16:37:07 +10:00
Damien George 6dd92d7742 lib/crypto-algorithms: Move crypto-algorithms code from extmod to lib.
It's third-party code, and not necessarily tied to extmod.

Signed-off-by: Damien George <damien@micropython.org>
2021-07-12 16:36:37 +10:00
Damien George d1bfb271d7 lib/uzlib: Move uzlib code from extmod to lib.
It's third-party code, and not necessarily tied to extmod.

Signed-off-by: Damien George <damien@micropython.org>
2021-07-12 16:36:34 +10:00
Damien George d1d172f536 lib/re1.5: Move re1.5 code from extmod to lib.
It's third-party code, and not necessarily tied to extmod.

Signed-off-by: Damien George <damien@micropython.org>
2021-07-12 16:36:26 +10:00
Damien George e2f0b181f9 extmod/axtls-include: Add axtls_os_port.h to customise axTLS.
Signed-off-by: Damien George <damien@micropython.org>
2021-07-08 23:51:49 +10:00
Damien George 7ec95c2768 extmod/uasyncio: Get addr and bind server socket before creating task.
Currently when using uasyncio.start_server() the socket configuration is
done inside a uasyncio.create_task() background function.  If the address
and port are already in use however this throws an OSError which cannot be
cleanly caught behind the create_task().

This commit moves the getaddrinfo and socket binding to the start_server()
function, and only creates the task if that succeeds.  This means that any
OSError from the initial socket configuration is propagated directly up the
call stack, compatible with CPython behaviour.

See #7444.

Signed-off-by: Damien George <damien@micropython.org>
2021-06-26 22:30:22 +10:00
Jeff Epler 413f34cd8f all: Fix signed shifts and NULL access errors from -fsanitize=undefined.
Fixes the following (the line numbers match commit 0e87459e2b):

../../extmod/crypto-algorithms/sha256.c:49:19: runtime error: left shif...
../../extmod/moduasyncio.c:106:35: runtime error: member access within ...
../../py/binary.c:210:13: runtime error: left shift of negative value -...
../../py/mpz.c:744:16: runtime error: negation of -9223372036854775808 ...
../../py/objint.c:109:22: runtime error: left shift of 1 by 31 places c...
../../py/objint_mpz.c:374:9: runtime error: left shift of 4611686018427...
../../py/objint_mpz.c:374:9: runtime error: left shift of negative valu...
../../py/parsenum.c:106:14: runtime error: left shift of 46116860184273...
../../py/runtime.c:395:33: runtime error: left shift of negative value ...
../../py/showbc.c:177:28: runtime error: left shift of negative value -...
../../py/vm.c:321:36: runtime error: left shift of negative value -1```

Testing was done on an amd64 Debian Buster system using gcc-8.3 and these
settings:

    CFLAGS += -g3 -Og -fsanitize=undefined
    LDFLAGS += -fsanitize=undefined

The introduced TASK_PAIRHEAP macro's conditional (x ? &x->i : NULL)
assembles (under amd64 gcc 8.3 -Os) to the same as &x->i, since i is the
initial field of the struct.  However, for the purposes of undefined
behavior analysis the conditional is needed.

Signed-off-by: Jeff Epler <jepler@gmail.com>
2021-06-24 23:01:04 +10:00
Damien George 6ed5b843cf extmod/btstack: Check that BLE is active before performing operations.
Otherwise it can easily lead to a hard crash.

Signed-off-by: Damien George <damien@micropython.org>
2021-06-23 14:58:39 +10:00
Damien George 0fc0f7536b extmod/btstack: Add missing call to mp_bluetooth_hci_uart_deinit.
Signed-off-by: Damien George <damien@micropython.org>
2021-06-23 13:14:26 +10:00
Damien George 38bc5a9f67 stm32: Provide a custom BTstack runloop that integrates with soft timer.
It reschedules the BT HCI poll soft timer so that it is called exactly when
the next timer expires.

Signed-off-by: Damien George <damien@micropython.org>
2021-06-23 13:14:26 +10:00
Damien George 8107c9b75b extmod/nimble: Remove TODO comment about notify_custom freeing om.
The comments in NimBLE for ble_gattc_notify_custom() state that "This
function consumes the supplied mbuf regardless of the outcome.".  And
inspection of NimBLE code shows that this is the case.  So the comment can
be removed.

Signed-off-by: Damien George <damien@micropython.org>
2021-06-17 14:54:04 +10:00
Damien George 514bf1a191 extmod/uasyncio: Fix race with cancelled task waiting on finished task.
This commit fixes a problem with a race between cancellation of task A and
completion of task B, when A waits on B.  If task B completes just before
task A is cancelled then the cancellation of A does not work.  Instead,
the CancelledError meant to cancel A gets passed through to B (that's
expected behaviour) but B handles it as a "Task exception wasn't retrieved"
scenario, printing out such a message (this is because finished tasks point
their "coro" attribute to themselves to indicate they are done, and
implement the throw() method, but that method inadvertently catches the
CancelledError).  The correct behaviour is for B to bounce that
CancelledError back out.

This bug is mainly seen when wait_for() is used, and in that context the
symptoms are:
- occurs when using wait_for(T, S), if the task T being waited on finishes
  at exactly the same time as the wait-for timeout S expires
- task T will have run to completion
- the "Task exception wasn't retrieved message" is printed with
  "<class 'CancelledError'>" as the error (ie no traceback)
- the wait_for(T, S) call never returns (it's never put back on the
  uasyncio run queue) and all tasks waiting on this are blocked forever
  from running
- uasyncio otherwise continues to function and other tasks continue to be
  scheduled as normal

The fix here reworks the "waiting" attribute of Task to be called "state"
and uses it to indicate whether a task is: running and not awaited on,
running and awaited on, finished and not awaited on, or finished and
awaited on.  This means the task does not need to point "coro" to itself to
indicate finished, and also allows removal of the throw() method.

A benefit of this is that "Task exception wasn't retrieved" messages can go
back to being able to print the name of the coroutine function.

Fixes issue #7386.

Signed-off-by: Damien George <damien@micropython.org>
2021-06-16 13:02:37 +10:00
Mike Teachman b0b8ebc4f6 extmod/uasyncio: Add readinto() method to Stream class.
With docs and a multi-test using TCP server/client.

This method is a MicroPython extension, although there is discussion of
adding it to CPython: https://bugs.python.org/issue41305

Signed-off-by: Mike Teachman <mike.teachman@gmail.com>
2021-06-15 13:13:35 +10:00
Miguel Grinberg de2e081260 extmod/uasyncio: Fix start_server and wait_closed race condition.
This fix prevents server.wait_closed() from raising an AttributeError when
trying to access server.task.  This can happen if it is called immediately
after start_server().
2021-06-08 15:10:50 +10:00
Damien George c3199f5649 extmod/modurandom: Support an argument of bits=0 to getrandbits.
This was changed in CPython 3.9; see https://bugs.python.org/issue40282.

Signed-off-by: Damien George <damien@micropython.org>
2021-05-30 17:05:56 +10:00
Macarthur Inbody 34d4dab683 extmod/modurandom: Add error message when getrandbits has bad value.
The random module's getrandbits() method didn't give a proper error message
when calling it with a value that was outside of the range of 1-32, which
can lead to confusion using this function (which under CPython can accept
numbers larger than 32).  Now instead of simply giving a ValueError it
gives an error message that states that the number of bits is constrained.

Also, since the random module's functions getrandbits() and randint()
differ from CPython, tests have been added to describe these differences.
For getrandbits the relevant documentation is shown and added to the docs.
The same is given for randint method so that the information is more easily
found.

Finally, since the int object lacks the bit_length() method there is a test
for that method also to include within the docs, showing the difference to
CPython.
2021-05-30 16:41:30 +10:00
Damien George 6a127810c0 extmod/moduhashlib: Put hash obj in final state after digest is called.
If digest is called then the hash object is put in a "final" state and
calling update() or digest() again will raise a ValueError (instead of
silently producing the wrong result).

See issue #4119.

Signed-off-by: Damien George <damien@micropython.org>
2021-05-26 21:44:46 +10:00
Damien George 47583d8cbd extmod/moductypes: Fix size and offset calculation for ARRAY of FLOAT32.
uctypes.FLOAT32 has a special value representation and
uctypes_struct_scalar_size() should be used instead of GET_SCALAR_SIZE().

Signed-off-by: Damien George <damien@micropython.org>
2021-05-06 13:11:33 +10:00
Damien George 350a66a863 extmod/moductypes: Replace numbers with macro constants.
Signed-off-by: Damien George <damien@micropython.org>
2021-05-06 12:40:53 +10:00
Damien George 02dc1644b6 extmod/moductypes: Remove double blank lines and debugging printf's.
Signed-off-by: Damien George <damien@micropython.org>
2021-05-06 12:32:09 +10:00
Damien George 4791d290c6 extmod: Remove old comments used for auto-doc generation.
They are no longer used, and the text in the docs is more up to date.

Signed-off-by: Damien George <damien@micropython.org>
2021-05-06 12:27:31 +10:00
Damien George e9e9c76ddf all: Rename mp_keyboard_interrupt to mp_sched_keyboard_interrupt.
To match mp_sched_exception() and mp_sched_schedule().

Signed-off-by: Damien George <damien@micropython.org>
2021-04-30 15:13:43 +10:00
Damien George 342d55529d extmod/uasyncio: Use .errno instead of .args[0] for OSError exceptions.
Signed-off-by: Damien George <damien@micropython.org>
2021-04-23 22:03:46 +10:00
Damien George 212fe7f33e extmod/extmod.cmake: Add support to build btree module with CMake.
Signed-off-by: Damien George <damien@micropython.org>
2021-04-09 13:33:26 +10:00
Jeff Epler 172fb5230a extmod/re1.5: Check and report byte overflow errors in _compilecode.
The generated regex code is limited in the range of jumps and counts, and
this commit checks all cases which can overflow given the right kind of
input regex, and returns an error in such a case.

This change assumes that the results that overflow an int8_t do not
overflow a platform int.

Closes: #7078

Signed-off-by: Jeff Epler <jepler@gmail.com>
2021-04-06 13:36:42 +10:00
Damien George 2b888aa2f3 extmod/modbluetooth: Free temp arrays in gatts register services.
This helps to reduce memory fragmentation, by freeing the heap data as soon
as it is not needed.  It also helps the compiler keeps a reference to the
beginning of both arrays, which need to be traceable by the GC (otherwise
some compilers may optimise this reference to something else).

Signed-off-by: Damien George <damien@micropython.org>
2021-03-16 13:55:45 +11:00
Damien George eccd73a403 extmod/extmod.cmake: Add modonewire.c to MICROPY_SOURCE_EXTMOD list.
Signed-off-by: Damien George <damien@micropython.org>
2021-03-14 15:51:28 +11:00
Jim Mussared a76604afba extmod/modbluetooth: Separate enabling of "client" from "central".
Previously, the MICROPY_PY_BLUETOOTH_ENABLE_CENTRAL_MODE macro
controlled enabling both the central mode and the GATT client
functionality (because usually the two go together).

This commits adds a new MICROPY_PY_BLUETOOTH_ENABLE_GATT_CLIENT
macro that separately enables the GATT client functionality.
This defaults to MICROPY_PY_BLUETOOTH_ENABLE_CENTRAL_MODE.

This also fixes a bug in the NimBLE bindings where a notification
or indication would not be received by a peripheral (acting as client)
as gap_event_cb wasn't handling it. Now both central_gap_event_cb
and peripheral_gap_event_cb share the same common handler for these
events.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2021-02-19 17:53:43 +11:00
Damien George cf6a015880 extmod/btstack: Use MICROPY_HW_BLE_UART_BAUDRATE for first UART init.
Otherwise the UART may be left in a state at baudrate=0.

Signed-off-by: Damien George <damien@micropython.org>
2021-02-17 16:06:56 +11:00
Thorsten von Eicken 2c1299b007 extmod/modussl: Fix ussl read/recv/send/write errors when non-blocking.
Also fix related problems with socket on esp32, improve docs for
wrap_socket, and add more tests.
2021-02-17 11:50:54 +11:00
Jim Mussared 4005138882 extmod/modbluetooth: Allow NimBLE to use Zephyr static address.
Zephyr controllers can be queried for a static address (computed from the
device ID).  BlueKitchen already supports this, but make them both use the
same macro to enable the feature.
2021-02-17 11:25:02 +11:00
Jim Mussared 236274f08f extmod/nimble/hal/hal_uart: Fix HCI_TRACE format specifiers.
Makes this work consistently on unix and stm32 ports.
2021-02-17 11:24:48 +11:00
Jim Mussared 5e96e89999 extmod/uasyncio: Add ThreadSafeFlag.
This is a MicroPython-extension that allows for code running in IRQ
(hard or soft) or scheduler context to sequence asyncio code.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2021-02-16 16:35:37 +11:00
Jim Mussared fce0bd1a2a extmod/moduselect: Fix unsigned/signed comparison for timeout!=-1.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2021-02-16 14:59:19 +11:00
Jim Mussared a1a2815799 extmod/nimble: Ensure handle is set on read error.
On error, the handle is only available on err->att_handle rather than
in attr->handle used in the non-error case.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2021-02-16 14:57:10 +11:00
Damien George 66098c0985 py,extmod: Add core cmake rule files.
These allow a port to use cmake natively instead of make.

Signed-off-by: Damien George <damien@micropython.org>
2021-02-15 12:48:18 +11:00
Jim Mussared 7ed99544e4 extmod/uasyncio: Add asyncio.current_task().
Matches CPython behavior.

Fixes #6686
2021-02-13 15:11:17 +11:00
Damien George 50615fef89 extmod/btstack: Enable SYNC_EVENTS, PAIRING_BONDING by default.
Synchronous events work on stm32 and unix ports.

Signed-off-by: Damien George <damien@micropython.org>
2021-02-12 12:08:09 +11:00
Damien George 24a8a408a9 extmod/btstack: Add stub functions for passkey, l2cap bindings.
Signed-off-by: Damien George <damien@micropython.org>
2021-02-12 12:07:51 +11:00
Damien George 7535f67dfb extmod/btstack: Add HCI trace debugging option in btstack_hci_uart.
Signed-off-by: Damien George <damien@micropython.org>
2021-02-12 12:07:29 +11:00
Damien George 26b4ef4c46 extmod/vfs_posix_file: Allow closing an already closed file.
Signed-off-by: Damien George <damien@micropython.org>
2021-02-11 22:54:41 +11:00
Jim Mussared 47d02b3104 extmod/nimble: Improve the flow control for l2cap recv path.
If the _IRQ_L2CAP_RECV handler does the actual consumption of the incoming
data (i.e. via l2cap_recvinto), rather than setting a flag for
non-scheduler-context to handle it later, then two things can happen:

- It can starve the VM (i.e. the scheduled task never terminates).  This is
  because calling l2cap_recvinto will empty the rx buffer, which will grant
  more credits to the channel (an HCI command), meaning more data can
  arrive.  This means that the loop in hal_uart.c that keeps reading HCI
  data from the uart and executing NimBLE events as they are created will
  not terminate, preventing other VM code from running.

- There's no flow control (i.e. data will arrive too quickly).  The channel
  shouldn't be given credits until after we return from scheduler context.

It's preferable that no work is done in scheduler/IRQ context.  But to
prevent this being a problem this commit changes l2cap_recvinto so that if
it is called in IRQ context, and the Python handler empties the rx buffer,
then don't grant credits until the Python handler is complete.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2021-01-30 13:22:40 +11:00
Damien George ec0503bd0c extmod/modonewire: Use pin_od_high/pin_od_low instead of pin_write.
The pin is configured in open-drain mode so these od_high/od_low methods
should be used.

Signed-off-by: Damien George <damien@micropython.org>
2021-01-29 23:57:10 +11:00
Damien George 15ac5a3df9 extmod/modframebuf: Change int to unsigned int in format methods args.
These args are already bounds checked and clipped, and using unsigned ints
can be more efficient.  It also eliminates possible issues and compiler
warnings with shifting of signed integers.

Signed-off-by: Damien George <damien@micropython.org>
2021-01-29 23:57:10 +11:00
Damien George 71ea438561 extmod/vfs: Check block 0 and 1 when auto-detecting littlefs.
The superblock for littlefs is in block 0 and 1, but block 0 may be erased
or partially written, so block 1 must be checked if block 0 does not have a
valid littlefs superblock in it.

Prior to this commit, the mount of a block device which auto-detected the
filysystem type would fail for littlefs if block 0 did not contain a valid
superblock.  That is now fixed.

Signed-off-by: Damien George <damien@micropython.org>
2021-01-29 15:02:55 +11:00
Jim Mussared aa136b4d78 extmod/modbluetooth: Add ble.hci_cmd(ogf, ocf, req, resp) function.
This allows sending arbitrary HCI commands and getting the response.  The
return value of the function is the status of the command.

This is intended for debugging and not to be a part of the public API, and
must be enabled via mpconfigboard.h.  It's currently only implemented for
NimBLE bindings.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2021-01-22 18:15:12 +11:00
Jim Mussared f7aafc0628 extmod/nimble: Don't assert on save-IRK failure. 2020-12-23 10:08:00 +11:00
Jim Mussared f42a190247 extmod/nimble: Reset NimBLE BSS in mp_bluetooth_init.
Without this fix, each time service registration happened it would do an
increasingly large malloc() for service state.

See https://github.com/apache/mynewt-nimble/issues/896.
2020-12-23 10:07:49 +11:00
Oliver Joos a13d1b50c9 extmod/vfs: Raise OSError(ENODEV) if mounting bdev without a filesystem.
This commit prevents uos.mount() from raising an AttributeError.
vfs_autodetect() is supposed to return an object that has a "mount" method,
so if no filesystem is found it should raise an OSError(ENODEV) and not
return the bdev itself which has no "mount" method.
2020-12-17 22:44:03 +11:00
Damien George 1719459c28 extmod/modubinascii: Update code, docs for hexlify now CPython has sep.
Since CPython 3.8 the optional "sep" argument to hexlify is officially
supported, so update comments in the code and the docs to reflect this.

Signed-off-by: Damien George <damien@micropython.org>
2020-12-14 14:35:29 +11:00
Jim Mussared d79b9c6c7c extmod/nimble: Generate and persist a unique IRK.
This provides a workaround for
https://github.com/apache/mynewt-nimble/issues/887.

Without this, all devices would share a fixed default IRK.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2020-12-02 14:44:39 +11:00
Jim Mussared e4f27cbee7 extmod/modbluetooth: Add support for passkey authentication.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2020-12-02 14:43:32 +11:00
Jim Mussared 4bcbbfdb6c extmod/modbluetooth: Simplify synchronous invoke_irq_handler signature.
Rather than dealing with the different int types, just pass them all as a
single array of mp_int_t with n_unsigned (before addr) and n_signed (after
addr).

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2020-12-02 14:43:01 +11:00
Jim Mussared c4d08aa4e3 extmod/modbluetooth: Add support for bonding (key persistence).
This adds `_IRQ_GET_SECRET` and `_IRQ_SET_SECRET` events to allow the BT
stack to request the Python code retrive/store/delete secret key data.  The
actual keys and values are opaque to Python and stack-specific.

Only NimBLE is implemented (pending moving btstack to sync events).  The
secret store is designed to be compatible with BlueKitchen's TLV store API.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2020-12-02 14:41:41 +11:00
Jim Mussared 801e8ffacf extmod/modbluetooth: Add gap_pair(conn_handle) func to intiate pairing.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2020-12-02 14:41:26 +11:00
Jim Mussared a1fcf30121 extmod/modbluetooth: Allow configuration of pairing/bonding parameters.
This allows setting the security and MITM-protection requirements.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2020-12-02 14:40:49 +11:00
Andrew Leech 05fef8c6a4 extmod/modbluetooth: Add _IRQ_ENCRYPTION_UPDATE event.
This allows the application to be notified if any of encrypted,
authenticated and bonded state change, as well as the encryption key size.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2020-12-02 14:40:15 +11:00
Jim Mussared ac89267fef extmod/modbluetooth: Add compile-config flag to enable pairing/bonding.
Enable it on STM32/Unix NimBLE only (pairing/bonding requires synchronous
events and full bindings).

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2020-12-02 14:39:41 +11:00
Jim Mussared 60830bcba4 extmod/modbluetooth: Allow user-specified reason in read request IRQ.
Instead of returning None/bool from the IRQ, return None/int (where a zero
value means success).  This mirrors how the L2CAP_ACCEPT return value
works.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2020-12-02 14:37:55 +11:00
Andrew Leech 1697ff335d extmod/modbluetooth: Allow setting char/desc enc/auth options.
This widens the characteristic/descriptor flags to 16-bit, to allow setting
encryption/authentication requirements.

Sets the required flags for NimBLE and btstack implementations.

The BLE.FLAG_* constants will eventually be deprecated in favour of copy
and paste Python constants (like the IRQs).

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2020-12-02 14:36:50 +11:00
Andrew Leech c70665fb0b extmod/modbluetooth: Add _IRQ_CONNECTION_UPDATE event.
This allows the application to be notified of changes to the connection
interval, connection latency and supervision timeout.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2020-12-02 14:35:39 +11:00
Jim Mussared f2a9a0ac41 extmod/nimble: Fail read if the characteristic is too big.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2020-12-02 14:35:17 +11:00
Damien George b505971069 extmod/uasyncio: Fix cancellation handling of wait_for.
This commit switches the roles of the helper task from a cancellation task
to a runner task, to get the correct semantics for cancellation of
wait_for.

Some uasyncio tests are now disabled for the native emitter due to issues
with native code generation of generators and yield-from.

Fixes #5797.

Signed-off-by: Damien George <damien@micropython.org>
2020-12-02 12:31:37 +11:00
Damien George 309dfe39e0 extmod/uasyncio: Add Task.done() method.
This is added because task.coro==None is no longer the way to detect if a
task is finished.  Providing a (CPython compatible) function for this
allows the implementation to be abstracted away.

Signed-off-by: Damien George <damien@micropython.org>
2020-12-02 12:07:06 +11:00
Damien George ca40eb0fda extmod/uasyncio: Delay calling Loop.call_exception_handler by 1 loop.
When a tasks raises an exception which is uncaught, and no other task
await's on that task, then an error message is printed (or a user function
called) via a call to Loop.call_exception_handler.  In CPython this call is
made when the Task object is freed (eg via reference counting) because it's
at that point that it is known that the exception that was raised will
never be handled.

MicroPython does not have reference counting and the current behaviour is
to deal with uncaught exceptions as early as possible, ie as soon as they
terminate the task.  But this can be undesirable because in certain cases
a task can start and raise an exception immediately (before any await is
executed in that task's coro) and before any other task gets a chance to
await on it to catch the exception.

This commit changes the behaviour so that tasks which end due to an
uncaught exception are scheduled one more time for execution, and if they
are not await'ed on by the next scheduling loop, then the exception handler
is called (eg the exception is printed out).

Signed-off-by: Damien George <damien@micropython.org>
2020-12-02 12:07:06 +11:00
Jim Mussared 5a7027915c extmod/nimble/modbluetooth_nimble: Fix build when l2cap unavailable.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2020-11-24 11:57:29 +11:00
Jim Mussared 0e8af2b370 extmod/modbluetooth: Add API for L2CAP channels.
Also known as L2CAP "connection oriented channels". This provides a
socket-like data transfer mechanism for BLE.

Currently only implemented for NimBLE on STM32 / Unix.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2020-11-24 01:07:17 +11:00
Damien George 64180f0742 extmod/machine_i2c: Add init protocol method for generic I2C bindings.
Hardware I2C implementations must provide a .init() protocol method if they
want to support reconfiguration.  Otherwise the default is that i2c.init()
raises an OSError (currently the case for all ports).

mp_machine_soft_i2c_locals_dict is renamed to mp_machine_i2c_locals_dict to
match the generic SPI bindings.

Fixes issue #6623 (where calling .init() on a HW I2C would crash).

Signed-off-by: Damien George <damien@micropython.org>
2020-11-23 19:45:04 +11:00
Jim Mussared 61d1e4b01b extmod/nimble: Make stm32 and unix NimBLE ports use synchronous events.
This changes stm32 from using PENDSV to run NimBLE to use the MicroPython
scheduler instead.  This allows Python BLE callbacks to be invoked directly
(and therefore synchronously) rather than via the ringbuffer.

The NimBLE UART HCI and event processing now happens in a scheduled task
every 128ms.  When RX IRQ idle events arrive, it will also schedule this
task to improve latency.

There is a similar change for the unix port where the background thread now
queues the scheduled task.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2020-11-13 17:19:05 +11:00
Jim Mussared 81e92d3d6e extmod/modbluetooth: Re-instate optional no-ringbuf modbluetooth.
This requires that the event handlers are called from non-interrupt context
(i.e. the MicroPython scheduler).

This will allow the BLE stack (e.g. NimBLE) to run from the scheduler
rather than an IRQ like PENDSV, and therefore be able to invoke Python
callbacks directly/synchronously.  This allows writing Python BLE handlers
for events that require immediate response such as _IRQ_READ_REQUEST (which
was previous a hard IRQ) and future events relating to pairing/bonding.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2020-11-13 17:19:05 +11:00
Jim Mussared 6d9fdff8d0 extmod/nimble: Poll startup directly rather than using NimBLE sem.
Using a semaphore (the previous approach) will only run the UART, whereas
for startup we need to also run the event queue.

This change makes it run the full scheduler hook.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2020-11-13 17:19:05 +11:00
Jim Mussared c398e46b29 extmod/modbluetooth: Combine gattc-data-available callbacks into one.
Instead of having the stack indicate a "start", "data"..., "end", pass
through the data in one callback as an array of chunks of data.

This is because the upcoming non-ringbuffer modbluetooth implementation
cannot buffer the data in the ringbuffer and requires instead a single
callback with all the data, to pass to the Python callback.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2020-11-13 17:19:05 +11:00
Jim Mussared 3d890e7ab4 extmod/modbluetooth: Make UUID type accessible outside modbluetooth.c.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2020-11-13 17:19:05 +11:00
Arrowana 922f81dfd1 extmod/machine_mem: Only allow integers in machine.memX subscript.
Prior to this change machine.mem32['foo'] (or using any other non-integer
subscript) could result in a fault due to 'foo' being interpreted as an
integer.  And when writing code it's hard to tell if the fault is due to a
bad subscript type, or an integer subscript that specifies an invalid
memory address.

The type of the object used in the subscript is now tested to be an
integer by using mp_obj_get_int_truncated instead of
mp_obj_int_get_truncated.  The performance hit of this change is minimal,
and machine.memX objects are more for convenience than performance (there
are many other ways to read/write memory in a faster way),

Fixes issue #6588.
2020-11-13 11:13:37 +11:00
Damien George bdfb584b29 extmod/moductypes: Fix storing to (U)INT64 arrays on 32-bit archs.
Fixes issue #6583.

Signed-off-by: Damien George <damien@micropython.org>
2020-11-11 22:18:24 +11:00
Jim Mussared b7883ce74c extmod/nimble/nimble.mk: Add -Wno-old-style-declaration.
This is needed since -Wextra was added to the build in
bef412789e

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2020-11-05 10:14:19 +11:00
Jim Mussared 2ae3c890bd extmod/btstack/btstack.mk: Add -Wimplicit-fallthrough=0.
This is needed since -Wextra was added to the build in
bef412789e

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2020-11-05 10:13:12 +11:00
Damien George 6f34800884 extmod/modurandom: Support urandom.seed() without an argument.
If a port provides MICROPY_PY_URANDOM_SEED_INIT_FUNC as a source of
randomness then this will be used when urandom.seed() is called without
an argument (or with None as the argument) to seed the pRNG.

Other related changes in this commit:
- mod_urandom___init__ is changed to call seed() without arguments, instead
  of explicitly passing in the result of MICROPY_PY_URANDOM_SEED_INIT_FUNC.
- mod_urandom___init__ will only ever seed the pRNG once (before it could
  seed it again if imported by, eg, random and then urandom).
- The Yasmarang state is moved to the BSS for builds where the state is
  guaranteed to be initialised on import of the (u)random module.

Signed-off-by: Damien George <damien@micropython.org>
2020-10-29 14:15:16 +11:00
Damien George 03a1f94ea1 extmod/vfs_lfs: Support mounting LFS filesystems in read-only mode.
Signed-off-by: Damien George <damien@micropython.org>
2020-10-29 11:43:52 +11:00
Emil Renner Berthing ccd92335a1 py, extmod: Introduce and use MP_FALLTHROUGH macro.
Newer GCC versions are able to warn about switch cases that fall
through.  This is usually a sign of a forgotten break statement, but in
the few cases where a fall through is intended we annotate it with this
macro to avoid the warning.
2020-10-22 11:53:16 +02:00
Emil Renner Berthing dde3db21fc extmod: Disable -Wmissing-field-initializers for lfs2. 2020-10-22 11:47:36 +02:00
Emil Renner Berthing 9aa58cf8ba py, extmod: Add explicit initializers for default values.
When compiling with -Wextra which includes -Wmissing-field-initializers
GCC will warn that the defval field of mp_arg_val_t is not initialized.
This is just a warning as it is defined to be zero initialized, but since
it is a union it makes sense to be explicit about which member we're
going to use, so add the explicit initializers and get rid of the
warning.
2020-10-22 11:47:36 +02:00
Damien George d4b61b0017 extmod/utime_mphal: Add generic utime.time_ns() function.
It requires mp_hal_time_ns() to be provided by a port.  This function
allows very accurate absolute timestamps.

Enabled on unix, windows, stm32, esp8266 and esp32.

Signed-off-by: Damien George <damien@micropython.org>
2020-10-01 14:20:42 +10:00
Damien George 71f3ade770 ports: Support legacy soft I2C/SPI construction via id=-1 arg.
With a warning that this way of constructing software I2C/SPI is
deprecated.  The check and warning will be removed in a future release.

This should help existing code to migrate to the new SoftI2C/SoftSPI types.

Signed-off-by: Damien George <damien@micropython.org>
2020-10-01 12:57:10 +10:00
Damien George 9e0533b9e1 extmod/machine_spi: Remove "id" arg in SoftSPI constructor.
The SoftSPI constructor is now used soley to create SoftSPI instances, it
can no longer delegate to create a hardware-based SPI instance.

Signed-off-by: Damien George <damien@micropython.org>
2020-10-01 12:57:10 +10:00
Damien George aaed33896b extmod/machine_i2c: Remove "id" arg in SoftI2C constructor.
The SoftI2C constructor is now used soley to create SoftI2C instances, it
can no longer delegate to create a hardware-based I2C instance.

Signed-off-by: Damien George <damien@micropython.org>
2020-10-01 12:57:10 +10:00
Damien George c35deb2625 extmod/machine_i2c: Rename type to SoftI2C and add custom print method.
Also rename machine_i2c_type to mp_machine_soft_i2c_type.  These changes
make it clear that it's a soft-I2C implementation, and match SoftSPI.

Signed-off-by: Damien George <damien@micropython.org>
2020-10-01 12:57:10 +10:00
Andrew Leech 319437d4bd extmod/modure: Allow \\ in re.sub replacements. 2020-09-30 23:18:34 +10:00
Damien George 81f2162ca0 extmod/modbluetooth: Change module-owned bytes objects to memoryview.
A read-only memoryview object is a better representation of the data, which
is owned by the ubluetooth module and may change between calls to the
user's irq callback function.

Signed-off-by: Damien George <damien@micropython.org>
2020-09-25 12:23:11 +10:00
Damien George 71adf506ce extmod/vfs: Fix lookup of entry in root dir so it fails correctly.
Prior to this commit, uos.chdir('/') followed by uos.stat('noexist') would
succeed that stat even though the entry did not exist (some other functions
like listdir would have similar issues).  This is because, if the current
directory was the root and the path was relative, mp_vfs_lookup_path would
return success for bad paths.

Signed-off-by: Damien George <damien@micropython.org>
2020-09-23 16:23:35 +10:00
Damien George 8f20cdc353 all: Rename absolute time-based functions to include "epoch".
For time-based functions that work with absolute time there is the need for
an Epoch, to set the zero-point at which the absolute time starts counting.
Such functions include time.time() and filesystem stat return values.  And
different ports may use a different Epoch.

To make it clearer what functions use the Epoch (whatever it may be), and
make the ports more consistent with their use of the Epoch, this commit
renames all Epoch related functions to include the word "epoch" in their
name (and remove references to "2000").

Along with this rename, the following things have changed:

- mp_hal_time_ns() is now specified to return the number of nanoseconds
  since the Epoch, rather than since 1970 (but since this is an internal
  function it doesn't change anything for the user).

- littlefs timestamps on the esp8266 have been fixed (they were previously
  off by 30 years in nanoseconds).

Otherwise, there is no functional change made by this commit.

Signed-off-by: Damien George <damien@micropython.org>
2020-09-18 17:20:34 +10:00
Jim Mussared 857e2c8fd5 extmod/modbluetooth: Implement MTU. 2020-09-18 12:51:21 +10:00
Jim Mussared 5be3bfcb7e extmod/modbluetooth: Print UUIDs correctly.
In particular, the printed string can now be re-evaluated to construct the
same UUID object.
2020-09-15 15:29:13 +10:00
Jim Mussared 6a6a5f9e15 extmod/modbluetooth: Make BLE.irq() method positional only.
Simplifcation now that the trigger arg has been removed.
2020-09-15 15:28:45 +10:00
Jim Mussared 504522bd02 extmod/modbluetooth: Fix handling of optional data/uuid args.
For the following 3 functions, previously the code relied on whether the
arg was passed at all, to make it optional.  Now it allows them to be
explicitly `None` to indicate they are not used:

- gatts_notify(..., [data])
- gattc_discover_services(..., [uuid])
- gattc_discover_characteristics(..., [uuid])

Also ensure that the uuid arguments are actually instances of the uuid
type, and fix handling of the 5th arg in gattc_discover_characteristics().
2020-09-15 15:23:51 +10:00
Damien George 373b400632 extmod/modussl_axtls: Reduce size of code that makes exception.
Change in code size (for ports that use axtls) is:

   unix x64:  -152 -0.030% [incl -160(data)]
unix nanbox:  -112 -0.025% [incl -96(data)]
    esp8266:   -64 -0.009% GENERIC

Signed-off-by: Damien George <damien@micropython.org>
2020-09-11 10:22:19 +10:00
Jim Mussared 126f972c34 extmod/nimble: Add timeout for HCI sync on startup.
This allows `ble.active(1)` to fail correctly if the HCI controller is
unavailable.

It also avoids an infine loop in the NimBLE event handler where NimBLE
doesn't correctly detect that the HCI controller is unavailable and keeps
trying to reset.

Furthermore, it fixes an issue where GATT service registrations were left
allocated, which led to a bad realloc if the stack was activated multiple
times.
2020-09-08 12:53:24 +10:00
Jim Mussared 99a29ec705 extmod/btstack: Detect HCI UART init failure. 2020-09-08 12:53:24 +10:00
Andrew Leech 8b00aeab8f extmod/btstack: Add btstack support for _IRQ_GATTS_READ_REQUEST. 2020-09-08 12:53:24 +10:00
Jim Mussared 52a2ce45de extmod/modbluetooth: Allow using mp_hal_get_mac as a static address.
Generally a controller should either have its own public address hardcoded,
or loaded by the driver (e.g. cywbt43).

However, for a controller that has no public address where you still want a
long-term stable address, this allows you to use a static address generated
by the port.  Typically on STM32 this will be an LAA, but a board might
override this.
2020-09-08 12:53:24 +10:00
Jim Mussared c4af714d58 extmod/modbluetooth: Implement configuration of address modes.
Changes `BLE.config('mac')` to return a tuple (addr_mode, addr).

Adds `BLE.config(addr_mode=...)` to set the addressing mode.
2020-09-08 12:53:24 +10:00
Jim Mussared 1b1b22905e unix: Implement BLE H4 HCI UART for btstack/nimble.
This commit adds support for using Bluetooth on the unix port via a H4
serial interface (distinct from a USB dongle), with both BTstack and NimBLE
Bluetooth stacks.

Note that MICROPY_PY_BLUETOOTH is now disabled for the coverage variant.
Prior to this commit Bluetooth was anyway not being built on Travis because
libusb was not detected.  But now that bluetooth works in H4 mode it will
be built, and will lead to a large decrease in coverage because Bluetooth
tests cannot be run on Travis.
2020-09-08 12:53:24 +10:00
Jim Mussared aa18ab7db2 extmod/nimble: Implement NimBLE mutex. 2020-09-08 11:41:31 +10:00
Jim Mussared f3f31ac959 extmod/nimble: Make nimble_malloc work with allocated size. 2020-09-08 11:41:31 +10:00
Jim Mussared 5b08676d6a extmod/nimble: Set struct alignment correctly on 64-bit arch. 2020-09-08 11:41:31 +10:00
Jim Mussared ed14435a8e extmod/modbluetooth: Refactor stack/hci/driver/port bindings.
Previously the interaction between the different layers of the Bluetooth
stack was different on each port and each stack.  This commit defines
common interfaces between them and implements them for cyw43, btstack,
nimble, stm32, unix.
2020-09-08 11:41:31 +10:00
Jim Mussared e46aac24ba extmod/modbluetooth: Rename logging macro to be just DEBUG_printf.
And prefix the debug message with "btstack:" or "nimble:", depending on the
context.  Also use correct format specifier for %zu.
2020-09-08 10:48:23 +10:00
stijn 40ad8f1666 all: Rename "sys" module to "usys".
This is consistent with the other 'micro' modules and allows implementing
additional features in Python via e.g. micropython-lib's sys.

Note this is a breaking change (not backwards compatible) for ports which
do not enable weak links, as "import sys" must now be replaced with
"import usys".
2020-09-04 00:10:24 +10:00
Damien George 2a72e90ab8 extmod/vfs: Add option to use 1970 as Epoch.
By setting MICROPY_EPOCH_IS_1970 a port can opt to use 1970/1/1 as the
Epoch for timestamps returned by stat().  And this setting is enabled on
the unix and windows ports because that's what they use.

Signed-off-by: Damien George <damien@micropython.org>
2020-09-01 12:36:28 +10:00
Damien George c70e599659 extmod/vfs: Support larger integer range in VFS stat time fields.
On ports like unix where the Epoch is 1970/1/1 and atime/mtime/ctime are in
seconds since the Epoch, this value will overflow a small-int on 32-bit
systems.  So far this is only an issue on 32-bit unix builds that use the
VFS layer (eg dev and coverage unix variants) but the fix (using
mp_obj_new_int_from_uint instead of MP_OBJ_NEW_SMALL_INT) is there for all
ports so as to not complicate the code, and because they will need the
range one day.

Also apply a similar fix to other fields in VfsPosix.stat because they may
also be large.

Signed-off-by: Damien George <damien@micropython.org>
2020-09-01 12:36:28 +10:00
Damien George d1995e50eb extmod/modlwip: Fix error return for TCP recv when not connected.
This commit fixes the cases when a TCP socket is in STATE_NEW,
STATE_LISTENING or STATE_CONNECTING and recv() is called on it.  It now
raises ENOTCONN instead of a random error code due to it previously
indexing beyond the start of error_lookup_table[].

Signed-off-by: Damien George <damien@micropython.org>
2020-08-30 13:20:51 +10:00
Damien George 06659077a8 all: Update Python code to conform to latest black formatting.
Updating to Black v20.8b1 there are two changes that affect the code in
this repository:

- If there is a trailing comma in a list (eg [], () or function call) then
  that list is now written out with one line per element.  So remove such
  trailing commas where the list should stay on one line.

- Spaces at the start of """ doc strings are removed.

Signed-off-by: Damien George <damien@micropython.org>
2020-08-29 15:18:01 +10:00
Michael Buesch cef678b2db extmod/machine_i2c: Fix buffer overrun if 'addrsize' is bigger than 32.
The memory operation functions read_mem() and write_mem() create a
temporary buffer on the local C stack for the address bytes with the size
of 4 bytes.  This buffer is filled in a loop from the user supplied address
and address length.  If the user supplied 'addrsize' is bigger than 32, the
local buffer is overrun.

Fix this by raising an exception for invalid 'addrsize' values.

Signed-off-by: Michael Buesch <m@bues.ch>
2020-08-27 12:39:11 +10:00
Andrew Leech a80a146858 extmod/bluetooth: Support active scanning in BLE.gap_scan().
This adds an additional optional parameter to gap_scan() to select active
scanning, where scan responses are returned as well as normal scan results.
This parameter is False by default which retains the existing behaviour.
2020-08-26 15:00:11 +10:00
Jim Mussared 0bc2c1c105 extmod/modbluetooth: Fix race between READ_REQUEST and other IRQs.
The READ_REQUEST callback is handled as a hard interrupt (because the BLE
stack needs an immediate response from it so it can continue) and so calls
to Python require extra protection:

- the caller-owned tuple passed into the callback must be separate from the
  tuple used by other callback events (which are soft interrupts);

- the GC and scheduler must be locked during callback execution.
2020-08-26 14:57:36 +10:00