Commit Graph

635 Commits

Author SHA1 Message Date
Jim Mussared 724ebb9916 stm32/boards/make-pins.py: Update to use tools/boardgen.py.
Requires additions to tools/boardgen.py for stm32 pin generation.

This work was funded through GitHub Sponsors.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-11-03 14:16:49 +11:00
Jim Mussared 4bd6ec9ae4 tools/boardgen.py: Add initial implementation of a common make-pins.py.
For now, this implements the functionality required for esp32 and rp2,
including support for numeric pins, rp2 alternate functions, and rp2
extended pins.

This also updates the rp2 port to use the same structure for pins.h and
pins.csv as for esp32, and moves the pin definitions directly into the
table (rather than having a table of pointers), which is a small code size
improvement.

Support for "hidden" pins in pins.csv is added (matching the stm32
implementation).

This work was funded through GitHub Sponsors.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-11-03 14:06:27 +11:00
Jim Mussared bbd8760bd9 all: Update Python formatting to ruff-format.
This updates a small number of files that change with ruff-format's (vs
black's) rules.

This work was funded through GitHub Sponsors.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-11-03 13:30:41 +11:00
Jim Mussared 303ccca7c6 all: Replace "black" with "ruff format".
- Add config for [tool.ruff.format] to pyproject.toml.
- Update pre-commit to run both ruff and ruff-format (and only check C
  files when running codeformat.py)
- Update CI.
- Simplify codeformat.py to remove all the Python-specific logic (just run
  "ruff format" directly).

This work was funded through GitHub Sponsors.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-11-03 13:30:38 +11:00
Jim Mussared e579ebb11b teensy: Remove the teensy port.
This port is largely unmaintained, has limited features (the only hardware
support is for GPIO and timer, and no machine module), only supports a
small number of Teensy boards, and can be confused with the mimxrt support
for Teensy 4.x.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
Signed-off-by: Damien George <damien@micropython.org>
2023-10-31 13:04:41 +11:00
Jim Mussared 64c79a5423 py/qstr: Add support for sorted qstr pools.
This provides a significant performance boost for qstr_find_strn, which is
called a lot during parsing and loading of .mpy files, as well as interning
of string objects (which happens in most string methods that return new
strings).

Also adds comments to explain the "static" qstrs.  These are part of the
.mpy ABI and avoid needing to duplicate string data for QSTRs known to
already be in the firmware.  The static pool isn't currently sorted, but in
the future we could either split the static pool into the sorted regions,
or in the next .mpy version just sort them.

Based on initial work done by @amirgon in #6896.

This work was funded through GitHub Sponsors.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-10-30 11:10:02 +11:00
Jim Mussared 3bf70f16e9 py/mkrules.mk: Add MICROPY_PREVIEW_VERSION_2.
This provides a way to enable features and changes slated for MicroPython
2.x, by running `make MICROPY_PREVIEW_VERSION_2=1`. Also supported for
the cmake ports (except Zephyr).

This is an alternative to having a 2.x development branch (or equivalently,
keeping a 1.x release branch). Any feature or change that needs to be
"hidden" until 2.x can use this flag (either in the Makefile or the
preprocessor).

A good example is changing function arguments or other public API features,
in particular to aid in improving consistency between ports.

When `MICROPY_PREVIEW_VERSION_2` is enabled, the REPL banner is amended to
say "MicroPython (with v2.0 preview) vX.Y.Z", and sys.implementation gets a
new field `_v2` set to `True`.

This work was funded through GitHub Sponsors.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-10-27 15:28:46 +11:00
Damien George 6967ff3c58 py/persistentcode: Bump .mpy sub-version.
This is required because the previous commit changed the .mpy native ABI.

Signed-off-by: Damien George <damien@micropython.org>
2023-10-16 11:25:31 +11:00
Andrew Leech d2a9d70c09 tools/ci.sh: Ensure enough commits are fetched for a common ancestor.
This commit updates the ci script to automatically fetch all upstream if
the common commit hasn't been found; this should preserve the speed of CI
checks for most PR's, and use a reliable but slow fetch if needed for older
ones.

Signed-off-by: Andrew Leech <andrew.leech@planetinnovation.com.au>
2023-10-11 14:05:14 +11:00
Jim Mussared 69e34b6b6b all: Switch to new preview build versioning scheme.
See https://github.com/micropython/micropython/issues/12127 for details.

Previously at the point when a release is made, we update mpconfig.h
and set a git tag. i.e. the version increments at the release.

Now the version increments immediately after the release. The workflow is:
1. Final commit in the cycle updates mpconfig.h to set (X, Y, 0, 0) (i.e.
   clear the pre-release state).
2. This commit is tagged "vX.Y.0".
3. First commit for the new cycle updates mpconfig.h to set (X, Y+1, 0, 1)
   (i.e. increment the minor version, set the pre-release state).
4. This commit is tagged "vX.Y+1.0-preview".

The idea is that a nightly build is actually a "preview" of the _next_
release. i.e. any documentation describing the current release may not
actually match the nightly build. So we use "preview" as our semver
pre-release identifier.

Changes in this commit:
 - Add MICROPY_VERSION_PRERELEASE to mpconfig.h to allow indicating that
   this is not a release version.
 - Remove unused MICROPY_VERSION integer.
 - Append "-preview" to MICROPY_VERSION_STRING when the pre-release state
   is set.
 - Update py/makeversionhdr.py to no longer generate MICROPY_GIT_HASH.
 - Remove the one place MICROPY_GIT_HASH was used (it can use
   MICROPY_GIT_TAG instead).
 - Update py/makeversionhdr.py to also understand
   MICROPY_VERSION_PRERELEASE in mpconfig.h.
 - Update py/makeversionhdr.py to convert the git-describe output into
   semver-compatible "X.Y.Z-preview.N.gHASH".
 - Update autobuild.sh to generate filenames using the new scheme.
 - Update remove_old_firmware.py to match new scheme.
 - Update mpremote's pyproject.toml to handle the "-preview" suffix in the
   tag. setuptools_scm maps to this "rc0" to match PEP440.
 - Fix docs heading where it incorrectly said "vvX.Y.Z" for release docs.

This work was funded through GitHub Sponsors.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-10-06 12:10:14 +11:00
Damien George 040a96d00e docs,tools: Change remaining "urequests" references to "requests".
Signed-off-by: Damien George <damien@micropython.org>
2023-10-05 14:04:45 +11:00
Damien George 92717a95c0 tools/metrics.py: Fix esp32 and esp8266 board names after renaming.
Signed-off-by: Damien George <damien@micropython.org>
2023-10-04 15:44:25 +11:00
Damien George cf490a7091 all: Fix various spelling mistakes found by codespell 2.2.6.
Signed-off-by: Damien George <damien@micropython.org>
2023-10-03 11:24:50 +11:00
Jos Verlinde 58c2c503a9 tools/mpremote: Add support for rfc2217, serial over TCP.
Signed-off-by: Jos Verlinde <jos_verlinde@hotmail.com>
2023-09-29 14:40:17 +10:00
Jim Mussared 88ecc78eb3 tools/autobuild/build-downloads.py: Verify standard features.
Defines the list of standard features and ensures that each board.json
only uses those ones. This list can be extended, but needs to be a
deliberate decision.

This work was funded through GitHub Sponsors.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-09-29 14:18:33 +10:00
Damien George b0e03b3e07 tools/autobuild: Include .bin firmware in renesas-ra build output.
Signed-off-by: Damien George <damien@micropython.org>
2023-09-18 11:05:14 +10:00
iabdalkader 174bb28d8e tools/ci.sh: Add ARDUINO_PORTENTA_C33 to RA CI build.
Signed-off-by: iabdalkader <i.abdalkader@gmail.com>
2023-09-15 00:10:52 +10:00
Damien George 6aa404ca53 tools/metrics.py: Fix nrf and rp2 board names after renaming.
Signed-off-by: Damien George <damien@micropython.org>
2023-09-02 00:37:45 +10:00
Jim Mussared 607548f32d examples/natmod: Add features4 as a class definition example.
Also provide a basic README.md for dynamic native modules.

This work was funded through GitHub Sponsors.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-09-02 00:16:16 +10:00
Jim Mussared a64f2fdca0 py/dynruntime.h: Implement MP_OBJ_NEW_QSTR.
Because mpy_ld.py doesn't know the target object representation, it emits
instances of `MP_OBJ_NEW_QSTR(MP_QSTR_Foo)` as const string objects, rather
than qstrs. However this doesn't work for map keys (e.g. for a locals dict)
because the map has all_keys_are_qstrs flag is set (and also auto-complete
requires the map keys to be qstrs).

Instead, emit them as regular qstrs, and make a functioning MP_OBJ_NEW_QSTR
function available (via `native_to_obj`, also used for e.g. making
integers).

Remove the code from mpy_ld.py to emit qstrs as constant strings, but leave
behind the scaffold to emit constant objects in case we want to do use this
in the future.

Strictly this should be a .mpy sub-version bump, even though the function
table isn't changing, it does lead to a change in behavior for a new .mpy
running against old MicroPython. `mp_native_to_obj` will incorrectly return
the qstr value directly as an `mp_obj_t`, leading to unexpected results.
But given that it's broken at the moment, it seems unlikely that anyone is
relying on this, so it's not work the other downsides of a sub-version bump
(i.e. breaking pure-Python modules that use @native). The opposite case of
running an old .mpy on new MicroPython is unchanged, and remains broken in
exactly the same way.

This work was funded through GitHub Sponsors.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-09-01 16:14:22 +10:00
Jon Nordby 4837ec336a tools/mpy_ld.py: Support more complex rodata sections.
Sections sometimes named .rodata.str1.1 etc, instead of just .rodata.
Avoid crashing in that case.  Instead treat it like any other RO section.

Fix thanks to @phlash.

Fixes issue #8783.

Signed-off-by: Jon Nordby <jononor@gmail.com>
2023-09-01 15:30:21 +10:00
Jim Mussared bc1af5ab7f cc3200/Makefile: Build firmware.zip.
This allows the cc3200 port to be build with the standard autobuild script
rather than the custom build-cc3200-latest.sh (which is now removed).

This also fixes the path inside the zip file (by using the `-j` flag to
zip).

This work was funded through GitHub Sponsors.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-08-24 15:44:20 +10:00
Jim Mussared 873bd00ad4 rp2: Rename PICO, PICO_W to RPI_PICO, RPI_PICO_W.
PICO might not always be a unique name across all ports, and the
convention generally for other boards is to do VENDOR_BOARD.

This work was funded through GitHub Sponsors.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-08-23 15:49:37 +10:00
Jim Mussared ef03ca8bf2 esp8266: Add board variant support.
This merges the existing GENERIC, GENERIC_1M, and GENERIC_512k boards
into variants of the new ESP8266_GENERIC board (renamed from GENERIC so
as not to clash with other ports).

Also moves the generation of the "OTA" variant (previously generated by
autobuild/build-esp8266-latest.sh) into the variant.

Following the convention established for the WEACTSTUDIO rp2 board, the
names of the variants are FLASH_1M and FLASH_512K (but rename the .ld files
to use MiB and kiB).

Updates autobuild to build esp8266 firmware the same way as other ports.
This requires renaming the output from firmware-combined.bin to just
firmware.bin.

This work was funded through GitHub Sponsors.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-08-23 15:49:37 +10:00
Jim Mussared aa23698119 esp32: Rename GENERIC* boards to ESP32_GENERIC*.
Board names need to be unique across ports, and GENERIC clashes with
the ESP8266 (which will be renamed to ESP8266_GENERIC).

This work was funded through GitHub Sponsors.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-08-23 13:54:37 +10:00
Jim Mussared b5836e7252 esp32: Use uppercase variant names.
This is to support a future change to add the variant name to the build
directory and therefore should be the same style as the board name.

This work was funded through GitHub Sponsors.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-08-23 13:54:37 +10:00
Jim Mussared f007d0aa3d nrf/boards: Rename all nRF boards to use uppercase.
This is to match all the other ports.

This work was funded through GitHub Sponsors.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-08-23 13:54:37 +10:00
Angus Gratton 1a5c9b9da4 tools/mpy-tool.py: Ignore linter failure in Python 2 compatibility code.
Found by Ruff checking F821.

Signed-off-by: Angus Gratton <angus@redyak.com.au>
2023-08-16 16:16:44 +10:00
Angus Gratton b8189d039d tools/mpy_ld.py: Pre-declare some local variables to appease linter.
Spurious fix as the logic is structured such that these variables will be
set before dereferenced, but this keeps Ruff happy (no more F821
undefined-name).

Signed-off-by: Angus Gratton <angus@redyak.com.au>
2023-08-16 16:16:44 +10:00
Jim Mussared 91674c41b8 tools/autobuild: Automatically build all variants for each board.
Removes the special-case for stm32.

This work was funded through GitHub Sponsors.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-08-15 17:38:02 +10:00
Jim Mussared 4bbe879c26 tools/mpremote: Make soft-reset count as an action.
Otherwise `mpremote soft-reset` will implicitly run the repl command.

Fixes issue #10871.

This work was funded through GitHub Sponsors.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-08-10 14:11:53 +10:00
Jos Verlinde 232c29c2ed tools/codeformat.py: Use pyproject.toml for black config.
Signed-off-by: Jos Verlinde <jos_verlinde@hotmail.com>
2023-08-10 11:54:36 +10:00
Daniël van de Giessen 8dbb29da3f tools/codeformat.py: Skip formatting ESP-IDF managed components.
Signed-off-by: Daniël van de Giessen <daniel@dvdgiessen.nl>
2023-08-10 11:36:24 +10:00
Angus Gratton 597fcb4751 tools/mpy-tool.py: Use isinstance() for type checking.
Ruff version 283 expanded E721 to fail when making direct comparison
against a built-in type.  Change the code to use isinstance() as
suggested, these usages appear to have equivalent functionality.

Signed-off-by: Angus Gratton <angus@redyak.com.au>
2023-08-09 17:41:54 +10:00
Damien George d14ddcbdb5 tools/autobuild: Add support for application .bin files for esp32.
On esp32, the build output consists of:
- micropython.elf
- micropython.map
- micropython.bin -- application only
- micropython.uf2 -- application only
- firmware.bin -- bootloader, partition table and application

Currently everything is available at the download page except
micropython.bin.  This commit adds that file but with the extension changed
to .app-bin, to distinguish it from .bin (the full thing).

Signed-off-by: Damien George <damien@micropython.org>
2023-07-27 13:10:19 +10:00
Jim Mussared ea1a5e43d0 examples/natmod/deflate: Add deflate as a dynamic native module.
This replaces the previous zlib version.

This work was funded through GitHub Sponsors.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-07-21 19:32:42 +10:00
Jim Mussared add1200343 all: Remove the zlib module.
This will be replaced with a new deflate module providing the same
functionality, with an optional frozen Python wrapper providing a
replacement zlib module.

binascii.crc32 is temporarily disabled.

This work was funded through GitHub Sponsors.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-07-21 18:48:29 +10:00
Damien George 273ba28c07 tools/autobuild: Update auto-build code to build esp32 port with IDF v5.
Signed-off-by: Damien George <damien@micropython.org>
2023-06-23 15:53:04 +10:00
Damien George 6a9db521ed github/workflows: Update esp32 CI to use IDF v5.0.
Signed-off-by: Damien George <damien@micropython.org>
2023-06-23 15:34:53 +10:00
Damien George 0908d00453 tools/ci.sh: Add a H5 board to stm32 CI build.
Signed-off-by: Damien George <damien@micropython.org>
2023-06-15 11:09:37 +10:00
Damien George 33b403dfb4 tools/ci.sh: Build PICO_W board as part of rp2 CI.
Signed-off-by: Damien George <damien@micropython.org>
2023-06-14 22:20:20 +10:00
Wilko Nienhaus 034502bc72 esp32/esp32_ulp: Fix ULP (FSM) support for S2 and S3.
This change enables the ULP (FSM) for all ESP32 variants rather than
requiring it to be enabled for each board specifically.

It also ensures the correct header file is included for each variant.

Lastly, it updates the IDF version we're builing against to v4.4.2, as that
version contains important fixes to make the ULP actually work on S2/S3
chips. See: https://github.com/espressif/esp-idf/commit/a0e3d48

Signed-off-by: Wilko Nienhaus <wilko.nienhaus@gmail.com>
2023-06-14 16:41:03 +10:00
Jim Mussared b4de697ad1 tools/mpremote: Fix exec_ -> exec in commands.py.
This was missed in the pyboard refactor and is preventing `cp -r` from
working.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-06-09 13:36:12 +10:00
Damien George 0832fc53ce tools/ci.sh: Add mimxrt and samd ports to code size build.
The automatic code size build and GitHub comment is a really useful
feature.  This commit adds a few more builds to it (mimxrt and samd).

Signed-off-by: Damien George <damien@micropython.org>
2023-06-08 22:47:03 +10:00
Jim Mussared a1fbb1980c extmod/modtimeq: Remove timeq module.
This is a MicroPython-specific module that existed to support the old
version of uasyncio.  It's undocumented and not enabled on all ports and
takes up code size unnecessarily.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-06-08 17:54:28 +10:00
Jim Mussared 5fd042e7d1 all: Replace all uses of umodule in Python code.
Applies to drivers/examples/extmod/port-modules/tools.

This work was funded through GitHub Sponsors.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-06-08 17:54:24 +10:00
Jim Mussared 4216bc7d13 tests: Replace umodule with module everywhere.
This work was funded through GitHub Sponsors.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-06-08 17:54:24 +10:00
Jim Mussared eb85f4d4c9 examples/natmod: Rename umodule to module.
This work was funded through GitHub Sponsors.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-06-08 17:54:24 +10:00
Jim Mussared 45ac651d1a all: Rename *umodule*.c to remove the "u" prefix.
Updates any includes, and references from Makefiles/CMake.

This essentially reverts what was done long ago in commit
136b5cbd76

This work was funded through GitHub Sponsors.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-06-08 17:54:17 +10:00
Jim Mussared f5f9edf645 all: Rename UMODULE to MODULE in preprocessor/Makefile vars.
This work was funded through GitHub Sponsors.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-06-08 17:54:11 +10:00