CCaches are scoped per-job.
Uses https://github.com/hendrikmuhs/ccache-action to get desired behaviour
(updating the cache on each run).
Signed-off-by: Angus Gratton <angus@redyak.com.au>
Allows splitting the esp32 job into multiple parts without too much
boilerplate. The matrix is parameterised using the name of the function to
call in tools/ci.sh, to minimise the dependency on GitHub Actions.
This can get esp32 build times down around 3m if IDF is cached already.
If the cache is cold, the cache preparation step on each job can double up
against each other. However, restructuring the workflow to not do this
seems either complex or requires copy-pasting the entire cache step.
Signed-off-by: Angus Gratton <angus@redyak.com.au>
Cache is keyed on the ESP-IDF version used in CI, so there shouldn't be any
cache invalidation issues when ESP-IDF version changes.
Restoring from cache takes approx 15s, compared to 2-3m to perform these
steps (ESP-IDF tools install, ESP-IDF clone, ESP-IDF submodule clone) the
first time.
Cache size is approx 1.6GB, the git clone is tweaked as much as possible to
keep the size down.
Signed-off-by: Angus Gratton <angus@redyak.com.au>
Speeds up importing files from mounted filesystem.
Also fix the return code for invalid / unsupported ioctl requests.
Signed-off-by: Andrew Leech <andrew.leech@planetinnovation.com.au>
Remove port-specific test directories and excluded tests from
tinytest-codegen, and let it read said information from an external file.
This way tinytest-codegen is not limited to always generate tests for the
`qemu-arm` target.
This allows having port-specific test directory and excluded tests for more
than one QEMU bare-metal target.
The `qemu-arm` port Makefile was modified to work with the generator
changes and a tests profile file was added to said port.
Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
Requires additions to tools/boardgen.py for stm32 pin generation.
This work was funded through GitHub Sponsors.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
For now, this implements the functionality required for esp32 and rp2,
including support for numeric pins, rp2 alternate functions, and rp2
extended pins.
This also updates the rp2 port to use the same structure for pins.h and
pins.csv as for esp32, and moves the pin definitions directly into the
table (rather than having a table of pointers), which is a small code size
improvement.
Support for "hidden" pins in pins.csv is added (matching the stm32
implementation).
This work was funded through GitHub Sponsors.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
This updates a small number of files that change with ruff-format's (vs
black's) rules.
This work was funded through GitHub Sponsors.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
- Add config for [tool.ruff.format] to pyproject.toml.
- Update pre-commit to run both ruff and ruff-format (and only check C
files when running codeformat.py)
- Update CI.
- Simplify codeformat.py to remove all the Python-specific logic (just run
"ruff format" directly).
This work was funded through GitHub Sponsors.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
This port is largely unmaintained, has limited features (the only hardware
support is for GPIO and timer, and no machine module), only supports a
small number of Teensy boards, and can be confused with the mimxrt support
for Teensy 4.x.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
Signed-off-by: Damien George <damien@micropython.org>
This provides a significant performance boost for qstr_find_strn, which is
called a lot during parsing and loading of .mpy files, as well as interning
of string objects (which happens in most string methods that return new
strings).
Also adds comments to explain the "static" qstrs. These are part of the
.mpy ABI and avoid needing to duplicate string data for QSTRs known to
already be in the firmware. The static pool isn't currently sorted, but in
the future we could either split the static pool into the sorted regions,
or in the next .mpy version just sort them.
Based on initial work done by @amirgon in #6896.
This work was funded through GitHub Sponsors.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
This provides a way to enable features and changes slated for MicroPython
2.x, by running `make MICROPY_PREVIEW_VERSION_2=1`. Also supported for
the cmake ports (except Zephyr).
This is an alternative to having a 2.x development branch (or equivalently,
keeping a 1.x release branch). Any feature or change that needs to be
"hidden" until 2.x can use this flag (either in the Makefile or the
preprocessor).
A good example is changing function arguments or other public API features,
in particular to aid in improving consistency between ports.
When `MICROPY_PREVIEW_VERSION_2` is enabled, the REPL banner is amended to
say "MicroPython (with v2.0 preview) vX.Y.Z", and sys.implementation gets a
new field `_v2` set to `True`.
This work was funded through GitHub Sponsors.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
This commit updates the ci script to automatically fetch all upstream if
the common commit hasn't been found; this should preserve the speed of CI
checks for most PR's, and use a reliable but slow fetch if needed for older
ones.
Signed-off-by: Andrew Leech <andrew.leech@planetinnovation.com.au>
See https://github.com/micropython/micropython/issues/12127 for details.
Previously at the point when a release is made, we update mpconfig.h
and set a git tag. i.e. the version increments at the release.
Now the version increments immediately after the release. The workflow is:
1. Final commit in the cycle updates mpconfig.h to set (X, Y, 0, 0) (i.e.
clear the pre-release state).
2. This commit is tagged "vX.Y.0".
3. First commit for the new cycle updates mpconfig.h to set (X, Y+1, 0, 1)
(i.e. increment the minor version, set the pre-release state).
4. This commit is tagged "vX.Y+1.0-preview".
The idea is that a nightly build is actually a "preview" of the _next_
release. i.e. any documentation describing the current release may not
actually match the nightly build. So we use "preview" as our semver
pre-release identifier.
Changes in this commit:
- Add MICROPY_VERSION_PRERELEASE to mpconfig.h to allow indicating that
this is not a release version.
- Remove unused MICROPY_VERSION integer.
- Append "-preview" to MICROPY_VERSION_STRING when the pre-release state
is set.
- Update py/makeversionhdr.py to no longer generate MICROPY_GIT_HASH.
- Remove the one place MICROPY_GIT_HASH was used (it can use
MICROPY_GIT_TAG instead).
- Update py/makeversionhdr.py to also understand
MICROPY_VERSION_PRERELEASE in mpconfig.h.
- Update py/makeversionhdr.py to convert the git-describe output into
semver-compatible "X.Y.Z-preview.N.gHASH".
- Update autobuild.sh to generate filenames using the new scheme.
- Update remove_old_firmware.py to match new scheme.
- Update mpremote's pyproject.toml to handle the "-preview" suffix in the
tag. setuptools_scm maps to this "rc0" to match PEP440.
- Fix docs heading where it incorrectly said "vvX.Y.Z" for release docs.
This work was funded through GitHub Sponsors.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
Defines the list of standard features and ensures that each board.json
only uses those ones. This list can be extended, but needs to be a
deliberate decision.
This work was funded through GitHub Sponsors.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
Also provide a basic README.md for dynamic native modules.
This work was funded through GitHub Sponsors.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
Because mpy_ld.py doesn't know the target object representation, it emits
instances of `MP_OBJ_NEW_QSTR(MP_QSTR_Foo)` as const string objects, rather
than qstrs. However this doesn't work for map keys (e.g. for a locals dict)
because the map has all_keys_are_qstrs flag is set (and also auto-complete
requires the map keys to be qstrs).
Instead, emit them as regular qstrs, and make a functioning MP_OBJ_NEW_QSTR
function available (via `native_to_obj`, also used for e.g. making
integers).
Remove the code from mpy_ld.py to emit qstrs as constant strings, but leave
behind the scaffold to emit constant objects in case we want to do use this
in the future.
Strictly this should be a .mpy sub-version bump, even though the function
table isn't changing, it does lead to a change in behavior for a new .mpy
running against old MicroPython. `mp_native_to_obj` will incorrectly return
the qstr value directly as an `mp_obj_t`, leading to unexpected results.
But given that it's broken at the moment, it seems unlikely that anyone is
relying on this, so it's not work the other downsides of a sub-version bump
(i.e. breaking pure-Python modules that use @native). The opposite case of
running an old .mpy on new MicroPython is unchanged, and remains broken in
exactly the same way.
This work was funded through GitHub Sponsors.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
Sections sometimes named .rodata.str1.1 etc, instead of just .rodata.
Avoid crashing in that case. Instead treat it like any other RO section.
Fix thanks to @phlash.
Fixes issue #8783.
Signed-off-by: Jon Nordby <jononor@gmail.com>
This allows the cc3200 port to be build with the standard autobuild script
rather than the custom build-cc3200-latest.sh (which is now removed).
This also fixes the path inside the zip file (by using the `-j` flag to
zip).
This work was funded through GitHub Sponsors.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
PICO might not always be a unique name across all ports, and the
convention generally for other boards is to do VENDOR_BOARD.
This work was funded through GitHub Sponsors.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
This merges the existing GENERIC, GENERIC_1M, and GENERIC_512k boards
into variants of the new ESP8266_GENERIC board (renamed from GENERIC so
as not to clash with other ports).
Also moves the generation of the "OTA" variant (previously generated by
autobuild/build-esp8266-latest.sh) into the variant.
Following the convention established for the WEACTSTUDIO rp2 board, the
names of the variants are FLASH_1M and FLASH_512K (but rename the .ld files
to use MiB and kiB).
Updates autobuild to build esp8266 firmware the same way as other ports.
This requires renaming the output from firmware-combined.bin to just
firmware.bin.
This work was funded through GitHub Sponsors.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
Board names need to be unique across ports, and GENERIC clashes with
the ESP8266 (which will be renamed to ESP8266_GENERIC).
This work was funded through GitHub Sponsors.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
This is to support a future change to add the variant name to the build
directory and therefore should be the same style as the board name.
This work was funded through GitHub Sponsors.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
Spurious fix as the logic is structured such that these variables will be
set before dereferenced, but this keeps Ruff happy (no more F821
undefined-name).
Signed-off-by: Angus Gratton <angus@redyak.com.au>
Otherwise `mpremote soft-reset` will implicitly run the repl command.
Fixes issue #10871.
This work was funded through GitHub Sponsors.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
Ruff version 283 expanded E721 to fail when making direct comparison
against a built-in type. Change the code to use isinstance() as
suggested, these usages appear to have equivalent functionality.
Signed-off-by: Angus Gratton <angus@redyak.com.au>
On esp32, the build output consists of:
- micropython.elf
- micropython.map
- micropython.bin -- application only
- micropython.uf2 -- application only
- firmware.bin -- bootloader, partition table and application
Currently everything is available at the download page except
micropython.bin. This commit adds that file but with the extension changed
to .app-bin, to distinguish it from .bin (the full thing).
Signed-off-by: Damien George <damien@micropython.org>
This will be replaced with a new deflate module providing the same
functionality, with an optional frozen Python wrapper providing a
replacement zlib module.
binascii.crc32 is temporarily disabled.
This work was funded through GitHub Sponsors.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
This change enables the ULP (FSM) for all ESP32 variants rather than
requiring it to be enabled for each board specifically.
It also ensures the correct header file is included for each variant.
Lastly, it updates the IDF version we're builing against to v4.4.2, as that
version contains important fixes to make the ULP actually work on S2/S3
chips. See: https://github.com/espressif/esp-idf/commit/a0e3d48
Signed-off-by: Wilko Nienhaus <wilko.nienhaus@gmail.com>
The automatic code size build and GitHub comment is a really useful
feature. This commit adds a few more builds to it (mimxrt and samd).
Signed-off-by: Damien George <damien@micropython.org>
Damien George
Angus Gratton
Andrew Leech
Alessandro Gatti
Jim Mussared
Jos Verlinde
iabdalkader
Jon Nordby
Daniël van de Giessen
Wilko Nienhaus