Commit Graph

282 Commits

Author SHA1 Message Date
Thorsten von Eicken 3e758ef235 lib/mbedtls_errors: Add code to patch mbedtls for shortened error strs.
The file `mbedtls_errors/mp_mbedtls_errors.c` can be used instead of
`mbedtls/library/error.c` to give shorter error strings, reducing the build
size of the error strings from about 12-16kB down to about 2-5kB.
2020-07-20 23:53:27 +10:00
Andrew Leech 07f181a216 Revert "tools/pydfu.py: Respect longer timeouts requested by DFU dev..."
This reverts commit 4d6f60d428.

This implementation used the timeout as a maximum amount of time needed for
the operation, when actually the spec and other tools suggest that it's the
minumum delay needed between subsequent USB transfers.
2020-07-01 16:54:03 +10:00
Andrew Leech 4d6f60d428 tools/pydfu.py: Respect longer timeouts requested by DFU device/mboot. 2020-06-30 21:22:00 +10:00
Damien George 76faeed098 tools/makemanifest.py: Support freezing a subdirectory recursively.
This adds support for freezing an entire directory while keeping the
directory as part of the import path.  For example

    freeze("path/to/library", "module")

will recursively freeze all scripts in "path/to/library/module" and have
them importable as "from module import ...".

Signed-off-by: Damien George <damien@micropython.org>
2020-06-24 12:21:30 +10:00
David Lechner 77ed6f69ac tools/uncrustify: Enable more opts to remove space between func and '('.
With only `sp_func_proto_paren = remove` set there are some cases where
uncrustify misses removing a space between the function name and the
opening '('.  This sets all of the related options to `force` as well.
2020-06-19 22:07:32 +10:00
Damien George 026fda605e tools/codeformat.py: Include extmod/{btstack,nimble} in code formatting.
Signed-off-by: Damien George <damien@micropython.org>
2020-06-18 22:20:20 +10:00
David Lechner b4d0d7bf03 tools/uncrustify: Update config for v0.71.0.
This is the result of running...

    uncrustify -c tools/uncrustify.cfg --update-config-with-doc -o tools/uncrustify.cfg

...with some manual fixups to correct places where it changed things it
should not have.

Essentially it just adds new config parameters introduced in v0.71.0
with their default values.

Signed-off-by: David Lechner <david@pybricks.com>
2020-06-14 11:08:05 -05:00
David Lechner ecd7826316 tools/codeformat.py: Remove sizeof fixup.
Formatting for `* sizeof` was fixed in uncrustify v0.71, so we no longer
need the fixups for it.  Also, there was one file where the updated
uncrustify caught a problem that the regex didn't pick up, which is updated
in this commit.

Signed-off-by: David Lechner <david@pybricks.com>
2020-06-14 20:24:18 +10:00
David Lechner bd7c92e17d tools/uncrustify.cfg: Remove deprecated sp_word_brace option.
This option was removed in uncrustify v0.71.

Signed-off-by: David Lechner <david@pybricks.com>
2020-06-14 20:24:03 +10:00
David Lechner 8f642677f7 tools/codeformat.py: Add verbose option to pass to uncrustify and black.
This adds a new command line option `-v` to `tools/codeformat.py` to enable
verbose printing of all files that are scanned.

Normally `uncrustify` and `black` are called with the `-q` option so
setting verbose suppresses the `-q` option and on `black` also enables the
`-v` option which makes it print out all file names matching the filter
similar to how `uncrustify` does by default.
2020-05-29 22:59:56 +10:00
David Lechner 2d1fef7096 tools/codeformat.py: Use -q option on uncrustify to make output quiet.
This suppresses the Parsing: <file> as language C lines.  This makes
parsing run a bit faster and on CI it makes for less scrolling through logs
(and black already uses the -q option).
2020-05-28 10:08:38 +10:00
stijn 84fa3312cf all: Format code to add space after C++-style comment start.
Note: the uncrustify configuration is explicitly set to 'add' instead of
'force' in order not to alter the comments which use extra spaces after //
as a means of indenting text for clarity.
2020-04-23 11:24:25 +10:00
Damien George 6e2871df59 tools/check_code_size.sh: Remove unused script.
It's now replaced by tools/metrics.py.
2020-04-22 14:07:56 +10:00
Damien George 17dc86369f tools/metrics.py: Use OrderedDict when reading build log.
So that the output (eg of the diff command) always has the lines in the
same order.
2020-04-22 14:03:26 +10:00
Damien George 1cc24cd39a tools/metrics.py: Don't build mpy-cross if not needed by any ports.
To save build time.
2020-04-22 14:03:07 +10:00
Damien George a4423570e2 tools/metrics.py: Add option to diff to error if delta above threshold.
Useful for things like CI where the size check is automated.
2020-04-22 14:02:14 +10:00
stijn bcf01d1686 all: Fix implicit conversion from double to float.
These are found when building with -Wfloat-conversion.
2020-04-18 22:42:24 +10:00
Jim Mussared 85858e72df py/objexcept: Allow compression of exception message text.
The decompression of error-strings is only done if the string is accessed
via printing or via er.args.  Tests are added for this feature to ensure
the decompression works.
2020-04-05 15:02:06 +10:00
David Lechner 3dc324d3f1 tests: Format all Python code with black, except tests in basics subdir.
This adds the Python files in the tests/ directory to be formatted with
./tools/codeformat.py.  The basics/ subdirectory is excluded for now so we
aren't changing too much at once.

In a few places `# fmt: off`/`# fmt: on` was used where the code had
special formatting for readability or where the test was actually testing
the specific formatting.
2020-03-30 13:21:58 +11:00
Lars Kellogg-Stedman 3a0f64fc7a tools/pyboard.py: Add -d as an alias for --device. 2020-03-30 11:37:32 +11:00
Lars Kellogg-Stedman 1cf994c48b tools/pyboard.py: Support setting device/baudrate from shell env vars.
Allow defaults for --device and --baudrate to be set in the environment
using PYBOARD_DEVICE and PYBOARD_BAUDRATE.
2020-03-30 11:37:32 +11:00
Damien George 1a3e386c67 all: Remove spaces inside and around parenthesis.
Using new options enabled in the uncrustify configuration.
2020-03-28 23:36:44 +11:00
Damien George feb2577585 all: Remove spaces between nested paren and inside function arg paren.
Using new options enabled in the uncrustify configuration.
2020-03-25 00:39:46 +11:00
stijn 1b3e0e10b9 tools/codeformat.py: Include all msvc C code in auto-format. 2020-03-25 00:36:42 +11:00
Andrew Leech 38ccb4c643 tools/pydfu.py: Display any error strings from device/mboot. 2020-03-22 15:24:57 +11:00
Andrew Leech 2966d83a65 tools/pydfu.py: Add args for VID/PID & exit with cleaner error handling. 2020-03-22 13:33:04 +11:00
David Lechner 8a4ce6b79a tools/codeformat.py: Eliminate need for sizeof fixup.
This eliminates the need for the sizeof regex fixup by rearranging things a
bit.  All other bitfields already use the parentheses around expressions
with sizeof, so one case is fixed by following this convention.

VM_MAX_STATE_ON_STACK is the only remaining problem and it can be worked
around by changing the order of the operands.
2020-03-11 14:34:40 +11:00
Damien George 554c01fc25 tools/metrics.py: Use check_call instead of run to error out on error. 2020-03-11 14:23:39 +11:00
MikeTeachman 8db5d2d1f1 tools/makemanifest.py: Fix build on Windows by adding .exe to mpy-cross.
When using a manifest on Windows the reference to mpy-cross compiler was
missing the .exe file extension, so add it when appropriate.

Also allow the default path to mpy-cross to be overridden by the (optional)
MICROPY_MPYCROSS environment variable, to allow full flexibility on any OS.
2020-03-05 10:26:31 +11:00
Damien George 69661f3343 all: Reformat C and Python source code with tools/codeformat.py.
This is run with uncrustify 0.70.1, and black 19.10b0.
2020-02-28 10:33:03 +11:00
Damien George 4b23e98fb0 tools/codeformat.py: Add formatter using uncrustify for C, black for Py.
This commit adds a tool, codeformat.py, which will reformat C and Python
code to fit a certain style.  By default the tool will reformat (almost)
all the original (ie not 3rd-party) .c, .h and .py files in this
repository.  Passing filenames on the command-line to codeformat.py will
reformat only those.  Reformatting is done in-place.

uncrustify is used for C reformatting, which is available for many
platforms and can be easily built from source, see
https://github.com/uncrustify/uncrustify.  The configuration for uncrustify
is also added in this commit and values are chosen to best match the
existing code style.  A small post-processing stage on .c and .h files is
done by codeformat.py (after running uncrustify) to fix up some minor
items:
- space inserted after * when used as multiplication with sizeof
- #if/ifdef/ifndef/elif/else/endif are dedented by one level when they are
  configuring if-blocks and case-blocks.

For Python code, the formatter used is black, which can be pip-installed;
see https://github.com/psf/black.  The defaults are used, except for line-
length which is set at 99 characters to match the "about 100" line-length
limit used in C code.

The formatting tools used and their configuration were chosen to strike a
balance between keeping existing style and not changing too many lines of
code, and enforcing a relatively strict style (especially for Python code).
This should help to keep the code consistent across everything, and reduce
cognitive load when writing new code to match the style.
2020-02-28 10:14:28 +11:00
Damien George a636837987 tools/makemanifest.py: Support freezing with empty list of mpy files.
Fixes issue #5655.
2020-02-20 10:41:50 +11:00
Michael Buesch 1604606238 tools/pyboard.py: Change shebang to use python3.
This script still works with Python 2 but Python 3 is recommended.
2020-02-01 00:06:26 +11:00
Michael Buesch 1cadb12d1c tools/pyboard.py: Use slice del instead of list.clear() for Py2 compat.
Python 2 does not have list.clear().
2020-02-01 00:05:29 +11:00
Michael Buesch 83afd48ad9 tools/pyboard.py: Add option --no-follow to detach after sending script.
This option makes pyboard.py exit as soon as the script/command is
successfully sent to the device, ie it does not wait for any output.  This
can help to avoid hangs if the board is being rebooted with --comman (for
example).

Example usage:

    $ python3 ./tools/pyboard.py --device /dev/ttyUSB0 --no-follow \
        --command 'import machine; machine.reset()'
2020-02-01 00:03:37 +11:00
c0rejump 6db5cede06 tools/pydfu.py: Clean up syntax, update comments and docstrings.
Some parts of code have been aligned to increase readability.  In general
'' instead of "" were used wherever possible to keep the same convention
for entire file.  Import inspect line has been moved to the top according
to hints reported by pep8 tools.  A few extra spaces were removed, a few
missing spaces were added.  Comments have been updated, mostly in
"read_dfu_file" function.  Some other comments have been capitalized and/or
slightly updated.  A few docstrings were fixed as well.  No real code
changes intended.
2020-01-23 13:23:12 +11:00
Damien George a11e306227 tools: Add metrics.py script to build and compute port sizes/metrics. 2020-01-23 12:48:52 +11:00
Jason Neal de78a9e317 tools/gen-cpydiff.py: Adjust subsections to sentence case. 2020-01-06 22:16:18 +11:00
David Lechner fd0ba7be07 tools/tinytest-codegen.py: Add extra newline and result message.
This is an alternative to f4ed2df that adds a newline so that the output of
the test starts on a new line and the result of the test is prefixed with
"result: " to distinguish it from the test output.

Suggested-by: @dpgeorge
2019-12-19 17:55:50 +11:00
Damien George b3b9b11596 tools/pyboard.py: Support executing .mpy files directly.
This patch allows executing .mpy files (including native ones) directly on
a target, eg a board over a serial connection.  So there's no need to copy
the file to its filesystem to test it.

For example:

    $ mpy-cross foo.py
    $ pyboard.py foo.mpy
2019-12-19 17:00:52 +11:00
Damien George 0bd7d1f7f0 py/persistentcode: Move loading of rodata/bss to before obj/raw-code.
This makes the loading of viper-code-with-relocations a bit neater and
easier to understand, by treating the rodata/bss like a special object to
be loaded into the constant table (which is how it behaves).
2019-12-17 13:22:11 +11:00
Damien George fc97d6d1b5 tools/mpy-tool.py: Raise exception if trying to freeze relocatable mpy. 2019-12-12 20:15:28 +11:00
Damien George abc642973d py/dynruntime: Add support for float API to make/get floats.
We don't want to add a feature flag to .mpy files that indicate float
support because it will get complex and difficult to use.  Instead the .mpy
is built using whatever precision it chooses (float or double) and the
native glue API will convert between this choice and what the host runtime
actually uses.
2019-12-12 20:15:28 +11:00
Damien George aad79adab7 tools/mpy_ld.py: Add new mpy_ld.py tool and associated build files.
This commit adds a new tool called mpy_ld.py which is essentially a linker
that builds .mpy files directly from .o files.  A new header file
(dynruntime.h) and makefile fragment (dynruntime.mk) are also included
which allow building .mpy files from C source code.  Such .mpy files can
then be dynamically imported as though they were a normal Python module,
even though they are implemented in C.

Converting .o files directly (rather than pre-linked .elf files) allows the
resulting .mpy to be more efficient because it has more control over the
relocations; for example it can skip PLT indirection.  Doing it this way
also allows supporting more architectures, such as Xtensa which has
specific needs for position-independent code and the GOT.

The tool supports targets of x86, x86-64, ARM Thumb and Xtensa (windowed
and non-windowed).  BSS, text and rodata sections are supported, with
relocations to all internal sections and symbols, as well as relocations to
some external symbols (defined by dynruntime.h), and linking of qstrs.
2019-12-12 20:15:28 +11:00
Damien George 27879844d2 tools/mpy-tool.py: Add ability to merge multiple .mpy files into one.
Usage:

    mpy-tool.py -o merged.mpy --merge mod1.mpy mod2.mpy

The constituent .mpy files are executed sequentially when the merged file
is imported, and they all use the same global namespace.
2019-12-12 20:15:28 +11:00
Damien George 360d972c16 py/nativeglue: Add new header file with native function table typedef. 2019-12-12 20:15:28 +11:00
Damien George 7f24c29778 tools/mpy-tool.py: Support qstr linking when freezing Xtensa native mpy. 2019-11-28 13:11:51 +11:00
Yonatan Goldschmidt b2dd443d92 tools/makemanifest.py: Use sys.executable when invoking Python scripts.
So the version of Python used to run makemanifest.py is also used for the
sub-scripts.
2019-11-13 13:44:19 +11:00
Andrew Leech d2e6cfd8fd tools/makemanifest.py: Skip freezing unsupported files with warning. 2019-11-07 12:34:57 +11:00
Mirko Vogt 2f71d66ef7 tools/makemanifest.py: Follow symlinks when freezing linked directories.
While the new manifest.py style got introduced for freezing python code
into the resulting binary, the old way - where files and modules within
ports/*/modules where baked into the resulting binary - was still
supported via `freeze('$(PORT_DIR)/modules')` within manifest.py.

However behaviour changed for symlinked directories (=modules), as those
links weren't followed anymore.

This commit restores the original behaviour by explicitly following
symlinks within a modules/ directory
2019-11-06 11:41:06 +11:00