Use C macros to reduce the size of firmware images when the GC split-heap
feature is disabled.
The code size difference of this commit versus HEAD~2 (ie the commit prior
to MICROPY_GC_SPLIT_HEAP being introduced) when split-heap is disabled is:
bare-arm: +0 +0.000%
minimal x86: +0 +0.000%
unix x64: -16 -0.003%
unix nanbox: -20 -0.004%
stm32: -8 -0.002% PYBV10
cc3200: +0 +0.000%
esp8266: +8 +0.001% GENERIC
esp32: +0 +0.000% GENERIC
nrf: -20 -0.011% pca10040
rp2: +0 +0.000% PICO
samd: -4 -0.003% ADAFRUIT_ITSYBITSY_M4_EXPRESS
The code size difference of this commit versus HEAD~2 split-heap is enabled
with MICROPY_GC_MULTIHEAP=1 (but no extra code to add more heaps):
unix x64: +1032 +0.197% [incl +544(bss)]
esp32: +592 +0.039% GENERIC[incl +16(data) +264(bss)]
This commit adds a new option MICROPY_GC_SPLIT_HEAP (disabled by default)
which, when enabled, allows the GC heap to be split over multiple memory
areas/regions. The first area is added with gc_init() and subsequent areas
can be added with gc_add(). New areas can be added at runtime. Areas are
stored internally as a linked list, and calls to gc_alloc() can be
satisfied from any area.
This feature has the following use-cases (among others):
- The ESP32 has a fragmented OS heap, so to use all (or more) of it the
GC heap must be split.
- Other MCUs may have disjoint RAM regions and are now able to use them
all for the GC heap.
- The user could explicitly increase the size of the GC heap.
- Support a dynamic heap while running on an OS, adding more heap when
necessary.
This uses MP_REGISTER_ROOT_POINTER() to register sched_queue
instead of using a conditional inside of mp_state_vm_t.
Signed-off-by: David Lechner <david@pybricks.com>
This uses MP_REGISTER_ROOT_POINTER() to register cur_exception,
sys_exitfunc, mp_sys_path_obj, mp_sys_argv_obj and sys_mutable
instead of using a conditional inside of mp_state_vm_t.
Signed-off-by: David Lechner <david@pybricks.com>
This uses MP_REGISTER_ROOT_POINTER() to register track_reloc_code_list
instead of using a conditional inside of mp_state_vm_t.
Signed-off-by: David Lechner <david@pybricks.com>
This uses MP_REGISTER_ROOT_POINTER() to register `bluetooth`
instead of using a conditional inside of mp_state_vm_t.
Signed-off-by: David Lechner <david@pybricks.com>
This uses MP_REGISTER_ROOT_POINTER() to register vfs_cur and
vfs_mount_table instead of using a conditional inside of mp_state_vm_t.
Signed-off-by: David Lechner <david@pybricks.com>
This uses MP_REGISTER_ROOT_POINTER() to register lwip_slip_stream
instead of using a conditional inside of mp_state_vm_t.
Signed-off-by: David Lechner <david@pybricks.com>
This uses MP_REGISTER_ROOT_POINTER() to register dupterm_objs
instead of using a conditional inside of mp_state_vm_t.
Signed-off-by: David Lechner <david@pybricks.com>
This uses MP_REGISTER_ROOT_POINTER() to register repl_line
instead of using a conditional inside of mp_state_vm_t.
Signed-off-by: David Lechner <david@pybricks.com>
All in-tree uses of MICROPY_PORT_ROOT_POINTERS have been replaced with
MP_REGISTER_ROOT_POINTER(), so now we can remove both
MICROPY_PORT_ROOT_POINTERS and MICROPY_BOARD_ROOT_POINTERS from the code
and remaining config files.
Signed-off-by: David Lechner <david@pybricks.com>
This uses MP_REGISTER_ROOT_POINTER() to register the readline_history root
pointer array used by shared/readline.c and removes the registration from
all mpconfigport.h files.
This also required adding a new MICROPY_READLINE_HISTORY_SIZE config option
since not all ports used the same sized array.
Signed-off-by: David Lechner <david@pybricks.com>
This adds new compile-time infrastructure to parse source code files for
`MP_REGISTER_ROOT_POINTER()` and generates a new `root_pointers.h` header
file containing the collected declarations. This works the same as the
existing `MP_REGISTER_MODULE()` feature.
Signed-off-by: David Lechner <david@pybricks.com>
Zero effect on non debug builds, and also usually optimized out even in
debug builds as mp_obj_is_type() is called with a compile-time known type.
I'm not sure we even have dynamic uses of mp_obj_is_type() at the moment,
but if we ever will they will be protected from now on.
Signed-off-by: Yonatan Goldschmidt <yon.goldschmidt@gmail.com>
Commit d96cfd13e3 introduced a regression by breaking existing
users of mp_obj_is_type(.., &mp_obj_bool). This function (and associated
helpers like mp_obj_is_int()) have some specific nuances, and mistakes like
this one can happen again.
This commit adds mp_obj_is_exact_type() which behaves like the the old
mp_obj_is_type(). The new mp_obj_is_type() has the same prototype but it
attempts to statically assert that it's not called with types which should
be checked using mp_obj_is_type(). If called with any of these types: int,
str, bool, NoneType - it will cause a compilation error. Additional
checked types (e.g function types) can be added in the future.
Existing users of mp_obj_is_type() with the now "invalid" types, were
translated to use mp_obj_is_exact_type().
The use of MP_STATIC_ASSERT() is not bulletproof - usually GCC (and other
compilers) can't statically check conditions that are only known during
link-time (like variables' addresses comparison). However, in this case,
GCC is able to statically detect these conditions, probably because it's
the exact same object - `&mp_type_int == &mp_type_int` is detected.
Misuses of this function with runtime-chosen types (e.g:
`mp_obj_type_t *x = ...; mp_obj_is_type(..., x);` won't be detected. MSC
is unable to detect this, so we use MP_STATIC_ASSERT_NOT_MSC().
Compiling with this commit and without the fix for d96cfd13e3 shows
that it detects the problem.
Signed-off-by: Yonatan Goldschmidt <yon.goldschmidt@gmail.com>
The empty tuple is usually a constant object, but named tuples must be
allocated to allow modification. Added explicit allocation to fix this.
Also added a regression test to verify creating an empty named tuple works.
Fixes issue #7870.
Signed-off-by: Lars Haulin <lars.haulin@gmail.com>
The GENERATOR_EXIT_IF_NEEDED macro is only used once and it's easier to
read and understand the code if this macro body is written in the code.
Then the comment just before it makes more sense.
Signed-off-by: Damien George <damien@micropython.org>
This check for code_state->ip being NULL was added in
a7c02c4538 with a commit message that "When
generator raises exception, it is automatically terminated (by setting its
code_state.ip to 0)". It was also added without any tests to test for this
particular case. (The commit did mention that CPython's test_pep380.py
triggered a bug, but upon re-running this test it did not show any need for
this NULL check of code_state->ip.)
It is true that generators that have completed (either by running to their
end or raising an exception) set "code_state.ip = 0". But there is an
explicit check at the start of mp_obj_gen_resume() to return immediately
for any attempt to resume an already-stopped generator. So the VM can
never execute a generator with NULL ip (and this was true at the time of
the above-referenced commit).
Furthermore, the other parts of the VM just before and after this piece
of code do require (or at least assume) code_state->ip is non-NULL.
Signed-off-by: Damien George <damien@micropython.org>
The optimisation that allows a single check in the VM for either a pending
exception or non-empty scheduler queue doesn't work when threading is
enabled, as one thread can clear the sched_state if it has no pending
exception, meaning the thread with the pending exception will never see it.
This removes that optimisation for threaded builds.
Also fixes a race in non-scheduler builds where get-and-clear of the
pending exception is not protected by the atomic section.
Also removes the bulk of the inlining of pending exceptions and scheduler
handling from the VM. This just costs code size and complexity at no
performance benefit.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
Add .attr attribute which forwards to self->fun.
A closure is intended to wrap around a function object, so forward any
requested attributes to the wrapped function object.
Signed-off-by: Michael Bentley <mikebentley15@gmail.com>
Prior to this commit, complex("j") would return 0j, and complex("nanj")
would return nan+0j. This commit makes sure "j" is tested for after
parsing the number (nan, inf or a decimal), and also supports the case of
"j" on its own.
Signed-off-by: Damien George <damien@micropython.org>
This separates extmod source files from `py.mk`. Previously, `py.mk`
assumed that every consumer of the py/ directory also wanted to include
extmod/. However, this is not the case. For example, building mpy-cross
uses py/ but doesn't need extmod/.
This commit moves all extmod-specific items from `py.mk` to `extmod.mk` and
explicitly includes `extmod.mk` in ports that use it.
Signed-off-by: David Lechner <david@pybricks.com>
The following changes are made:
- Guard entire file with MICROPY_PY_LWIP, so it can be included in the
build while still being disabled (for consistency with other extmod
modules).
- Add modlwip.c to list of all extmod source in py/py.mk and
extmod/extmod.cmake so all ports can easily use it.
- Move generic modlwip GIT_SUBMODULES build configuration code from
ports/rp2/CMakeLists.txt to extmod/extmod.cmake, so it can be reused by
other ports.
- Remove now unnecessary inclusion of modlwip.c in EXTMOD_SRC_C in esp8266
port, and in SRC_QSTR in mimxrt port.
Signed-off-by: Damien George <damien@micropython.org>
This new logic tracks when an unconditional jump/raise occurs in the
emitted code stream (bytecode or native machine code) and suppresses all
subsequent code, until a label is assigned. This eliminates a lot of
cases of dead code, with relatively simple logic.
This commit combined with the previous one (that removed the existing
dead-code finding logic) has the following code size change:
bare-arm: -16 -0.028%
minimal x86: -60 -0.036%
unix x64: -368 -0.070%
unix nanbox: -80 -0.017%
stm32: -204 -0.052% PYBV10
cc3200: +0 +0.000%
esp8266: -232 -0.033% GENERIC
esp32: -224 -0.015% GENERIC[incl -40(data)]
mimxrt: -192 -0.054% TEENSY40
renesas-ra: -200 -0.032% RA6M2_EK
nrf: +28 +0.015% pca10040
rp2: -256 -0.050% PICO
samd: -12 -0.009% ADAFRUIT_ITSYBITSY_M4_EXPRESS
Signed-off-by: Damien George <damien@micropython.org>
The search in these cases should include all finally handlers that are
after the current ip. If a handler starts at exactly ip then it is
considered "after" the ip. This can happen when END_FINALLY is followed
immediately by a finally handler (from a different finally).
Consider the function:
def f():
try:
return 0
finally:
print(1)
The current bytecode emitter generates the following code:
00 SETUP_FINALLY 5
02 LOAD_CONST_SMALL_INT 0
03 RETURN_VALUE
04 LOAD_CONST_NONE ****
05 LOAD_GLOBAL print
07 LOAD_CONST_SMALL_INT 1
08 CALL_FUNCTION n=1 nkw=0
10 POP_TOP
11 END_FINALLY
12 LOAD_CONST_NONE
13 RETURN_VALUE
The LOAD_CONST_NONE marked with **** is dead code because it follows a
RETURN_VALUE, and nothing jumps to this LOAD_CONST_NONE. If the emitter
could remove this this dead code it would produce:
00 SETUP_FINALLY 4
02 LOAD_CONST_SMALL_INT 0
03 RETURN_VALUE
04 LOAD_GLOBAL print
06 LOAD_CONST_SMALL_INT 1
07 CALL_FUNCTION n=1 nkw=0
09 POP_TOP
10 END_FINALLY
11 LOAD_CONST_NONE
12 RETURN_VALUE
In this case the finally block (which starts at offset 4) immediately
follows the RETURN_VALUE. When RETURN_VALUE executes ip will point to
offset 4 in the bytecode (because the dispatch of the opcode does *ip++)
and so the finally handler will only be found if a >= comparison is used.
It's a similar story for break/continue:
while True:
try:
break
finally:
print(1)
Although technically in this case the > comparison still works because the
extra byte from the UNWIND_JUMP (encoding the number of exception handlers
to unwind) doesn't have a *ip++ (just a *ip) so ip remains pointing within
the UNWIND_JUMP opcode, and not at the start of the following finally
handler. Nevertheless, the change is made to use >= for consistency with
the RETURN_VALUE change.
Signed-off-by: Damien George <damien@micropython.org>
Catch calls to legacy:
MP_REGISTER_MODULE(name, module, enable)
Emit a friendly error suggesting they be rewritten to:
MP_REGISTER_MODULE(name, module).
Signed-off-by: Phil Howard <phil@pimoroni.com>
This file is not executable so shouldn't have the shebang line. This line
can cause issues when building on Windows msvc when the PyPython variable
is set to something other than "python", because it reverts back to using
the shebang line.
The top comment is also changed to """ style which matches all other
preprocessing scripts in the py/ directory.
Signed-off-by: Damien George <damien@micropython.org>
Without this, newer versions of gcc (eg 11.2.0) used with -O2 can warn
about `q_ptr` being maybe uninitialized, because it doesn't know that there
is at least one qstr being written in to this (alloca'd) memory.
As part of this, change the type of `n` to `size_t` so the compiler knows
it's unsigned and can generate better code.
Code size change for this commit:
bare-arm: -28 -0.049%
minimal x86: -4 -0.002%
unix x64: +0 +0.000%
unix nanbox: -16 -0.003%
stm32: -24 -0.006% PYBV10
cc3200: -32 -0.017%
esp8266: +8 +0.001% GENERIC
esp32: -52 -0.003% GENERIC
nrf: -24 -0.013% pca10040
rp2: -32 -0.006% PICO
samd: -28 -0.020% ADAFRUIT_ITSYBITSY_M4_EXPRESS
Signed-off-by: Damien George <damien@micropython.org>
This was made redundant by f2040bfc7e, which
also did not update this function for the change to qstr-opcode encoding,
so it does not work correctly anyway.
Signed-off-by: Damien George <damien@micropython.org>
Support for architecture-specific qstr linking was removed in
d4d53e9e11, where native code was changed to
access qstr values via qstr_table. The only remaining use for the special
qstr link table in persistentcode.c is to support native module written in
C, linked via mpy_ld.py. But native modules can also use the standard
module-level qstr_table (and obj_table) which was introduced in the .mpy
file reworking in f2040bfc7e.
This commit removes the remaining native qstr liking support in
persistentcode.c's load_raw_code function, and adds two new relocation
options for constants.qstr_table and constants.obj_table. mpy_ld.py is
updated to use these relocations options instead of the native qstr link
table.
Signed-off-by: Damien George <damien@micropython.org>
Rob Knegjens
Ayke van Laethem
stijn
David Lechner
Yonatan Goldschmidt
Lars Haulin
Damien George
Jim Mussared
Michael Bentley
Phil Howard
Jeremy Herbert
Andrew Leech