32-bit platforms only support a slice offset start of 24 bit max due to the
limited size of the mp_obj_array_t.free member. Similarly on 64-bit
platforms the limit is 56 bits.
This commit adds an OverflowError if the user attempts to slice a
memoryview beyond this limit.
Signed-off-by: Damien George <damien@micropython.org>
Showing 8 digits instead of 5, supporting devices with more than 1 MByte of
RAM (which is common these days). The masking was never needed, and the
related commented-out line can go.
To adhere to the contract of mp_map_lookup, namely:
MP_MAP_LOOKUP_ADD_IF_NOT_FOUND behaviour:
- returns slot, with key non-null and value=MP_OBJ_NULL if it was added
In @micropython.native code the types of variables and expressions are
always Python objects, so they can be initialised as such. This prevents
problems with compiling optimised code like while-loops where a local may
be referenced before it is assigned to.
Signed-off-by: Damien George <damien@micropython.org>
This was previously used for the definition of NIC types, but they have
been updated to use a protocol instead.
This work was funded through GitHub Sponsors.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
The assertion that is added here (to gc.c) fails when running this new test
if ALLOC_TABLE_GAP_BYTE is set to 0.
Signed-off-by: Jeff Epler <jepler@gmail.com>
Signed-off-by: Damien George <damien@micropython.org>
Prior to this fix the follow crash occurred. With a GC layout of:
GC layout:
alloc table at 0x3fd80428, length 32001 bytes, 128004 blocks
finaliser table at 0x3fd88129, length 16001 bytes, 128008 blocks
pool at 0x3fd8bfc0, length 2048064 bytes, 128004 blocks
Block 128003 is an AT_HEAD and eventually is passed to gc_mark_subtree.
This causes gc_mark_subtree to call ATB_GET_KIND(128004). When block 1 is
created with a finaliser, the first byte of the finaliser table becomes
0x2, but ATB_GET_KIND(128004) reads these bits as AT_TAIL, and then
gc_mark_subtree references past the end of the heap, which happened to be
past the end of PSRAM on the esp32-s2.
The fix in this commit is to ensure there is a one-byte gap after the ATB
filled permanently with AT_FREE.
Fixes issue #7116.
See also https://github.com/adafruit/circuitpython/issues/5021
Signed-off-by: Jeff Epler <jepler@gmail.com>
Signed-off-by: Damien George <damien@micropython.org>
When you want to use the valgrind memory analysis tool on MicroPython, you
can arrange to define MICROPY_DEBUG_VALGRIND to enable use of special
valgrind macros. For now, this only fixes `gc_get_ptr` so that it never
emits the diagnostic "Conditional jump or move depends on uninitialised
value(s)".
Signed-off-by: Jeff Epler <jepler@gmail.com>
This change makes it so the compiler and persistent code loader take a
mp_compiled_module_t* as their last argument, instead of returning this
struct. This eliminates a duplicate context variable for all callers of
these functions (because the context is now stored in the
mp_compiled_module_t by the caller), and also eliminates any confusion
about which context to use after the mp_compile_to_raw_code or
mp_raw_code_load function returns (because there is now only one context,
that stored in mp_compiled_module_t.context).
Reduces code size by 16 bytes on ARM Cortex-based ports.
Signed-off-by: Damien George <damien@micropython.org>
These unimplemented features may never be implemented, and having the word
"yet" there takes up space.
Signed-off-by: Damien George <damien@micropython.org>
This module is useful, but it is not always needed. Disabling it saves
several kilobytes of build size, depending on other config options.
Signed-off-by: Laurens Valk <laurens@pybricks.com>
The code was already checking for duplicate kwargs for named parameters but
if `**kwargs` was given as a parameter, it did not check for multiples of
the same argument name.
This fixes the issue by adding an addition test to catch duplicates and
adds a test to exercise the code.
Fixes issue #10083.
Signed-off-by: David Lechner <david@pybricks.com>
Implements dictionary union according to PEP 584's specifications, minus
the fact that dictionary entries are not guaranteed to be in insertion
order. This feature is enabled with MICROPY_CPYTHON_COMPAT.
Includes a new test.
With the assistance of Fangrui Qin <qinf@purdue.edu>
Signed-off-by: Rayane Chatrieux <rayane.chatrieux@gmail.com>
Signed-off-by: Damien George <damien@micropython.org>
If a CMake-build is run with `make BUILD=/outside/path` then
makeversionheader.py is run with the CWD set to the build directory, which
means the git version lookup will fail and silently fall back to the
mpconfig.h mode (giving the wrong result).
This commit:
- Uses the location of makeversionheader.py to find the repo path.
- Allows overriding this path via --repo-path.
This work was funded through GitHub Sponsors.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
This can be tested using ports/minimal and qemu:
make CC=mips-linux-gnu-gcc-8
Then run with qemu-mips:
stty raw opost -echo;
QEMU_LD_PREFIX=/usr/mips-linux-gnu/ qemu-mips build/firmware.elf;
sleep 1; reset
Signed-off-by: Jan Willeke <willeke@smartmote.de>
This prevents a very subtle bug caused by writing e.g. `bytearray('\xfd')`
which gives you `(0xc3, 0xbd)`.
This work was funded through GitHub Sponsors.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
There are two calls to mp_builtin___import__():
1. ports/unix/main.c:main_() which provides a str in args[0]
2. py/runtime.c:mp_import_name() which provides a qstr in args[0]
The default implementation of mp_builtin___import__() is
mp_builtin___import___default() which has a different implementation based
on MICROPY_ENABLE_EXTERNAL_IMPORT.
If MICROPY_ENABLE_EXTERNAL_IMPORT is disabled then the handling of weak
links assumes that args[0] is a `const char *`, when it is either a str or
qstr object.
Use the existing qstr of the module name instead, and also use a vstr
instead of strcpy() to ensure no overflow occurs.
Commit 64af916c11 removed the version string
from docs/conf.py. py/mpconfig.h is a better place to get the version
from, so use that (when there is no git repository).
Signed-off-by: Damien George <damien@micropython.org>
In order for v1.19.1 to load a .mpy, the formerly-feature-flags which are
now used for the sub-version must be zero.
The sub-version is only used to indicate a native version change, so it
should be zero when emitting bytecode-only .mpy files.
This work was funded through GitHub Sponsors.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
Prevents double-precision floats being enabled on 32-bit architectures
where they will not fit into the mp_obj_t encoding.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
Since there is only one flag, we don't need to use a bitfield in vstr_t.
Compilers emit extra instructions to access a bitfield, so this should
reduce the binary size a small amount.
Signed-off-by: David Lechner <david@pybricks.com>
This makes it so that all a port needs to do is set the relevant variables
and "include extmod.mk" and doesn't need to worry about adding anything to
OBJ, CFLAGS, SRC_QSTR, etc.
Make all extmod variables (src, flags, etc) private to extmod.mk.
Also move common/shared, extmod-related fragments (e.g. wiznet, cyw43,
bluetooth) into extmod.mk.
Now that SRC_MOD, CFLAGS_MOD, CXXFLAGS_MOD are unused by both extmod.mk
(and user-C-modules in a previous commit), remove all uses of them from
port makefiles.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
Removes the need for the port to add anything to OBJS or SRC_QSTR.
Also makes it possible for user-C-modules to differentiate between code
that should be processed for QSTR vs other files (e.g. helpers and
libraries).
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
Only intended to be used on Unix and other "OS" ports. Matches CPython.
This should give the absolute path to the executing binary.
This work was funded through GitHub Sponsors.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
Signed-off-by: Damien George <damien@micropython.org>
`b'\xaa \xaa'.count(b'\xaa')` now (correctly) returns 2 instead of 1.
Fixes issue #9404.
This work was funded through GitHub Sponsors.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
Allows optimisation of cases like:
import micropython
_DEBUG = micropython.const(False)
if _DEBUG:
print('Debugging info')
Previously the 'if' statement was only optimised out if the type of the
const() argument was integer.
The change is implemented in a way that makes the compiler slightly smaller
(-16 bytes on PYBV11) but compilation will also be very slightly slower.
As a bonus, if const support is enabled then the compiler can now optimise
const truthy/falsey expressions of other types, like:
while "something":
pass
... unclear if that is useful, but perhaps it could be.
Signed-off-by: Angus Gratton <angus@redyak.com.au>
This improves error messages in mpy-cross:
- When loading a .py file that doesn't exist (or can't be opened) it now
includes the filename in the OSError.
- When saving a .mpy file that can't be opened it now raises an exception
(prior, it would silently fail), and includes the filename in the
OSError.
Signed-off-by: Damien George <damien@micropython.org>
This matches class `__dict__`, and is similarly gated on
MICROPY_CPYTHON_COMPAT. Unlike class though, because modules's globals are
actually dict instances, the result is a mutable dictionary.
This work was funded through GitHub Sponsors.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
The intent is to allow us to make breaking changes to the native ABI (e.g.
changes to dynruntime.h) without needing the bytecode version to increment.
With this commit the two bits previously used for the feature flags (but
now unused as of .mpy version 6) encode a sub-version. A bytecode-only
.mpy file can be loaded as long as MPY_VERSION matches, but a native .mpy
(i.e. one with an arch set) must also match MPY_SUB_VERSION. This allows 3
additional updates to the native ABI per bytecode revision.
The sub-version is set to 1 because the previous commits that changed the
layout of mp_obj_type_t have changed the native ABI.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
Signed-off-by: Damien George <damien@micropython.org>
The check for make_new (i.e. used to determine something's type) is now
more complicated due to the slot access. This commit changes the inlining
of a few frequently-used helpers to overall improve code size and
performance.
Instead of being an explicit field, it's now a slot like all the other
methods.
This is a marginal code size improvement because most types have a make_new
(100/138 on PYBV11), however it improves consistency in how types are
declared, removing the special case for make_new.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
The goal here is to remove a slot (making way to turn make_new into a slot)
as well as reduce code size by the ~40 references to mp_identity_getiter
and mp_stream_unbuffered_iter.
This introduces two new type flags:
- MP_TYPE_FLAG_ITER_IS_ITERNEXT: This means that the "iter" slot in the
type is "iternext", and should use the identity getiter.
- MP_TYPE_FLAG_ITER_IS_CUSTOM: This means that the "iter" slot is a pointer
to a mp_getiter_iternext_custom_t instance, which then defines both
getiter and iternext.
And a third flag that is the OR of both, MP_TYPE_FLAG_ITER_IS_STREAM: This
means that the type should use the identity getiter, and
mp_stream_unbuffered_iter as iternext.
Finally, MP_TYPE_FLAG_ITER_IS_GETITER is defined as a no-op flag to give
the default case where "iter" is "getiter".
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
Rather than reserving a full 12-slot mp_obj_type_t, reserve enough room for
seven and cast as necessary.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
In all cases other than where you have a native base with a protocol, it
now fits into 4 GC blocks (like it did before the slots representation).
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
The existings mp_obj_type_t uses a sparse representation for slots for the
capability methods of the type (eg print, make_new). This commit adds a
compact slot-index representation. The basic idea is that where the
mp_obj_type_t struct used to have 12 pointer fields, it now has 12 uint8_t
indices, and a variable-length array of pointers. So in the best case (no
fields used) it saves 12x4-12=36 bytes (on a 32-bit machine) and in the
common case (three fields used) it saves 9x4-12=24 bytes.
Overall with all associated changes, this slot-index representation reduces
code size by 1000 to 3000 bytes on bare-metal ports. Performance is
marginally better on a few tests (eg about 1% better on misc_pystone.py and
misc_raytrace.py on PYBv1.1), but overall marginally worse by a percent or
so.
See issue #7542 for further analysis and discussion.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
This will always have the maximum/minimum size of a mp_obj_type_t
representation and can be used as a member in other structs.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
Andrew Leech
robert-hh
Philip Peitsch
Damien George
Jim Mussared
Jeff Epler
Laurens Valk
David Lechner
Rayane Chatrieux
Jan Willeke
Simon Arlott
stijn
Matt Trentini
Angus Gratton