Commit Graph

431 Commits

Author SHA1 Message Date
Damien George 1fb01bd6c5 py/emitnative: Put a pointer to the native prelude in child_table array.
Some architectures (like esp32 xtensa) cannot read byte-wise from
executable memory.  This means the prelude for native functions -- which is
usually located after the machine code for the native function -- must be
placed in separate memory that can be read byte-wise.  Prior to this commit
this was achieved by enabling N_PRELUDE_AS_BYTES_OBJ for the emitter and
MICROPY_EMIT_NATIVE_PRELUDE_AS_BYTES_OBJ for the runtime.  The prelude was
then placed in a bytes object, pointed to by the module's constant table.

This behaviour is changed by this commit so that a pointer to the prelude
is stored either in mp_obj_fun_bc_t.child_table, or in
mp_obj_fun_bc_t.child_table[num_children] if num_children > 0.  The reasons
for doing this are:

1. It decouples the native emitter from runtime requirements, the emitted
   code no longer needs to know if the system it runs on can/can't read
   byte-wise from executable memory.

2. It makes all ports have the same emitter behaviour, there is no longer
   the N_PRELUDE_AS_BYTES_OBJ option.

3. The module's constant table is now used only for actual constants in the
   Python code.  This allows further optimisations to be done with the
   constants (eg constant deduplication).

Code size change for those ports that enable the native emitter:
   unix x64:   +80 +0.015%
      stm32:   +24 +0.004% PYBV10
    esp8266:   +88 +0.013% GENERIC
      esp32:   -20 -0.002% GENERIC[incl -112(data)]
        rp2:   +32 +0.005% PICO

Signed-off-by: Damien George <damien@micropython.org>
2022-05-17 16:44:49 +10:00
Damien George c49d5207e9 py/persistentcode: Remove unicode feature flag from .mpy file.
Prior to this commit, even with unicode disabled .py and .mpy files could
contain unicode characters, eg by entering them directly in a string as
utf-8 encoded.

The only thing the compiler disallowed (with unicode disabled) was using
\uxxxx and \Uxxxxxxxx notation to specify a character within a string with
value >= 0x100; that would give a SyntaxError.

With this change mpy-cross will now accept \u and \U notation to insert a
character with value >= 0x100 into a string (because the -mno-unicode
option is now gone, there's no way to forbid this).  The runtime will
happily work with strings with such characters, just like it already works
with strings with characters that were utf-8 encoded directly.

This change simplifies things because there are no longer any feature
flags in .mpy files, and any bytecode .mpy will now run on any target.

Signed-off-by: Damien George <damien@micropython.org>
2022-05-17 12:51:54 +10:00
Damien George fca5701f74 py/malloc: Introduce m_tracked_calloc, m_tracked_free functions.
Enabled by MICROPY_TRACKED_ALLOC.

Signed-off-by: Damien George <damien@micropython.org>
2022-05-05 10:31:50 +10:00
Damien George 402df833fe py/modsys: Introduce sys.implementation._machine constant.
This contains a string useful for identifying the underlying machine.  This
string is kept consistent with the second part of the REPL banner via the
new config option MICROPY_BANNER_MACHINE.

This makes os.uname() more or less redundant, as all the information in
os.uname() is now available in the sys module.

Signed-off-by: Damien George <damien@micropython.org>
2022-04-28 17:23:03 +10:00
Damien George 40047823bc py/modsys: Append MicroPython git version and build date to sys.version.
This commit adds the git hash and build date to sys.version.  This is
allowed according to CPython docs, and is what PyPy does.  The docs state:

    A string containing the version number of the Python interpreter plus
    additional information on the build number and compiler used.

Eg on CPython:

    Python 3.10.4 (main, Mar 23 2022, 23:05:40) [GCC 11.2.0] on linux
    Type "help", "copyright", "credits" or "license" for more information.
    >>> import sys
    >>> sys.version
    '3.10.4 (main, Mar 23 2022, 23:05:40) [GCC 11.2.0]'

and PyPy:

    Python 2.7.12 (5.6.0+dfsg-4, Nov 20 2016, 10:43:30)
    [PyPy 5.6.0 with GCC 6.2.0 20161109] on linux2
    Type "help", "copyright", "credits" or "license" for more information.
    >>>> import sys
    >>>> sys.version
    '2.7.12 (5.6.0+dfsg-4, Nov 20 2016, 10:43:30)\n[PyPy 5.6.0 with GCC ...

With this commit on MicroPython we now have:

    MicroPython v1.18-371-g9d08eb024 on 2022-04-28; linux [GCC 11.2.0] v...
    Use Ctrl-D to exit, Ctrl-E for paste mode
    >>> import sys
    >>> sys.version
    '3.4.0; MicroPython v1.18-371-g9d08eb024 on 2022-04-28'

Note that the start of the banner is the same as the end of sys.version.
This helps to keep code size under control because the string can be reused
by the compiler.

Signed-off-by: Damien George <damien@micropython.org>
2022-04-28 15:23:17 +10:00
Damien George 35c0cff92b py/parse: Add MICROPY_COMP_CONST_TUPLE option to build const tuples.
This commit adds support to the parser so that tuples which contain only
constant elements (bool, int, str, bytes, etc) are immediately converted to
a tuple object.  This makes it more efficient to use tuples containing
constant data because they no longer need to be created at runtime by the
bytecode (or native code).

Furthermore, with this improvement constant tuples that are part of frozen
code are now able to be stored fully in ROM (this will be implemented in
later commits).

Code size is increased by about 400 bytes on Cortex-M4 platforms.

See related issue #722.

Signed-off-by: Damien George <damien@micropython.org>
2022-04-14 23:52:12 +10:00
Damien George 75506e496f py/scheduler: Add support for scheduling static C-based callbacks.
If MICROPY_SCHEDULER_STATIC_NODES is enabled then C code can declare a
static mp_sched_node_t and schedule a callback using
mp_sched_schedule_node().  In contrast to using mp_sched_schedule(), the
node version will have at most one pending callback outstanding, and will
always be able to schedule if there is nothing already scheduled on this
node.  This guarantees that the the callback will be called exactly once
after it is scheduled.

Signed-off-by: Damien George <damien@micropython.org>
2022-04-14 12:31:53 +10:00
Damien George aab005c75b extmod/modusocket: Provide config macro for socket.listen backlog deflt.
To make it possible to change this value for any given port or board.

Signed-off-by: Damien George <damien@micropython.org>
2022-04-11 15:28:56 +10:00
Damien George ac2293161e py/modsys: Add optional mutable attributes sys.ps1/ps2 and use them.
This allows customising the REPL prompt strings.

Signed-off-by: Damien George <damien@micropython.org>
2022-03-10 10:58:33 +11:00
Damien George cac939ddc3 py/modsys: Add optional sys.tracebacklimit attribute.
With behaviour as per CPython.

Signed-off-by: Damien George <damien@micropython.org>
2022-03-10 10:43:21 +11:00
Damien George bc181550a4 py/modsys: Add optional attribute delegation.
To be enabled when needed by specific sys attributes.

Signed-off-by: Damien George <damien@micropython.org>
2022-03-10 10:43:21 +11:00
Damien George 3356b5ef8d py/objmodule: Support delegating failed attr lookups.
This commit adds generic support for mutable module attributes on built in
modules, by adding support for an optional hook function for module
attribute lookup.  If a module wants to support additional attribute load/
store/delete (beyond what is in the constant, globals dict) then it should
add at the very end of its globals dict MP_MODULE_ATTR_DELEGATION_ENTRY().
This should point to a custom function which will handle any additional
attributes.

The mp_module_generic_attr() function is provided as a helper function for
additional attributes: it requires an array of qstrs (terminated in
MP_QSTRnull) and a corresponding array of objects (with a 1-1 mapping
between qstrs and objects).  If the qstr is found in the array then the
corresponding object is loaded/stored/deleted.

Signed-off-by: Damien George <damien@micropython.org>
2022-03-10 10:35:44 +11:00
Damien George 0149cd6b8b windows: Switch to VFS subsystem and use VfsPosix.
Following the unix port.

Signed-off-by: Damien George <damien@micropython.org>
2022-03-10 00:26:36 +11:00
Damien George 926b554daf extmod/moduos: Create general uos module to be used by all ports.
Based on the rp2 port version, with the rp2 port converted to use this
module.

Signed-off-by: Damien George <damien@micropython.org>
2022-03-09 10:03:23 +11:00
Damien George f2040bfc7e py: Rework bytecode and .mpy file format to be mostly static data.
Background: .mpy files are precompiled .py files, built using mpy-cross,
that contain compiled bytecode functions (and can also contain machine
code). The benefit of using an .mpy file over a .py file is that they are
faster to import and take less memory when importing.  They are also
smaller on disk.

But the real benefit of .mpy files comes when they are frozen into the
firmware.  This is done by loading the .mpy file during compilation of the
firmware and turning it into a set of big C data structures (the job of
mpy-tool.py), which are then compiled and downloaded into the ROM of a
device.  These C data structures can be executed in-place, ie directly from
ROM.  This makes importing even faster because there is very little to do,
and also means such frozen modules take up much less RAM (because their
bytecode stays in ROM).

The downside of frozen code is that it requires recompiling and reflashing
the entire firmware.  This can be a big barrier to entry, slows down
development time, and makes it harder to do OTA updates of frozen code
(because the whole firmware must be updated).

This commit attempts to solve this problem by providing a solution that
sits between loading .mpy files into RAM and freezing them into the
firmware.  The .mpy file format has been reworked so that it consists of
data and bytecode which is mostly static and ready to run in-place.  If
these new .mpy files are located in flash/ROM which is memory addressable,
the .mpy file can be executed (mostly) in-place.

With this approach there is still a small amount of unpacking and linking
of the .mpy file that needs to be done when it's imported, but it's still
much better than loading an .mpy from disk into RAM (although not as good
as freezing .mpy files into the firmware).

The main trick to make static .mpy files is to adjust the bytecode so any
qstrs that it references now go through a lookup table to convert from
local qstr number in the module to global qstr number in the firmware.
That means the bytecode does not need linking/rewriting of qstrs when it's
loaded.  Instead only a small qstr table needs to be built (and put in RAM)
at import time.  This means the bytecode itself is static/constant and can
be used directly if it's in addressable memory.  Also the qstr string data
in the .mpy file, and some constant object data, can be used directly.
Note that the qstr table is global to the module (ie not per function).

In more detail, in the VM what used to be (schematically):

    qst = DECODE_QSTR_VALUE;

is now (schematically):

    idx = DECODE_QSTR_INDEX;
    qst = qstr_table[idx];

That allows the bytecode to be fixed at compile time and not need
relinking/rewriting of the qstr values.  Only qstr_table needs to be linked
when the .mpy is loaded.

Incidentally, this helps to reduce the size of bytecode because what used
to be 2-byte qstr values in the bytecode are now (mostly) 1-byte indices.
If the module uses the same qstr more than two times then the bytecode is
smaller than before.

The following changes are measured for this commit compared to the
previous (the baseline):
- average 7%-9% reduction in size of .mpy files
- frozen code size is reduced by about 5%-7%
- importing .py files uses about 5% less RAM in total
- importing .mpy files uses about 4% less RAM in total
- importing .py and .mpy files takes about the same time as before

The qstr indirection in the bytecode has only a small impact on VM
performance.  For stm32 on PYBv1.0 the performance change of this commit
is:

diff of scores (higher is better)
N=100 M=100             baseline -> this-commit  diff      diff% (error%)
bm_chaos.py               371.07 ->  357.39 :  -13.68 =  -3.687% (+/-0.02%)
bm_fannkuch.py             78.72 ->   77.49 :   -1.23 =  -1.563% (+/-0.01%)
bm_fft.py                2591.73 -> 2539.28 :  -52.45 =  -2.024% (+/-0.00%)
bm_float.py              6034.93 -> 5908.30 : -126.63 =  -2.098% (+/-0.01%)
bm_hexiom.py               48.96 ->   47.93 :   -1.03 =  -2.104% (+/-0.00%)
bm_nqueens.py            4510.63 -> 4459.94 :  -50.69 =  -1.124% (+/-0.00%)
bm_pidigits.py            650.28 ->  644.96 :   -5.32 =  -0.818% (+/-0.23%)
core_import_mpy_multi.py  564.77 ->  581.49 :  +16.72 =  +2.960% (+/-0.01%)
core_import_mpy_single.py  68.67 ->   67.16 :   -1.51 =  -2.199% (+/-0.01%)
core_qstr.py               64.16 ->   64.12 :   -0.04 =  -0.062% (+/-0.00%)
core_yield_from.py        362.58 ->  354.50 :   -8.08 =  -2.228% (+/-0.00%)
misc_aes.py               429.69 ->  405.59 :  -24.10 =  -5.609% (+/-0.01%)
misc_mandel.py           3485.13 -> 3416.51 :  -68.62 =  -1.969% (+/-0.00%)
misc_pystone.py          2496.53 -> 2405.56 :  -90.97 =  -3.644% (+/-0.01%)
misc_raytrace.py          381.47 ->  374.01 :   -7.46 =  -1.956% (+/-0.01%)
viper_call0.py            576.73 ->  572.49 :   -4.24 =  -0.735% (+/-0.04%)
viper_call1a.py           550.37 ->  546.21 :   -4.16 =  -0.756% (+/-0.09%)
viper_call1b.py           438.23 ->  435.68 :   -2.55 =  -0.582% (+/-0.06%)
viper_call1c.py           442.84 ->  440.04 :   -2.80 =  -0.632% (+/-0.08%)
viper_call2a.py           536.31 ->  532.35 :   -3.96 =  -0.738% (+/-0.06%)
viper_call2b.py           382.34 ->  377.07 :   -5.27 =  -1.378% (+/-0.03%)

And for unix on x64:

diff of scores (higher is better)
N=2000 M=2000        baseline -> this-commit     diff      diff% (error%)
bm_chaos.py          13594.20 ->  13073.84 :  -520.36 =  -3.828% (+/-5.44%)
bm_fannkuch.py          60.63 ->     59.58 :    -1.05 =  -1.732% (+/-3.01%)
bm_fft.py           112009.15 -> 111603.32 :  -405.83 =  -0.362% (+/-4.03%)
bm_float.py         246202.55 -> 247923.81 : +1721.26 =  +0.699% (+/-2.79%)
bm_hexiom.py           615.65 ->    617.21 :    +1.56 =  +0.253% (+/-1.64%)
bm_nqueens.py       215807.95 -> 215600.96 :  -206.99 =  -0.096% (+/-3.52%)
bm_pidigits.py        8246.74 ->   8422.82 :  +176.08 =  +2.135% (+/-3.64%)
misc_aes.py          16133.00 ->  16452.74 :  +319.74 =  +1.982% (+/-1.50%)
misc_mandel.py      128146.69 -> 130796.43 : +2649.74 =  +2.068% (+/-3.18%)
misc_pystone.py      83811.49 ->  83124.85 :  -686.64 =  -0.819% (+/-1.03%)
misc_raytrace.py     21688.02 ->  21385.10 :  -302.92 =  -1.397% (+/-3.20%)

The code size change is (firmware with a lot of frozen code benefits the
most):

       bare-arm:  +396 +0.697%
    minimal x86: +1595 +0.979% [incl +32(data)]
       unix x64: +2408 +0.470% [incl +800(data)]
    unix nanbox: +1396 +0.309% [incl -96(data)]
          stm32: -1256 -0.318% PYBV10
         cc3200:  +288 +0.157%
        esp8266:  -260 -0.037% GENERIC
          esp32:  -216 -0.014% GENERIC[incl -1072(data)]
            nrf:  +116 +0.067% pca10040
            rp2:  -664 -0.135% PICO
           samd:  +844 +0.607% ADAFRUIT_ITSYBITSY_M4_EXPRESS

As part of this change the .mpy file format version is bumped to version 6.
And mpy-tool.py has been improved to provide a good visualisation of the
contents of .mpy files.

In summary: this commit changes the bytecode to use qstr indirection, and
reworks the .mpy file format to be simpler and allow .mpy files to be
executed in-place.  Performance is not impacted too much.  Eventually it
will be possible to store such .mpy files in a linear, read-only, memory-
mappable filesystem so they can be executed from flash/ROM.  This will
essentially be able to replace frozen code for most applications.

Signed-off-by: Damien George <damien@micropython.org>
2022-02-24 18:08:43 +11:00
stijn dd6967202a py/modmath: Add math.tau, math.nan and math.inf constants.
Configurable by the new MICROPY_PY_MATH_CONSTANTS option.
2022-01-23 09:28:33 +11:00
Damien George da4b38e756 all: Bump version to 1.18.
Signed-off-by: Damien George <damien@micropython.org>
2022-01-17 09:50:31 +11:00
Damien George 772058a6bd py/mpconfig.h: Define MICROPY_PY_USSL_FINALISER only if not defined.
So a port can define it even if MICROPY_PY_USSL is not defined.

Signed-off-by: Damien George <damien@micropython.org>
2022-01-07 23:59:17 +11:00
stijn 22cf0940e1 py/modbuiltins: Add additional macro for extending builtins.
Mainly useful for defining additional globals in boards and variants.
2022-01-07 11:36:52 +11:00
Damien George fe9ffff9c0 py/mpstate.h: Only include sys.path/argv objects in state when enabled.
The mp_sys_path_obj and mp_sys_argv_obj objects are only used by the
runtime and accessible from Python if MICROPY_PY_SYS is enabled.  So
exclude them from the runtime state if this option is disabled.

Signed-off-by: Damien George <damien@micropython.org>
2021-12-19 08:55:40 +11:00
Damien George de43b500bd py/runtime: Allow initialising sys.path/argv with defaults.
If MICROPY_PY_SYS_PATH_ARGV_DEFAULTS is enabled (which it is by default)
then sys.path and sys.argv will be initialised and populated with default
values.  This keeps all bare-metal ports aligned.

Signed-off-by: Damien George <damien@micropython.org>
2021-12-18 00:08:07 +11:00
Jim Mussared cc23e99f32 py/modio: Remove io.resource_stream function.
This feature is not enabled on any port, it's not in CPython's io module,
and functionality is better suited to the micropython-lib implementation of
pkg_resources.
2021-12-17 23:53:44 +11:00
Jim Mussared a7fa18c203 py/builtinimport: Refactor module importing.
Simplify and document/comment the handling of builtin import for:
- already-loaded modules
- built-in modules
- built-in umodules (formerly weak links)
- filesystem modules

Retains existing functionality with smaller code size but should also
facilitate potential new features (built-in packages, controlling the
frozen path).

Also makes the (unix-only) -m behavior a bit more obvious and configurable.

Code size change with this commit:

   bare-arm:    +0 +0.000%
minimal x86:   -64 -0.039%
   unix x64:   -32 -0.006%
unix nanbox:    -4 -0.001%
      stm32:  -184 -0.047% PYBV10
     cc3200:  -120 -0.065%
    esp8266:  -228 -0.033% GENERIC
      esp32:  -268 -0.018% GENERIC[incl +16(data)]
        nrf:  -152 -0.087% pca10040
        rp2:  -256 -0.052% PICO
       samd:   -80 -0.057% ADAFRUIT_ITSYBITSY_M4_EXPRESS
2021-12-01 13:23:34 +11:00
Laurens Valk fe120484b6 py/gc: Add hook to run code during time consuming GC operations.
This makes it possible for cooperative multitasking systems to keep running
event loops during garbage collector operations.

For example, this can be used to ensure that a motor control loop runs
approximately each 5 ms.  Without this hook, the loop time can jump to
about 15 ms.

Addresses #3475.

Signed-off-by: Laurens Valk <laurens@pybricks.com>
2021-11-01 15:39:37 +11:00
Damien George c62351fbd6 py/mpconfig.h: Revert MICROPY_REPL_INFO to disabled at all levels.
This is an stm32-specific feature that's accessed via the pyb module, so
not something that will be widely enabled.

Signed-off-by: Damien George <damien@micropython.org>
2021-11-01 15:18:22 +11:00
Jim Mussared 0e236eef08 py/mpconfig.h: Define the "extra" feature level.
Some of these will later be moved to CORE or BASIC, but EXTRA is a good
starting point based on what stm32 uses.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2021-11-01 14:57:28 +11:00
Damien George 8412568e7b py: Add wrapper macros so hot VM functions can go in fast code location.
For example, on esp32 they can go in iRAM to improve performance.

Signed-off-by: Damien George <damien@micropython.org>
2021-10-15 23:31:19 +11:00
Jim Mussared b326edf68c all: Remove MICROPY_OPT_CACHE_MAP_LOOKUP_IN_BYTECODE.
This commit removes all parts of code associated with the existing
MICROPY_OPT_CACHE_MAP_LOOKUP_IN_BYTECODE optimisation option, including the
-mcache-lookup-bc option to mpy-cross.

This feature originally provided a significant performance boost for Unix,
but wasn't able to be enabled for MCU targets (due to frozen bytecode), and
added significant extra complexity to generating and distributing .mpy
files.

The equivalent performance gain is now provided by the combination of
MICROPY_OPT_LOAD_ATTR_FAST_PATH and MICROPY_OPT_MAP_LOOKUP_CACHE (which has
been enabled on the unix port in the previous commit).

It's hard to provide precise performance numbers, but tests have been run
on a wide variety of architectures (x86-64, ARM Cortex, Aarch64, RISC-V,
xtensa) and they all generally agree on the qualitative improvements seen
by the combination of MICROPY_OPT_LOAD_ATTR_FAST_PATH and
MICROPY_OPT_MAP_LOOKUP_CACHE.

For example, on a "quiet" Linux x64 environment (i3-5010U @ 2.10GHz) the
change from CACHE_MAP_LOOKUP_IN_BYTECODE, to LOAD_ATTR_FAST_PATH combined
with MAP_LOOKUP_CACHE is:

diff of scores (higher is better)
N=2000 M=2000       bccache -> attrmapcache      diff      diff% (error%)
bm_chaos.py        13742.56 ->   13905.67 :   +163.11 =  +1.187% (+/-3.75%)
bm_fannkuch.py        60.13 ->      61.34 :     +1.21 =  +2.012% (+/-2.11%)
bm_fft.py         113083.20 ->  114793.68 :  +1710.48 =  +1.513% (+/-1.57%)
bm_float.py       256552.80 ->  243908.29 : -12644.51 =  -4.929% (+/-1.90%)
bm_hexiom.py         521.93 ->     625.41 :   +103.48 = +19.826% (+/-0.40%)
bm_nqueens.py     197544.25 ->  217713.12 : +20168.87 = +10.210% (+/-3.01%)
bm_pidigits.py      8072.98 ->    8198.75 :   +125.77 =  +1.558% (+/-3.22%)
misc_aes.py        17283.45 ->   16480.52 :   -802.93 =  -4.646% (+/-0.82%)
misc_mandel.py     99083.99 ->  128939.84 : +29855.85 = +30.132% (+/-5.88%)
misc_pystone.py    83860.10 ->   82592.56 :  -1267.54 =  -1.511% (+/-2.27%)
misc_raytrace.py   21490.40 ->   22227.23 :   +736.83 =  +3.429% (+/-1.88%)

This shows that the new optimisations are at least as good as the existing
inline-bytecode-caching, and are sometimes much better (because the new
ones apply caching to a wider variety of map lookups).

The new optimisations can also benefit code generated by the native
emitter, because they apply to the runtime rather than the generated code.
The improvement for the native emitter when LOAD_ATTR_FAST_PATH and
MAP_LOOKUP_CACHE are enabled is (same Linux environment as above):

diff of scores (higher is better)
N=2000 M=2000        native -> nat-attrmapcache  diff      diff% (error%)
bm_chaos.py        14130.62 ->   15464.68 :  +1334.06 =  +9.441% (+/-7.11%)
bm_fannkuch.py        74.96 ->      76.16 :     +1.20 =  +1.601% (+/-1.80%)
bm_fft.py         166682.99 ->  168221.86 :  +1538.87 =  +0.923% (+/-4.20%)
bm_float.py       233415.23 ->  265524.90 : +32109.67 = +13.756% (+/-2.57%)
bm_hexiom.py         628.59 ->     734.17 :   +105.58 = +16.796% (+/-1.39%)
bm_nqueens.py     225418.44 ->  232926.45 :  +7508.01 =  +3.331% (+/-3.10%)
bm_pidigits.py      6322.00 ->    6379.52 :    +57.52 =  +0.910% (+/-5.62%)
misc_aes.py        20670.10 ->   27223.18 :  +6553.08 = +31.703% (+/-1.56%)
misc_mandel.py    138221.11 ->  152014.01 : +13792.90 =  +9.979% (+/-2.46%)
misc_pystone.py    85032.14 ->  105681.44 : +20649.30 = +24.284% (+/-2.25%)
misc_raytrace.py   19800.01 ->   23350.73 :  +3550.72 = +17.933% (+/-2.79%)

In summary, compared to MICROPY_OPT_CACHE_MAP_LOOKUP_IN_BYTECODE, the new
MICROPY_OPT_LOAD_ATTR_FAST_PATH and MICROPY_OPT_MAP_LOOKUP_CACHE options:
- are simpler;
- take less code size;
- are faster (generally);
- work with code generated by the native emitter;
- can be used on embedded targets with a small and constant RAM overhead;
- allow the same .mpy bytecode to run on all targets.

See #7680 for further discussion.  And see also #7653 for a discussion
about simplifying mpy-cross options.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2021-09-16 16:04:03 +10:00
Jim Mussared 11ef8f22fe py/map: Add an optional cache of (map+index) to speed up map lookups.
The existing inline bytecode caching optimisation, selected by
MICROPY_OPT_CACHE_MAP_LOOKUP_IN_BYTECODE, reserves an extra byte in the
bytecode after certain opcodes, which at runtime stores a map index of the
likely location of this field when looking up the qstr.  This scheme is
incompatible with bytecode-in-ROM, and doesn't work with native generated
code.  It also stores bytecode in .mpy files which is of a different format
to when the feature is disabled, making generation of .mpy files more
complex.

This commit provides an alternative optimisation via an approach that adds
a global cache for map offsets, then all mp_map_lookup operations use it.
It's less precise than bytecode caching, but allows the cache to be
independent and external to the bytecode that is executing.  It also works
for the native emitter and adds a similar performance boost on top of the
gain already provided by the native emitter.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2021-09-16 16:02:19 +10:00
Jim Mussared 7b89ad8dbf py/vm: Add a fast path for LOAD_ATTR on instance types.
When the LOAD_ATTR opcode is executed there are quite a few different cases
that have to be handled, but the common case is accessing a member on an
instance type.  Typically, built-in types provide methods which is why this
is common.

Fortunately, for this specific case, if the member is found in the member
map then there's no further processing.

This optimisation does a relatively cheap check (type is instance) and then
forwards directly to the member map lookup, falling back to the regular
path if necessary.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2021-09-16 16:02:15 +10:00
Jim Mussared 01374d941f py/mpconfig.h: Define initial templates for "feature levels".
This is the beginning of a set of changes to simplify enabling/disabling
features.  The goals are:
- Remove redundancy from mpconfigport.h (never set a value to the default
  -- make it clear exactly what's being enabled).
- Improve consistency between ports.  All "similar" ports (i.e. approx same
  flash size) should get the same features.
- Simplify mpconfigport.h -- just get default/sensible options for the size
  of the port.
- Make it easy for defining constrained boards (e.g. STM32F0/L0), they can
  just set a lower level.

This commit makes a step towards this and defines the "core" level as the
current default feature set, and a "minimal" level to turn off everything.
And a few placeholder levels are added for where the other ports will
roughly land.

This is a no-op change for all ports.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2021-09-16 13:19:11 +10:00
Damien George d41f6dde56 extmod/modonewire: Make _onewire module configurable via macro option.
Signed-off-by: Damien George <damien@micropython.org>
2021-09-02 13:11:23 +10:00
Damien George afe0634c98 extmod/machine_spi: Make SoftSPI configurable via macro option.
Signed-off-by: Damien George <damien@micropython.org>
2021-09-02 13:11:23 +10:00
Damien George 122d901ef1 extmod/machine_i2c: Make SoftI2C configurable via macro option.
The zephyr port doesn't support SoftI2C so it's not enabled, and the legacy
I2C constructor check can be removed.

Signed-off-by: Damien George <damien@micropython.org>
2021-09-02 13:11:23 +10:00
Damien George 7c54b64280 all: Bump version to 1.17.
Signed-off-by: Damien George <damien@micropython.org>
2021-09-02 00:07:13 +10:00
Jim Mussared b51e7e9d01 stm32: Disable computed goto on constrained boards.
Saves ~1kiB.  Add comment to this effect to mpconfig.h.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2021-08-20 20:18:52 +10:00
Jim Mussared 870000f35b extmod: Add machine.bitstream.
This is a generic API for synchronously bit-banging data on a pin.

Initially this adds a single supported encoding, which supports controlling
WS2812 LEDs.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2021-08-19 22:50:11 +10:00
Jim Mussared 692d36d779 py: Implement partial PEP-498 (f-string) support.
This implements (most of) the PEP-498 spec for f-strings and is based on
https://github.com/micropython/micropython/pull/4998 by @klardotsh.

It is implemented in the lexer as a syntax translation to `str.format`:
  f"{a}" --> "{}".format(a)

It also supports:
  f"{a=}" --> "a={}".format(a)

This is done by extracting the arguments into a temporary vstr buffer,
then after the string has been tokenized, the lexer input queue is saved
and the contents of the temporary vstr buffer are injected into the lexer
instead.

There are four main limitations:
- raw f-strings (`fr` or `rf` prefixes) are not supported and will raise
  `SyntaxError: raw f-strings are not supported`.

- literal concatenation of f-strings with adjacent strings will fail
    "{}" f"{a}" --> "{}{}".format(a)    (str.format will incorrectly use
                                         the braces from the non-f-string)
    f"{a}" f"{a}" --> "{}".format(a) "{}".format(a) (cannot concatenate)

- PEP-498 requires the full parser to understand the interpolated
  argument, however because this entirely runs in the lexer it cannot
  resolve nested braces in expressions like
    f"{'}'}"

- The !r, !s, and !a conversions are not supported.

Includes tests and cpydiffs.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2021-08-14 16:58:40 +10:00
Peter Züger ffc854f17f extmod/modujson: Add support for dump/dumps separators keyword-argument.
Optionally enabled via MICROPY_PY_UJSON_SEPARATORS.  Enabled by default.

For dump, make sure mp_get_stream_raise is called after
mod_ujson_separators since CPython does it in this order (if both
separators and stream are invalid, separators will raise an exception
first).

Add separators argument in the docs as well.

Signed-off-by: Peter Züger <zueger.peter@icloud.com>
Signed-off-by: Damien George <damien@micropython.org>
2021-08-07 13:52:16 +10:00
David Lechner 8758504f0f extmod/moduselect: Conditionally compile select().
This adds #if MICROPY_PY_USELECT_SELECT around the uselect.select()
function. According to the docs, this function is only for CPython
compatibility and should not normally be used. So we can disable it
and save a few bytes of flash space where possible.

Signed-off-by: David Lechner <david@pybricks.com>
2021-07-17 23:32:39 +10:00
Damien George 136369d72f all: Update to point to files in new shared/ directory.
Signed-off-by: Damien George <damien@micropython.org>
2021-07-12 17:08:10 +10:00
Damien George 7c51cb2307 all: Bump version to 1.16.
Signed-off-by: Damien George <damien@micropython.org>
2021-06-18 16:38:06 +10:00
Damien George 916c3fd23f py/scheduler: Add optional port hook for when something is scheduled.
So that a port can "wake up" when there is work to do.

Signed-off-by: Damien George <damien@micropython.org>
2021-04-30 15:32:16 +10:00
Damien George e9e9c76ddf all: Rename mp_keyboard_interrupt to mp_sched_keyboard_interrupt.
To match mp_sched_exception() and mp_sched_schedule().

Signed-off-by: Damien George <damien@micropython.org>
2021-04-30 15:13:43 +10:00
Damien George 7cbf826a95 py/scheduler: Add mp_sched_exception() to schedule a pending exception.
This helper is added to properly set a pending exception, to mirror
mp_sched_schedule(), which schedules a function.

Signed-off-by: Damien George <damien@micropython.org>
2021-04-30 15:13:43 +10:00
Damien George d4b706c4d0 py: Add option to compile without any error messages at all.
This introduces a new option, MICROPY_ERROR_REPORTING_NONE, which
completely disables all error messages.  To be used in cases where
MicroPython needs to fit in very limited systems.

Signed-off-by: Damien George <damien@micropython.org>
2021-04-27 23:51:52 +10:00
Damien George 321d1897c3 all: Bump version to 1.15.
Signed-off-by: Damien George <damien@micropython.org>
2021-04-19 00:11:51 +10:00
matejcik 1a2ffda175 py/runtime: Make sys.modules preallocate to a configurable size.
This allows configuring the pre-allocated size of sys.modules dict, in
order to prevent unwanted reallocations at run-time (3 sys-modules is
really not quite enough for a larger project).
2021-04-12 22:36:16 +10:00
Damien George c891190c69 py: Rename WORD_MSBIT_HIGH to MP_OBJ_WORD_MSBIT_HIGH.
To make it clear it is for mp_obj_t/mp_uint_t "word" types, and to prefix
this macro with MP_.

Signed-off-by: Damien George <damien@micropython.org>
2021-02-04 22:46:42 +11:00
Damien George ad4656b861 all: Rename BYTES_PER_WORD to MP_BYTES_PER_OBJ_WORD.
The "word" referred to by BYTES_PER_WORD is actually the size of mp_obj_t
which is not always the same as the size of a pointer on the target
architecture.  So rename this config value to better reflect what it
measures, and also prefix it with MP_.

For uses of BYTES_PER_WORD in setting the stack limit this has been
changed to sizeof(void *), because the stack usually grows with
machine-word sized values (eg an nlr_buf_t has many machine words in it).

Signed-off-by: Damien George <damien@micropython.org>
2021-02-04 22:46:42 +11:00
Damien George 7e956fae28 py: Rename BITS_PER_BYTE to MP_BITS_PER_BYTE.
To give this macro a standard MP_ prefix.

Signed-off-by: Damien George <damien@micropython.org>
2021-02-04 22:46:42 +11:00
Damien George 8a41ee19c2 py: Remove BITS_PER_WORD definition.
It's only used in one location, to test if << or >> will overflow when
shifting mp_uint_t.  For such a test it's clearer to use sizeof(lhs_val),
which will be valid even if the type of lhs_val changes.

Signed-off-by: Damien George <damien@micropython.org>
2021-02-04 22:46:42 +11:00
Damien George 78b23c3a1f all: Bump version to 1.14.
Signed-off-by: Damien George <damien@micropython.org>
2021-02-03 00:59:07 +11:00
David CARLIER cb30928ac8 py/persistentcode: Introduce MICROPY_PERSISTENT_CODE_SAVE_FILE option.
This should be enabled when the mp_raw_code_save_file function is needed.

It is enabled for mpy-cross, and a check for defined(__APPLE__) is added to
cover Mac M1 systems.
2021-01-30 15:13:24 +11:00
graham sanderson 40d2010882 py/asmthumb: Add support for ARMv6M in native emitter.
Adds a new compile-time option MICROPY_EMIT_THUMB_ARMV7M which is enabled
by default (to get existing behaviour) and which should be disabled (set to
0) when building native emitter support (@micropython.native) on ARMv6M
targets.
2021-01-29 23:57:10 +11:00
Emil Renner Berthing ccd92335a1 py, extmod: Introduce and use MP_FALLTHROUGH macro.
Newer GCC versions are able to warn about switch cases that fall
through.  This is usually a sign of a forgotten break statement, but in
the few cases where a fall through is intended we annotate it with this
macro to avoid the warning.
2020-10-22 11:53:16 +02:00
Damien George 843dcd4f85 py/parse: Expose rule-name printing as MICROPY_DEBUG_PARSE_RULE_NAME.
So it can be enabled without modifying the source.

Signed-off-by: Damien George <damien@micropython.org>
2020-10-01 15:26:43 +10:00
stijn 2e54d9d146 py: Fix handling of NaN in certain pow implementations.
Adds a new compile-time option MICROPY_PY_MATH_POW_FIX_NAN for use with
toolchains that don't handle pow-of-NaN correctly.
2020-09-11 10:04:57 +10:00
Damien George b0932fcf2e all: Bump version to 1.13.
Signed-off-by: Damien George <damien@micropython.org>
2020-09-02 12:01:26 +10:00
Damien George 9883d8e818 py/persistentcode: Maintain root ptr list of imported native .mpy code.
On ports where normal heap memory can contain executable code (eg ARM-based
ports such as stm32), native code loaded from an .mpy file may be reclaimed
by the GC because there's no reference to the very start of the native
machine code block that is reachable from root pointers (only pointers to
internal parts of the machine code block are reachable, but that doesn't
help the GC find the memory).

This commit fixes this issue by maintaining an explicit list of root
pointers pointing to native code that is loaded from an .mpy file.  This
is not needed for all ports so is selectable by the new configuration
option MICROPY_PERSISTENT_CODE_TRACK_RELOC_CODE.  It's enabled by default
if a port does not specify any special functions to allocate or commit
executable memory.

A test is included to test that native code loaded from an .mpy file does
not get reclaimed by the GC.

Fixes #6045.

Signed-off-by: Damien George <damien@micropython.org>
2020-08-02 22:34:09 +10:00
Damien George 1783950311 py/compile: Implement PEP 572, assignment expressions with := operator.
The syntax matches CPython and the semantics are equivalent except that,
unlike CPython, MicroPython allows using := to assign to comprehension
iteration variables, because disallowing this would take a lot of code to
check for it.

The new compile-time option MICROPY_PY_ASSIGN_EXPR selects this feature and
is enabled by default, following MICROPY_PY_ASYNC_AWAIT.
2020-06-16 22:02:24 +10:00
stijn 81db22f693 py/modmath: Work around msvc float bugs in atan2, fmod and modf.
Older implementations deal with infinity/negative zero incorrectly.  This
commit adds generic fixes that can be enabled by any port that needs them,
along with new tests cases.
2020-05-28 09:54:54 +10:00
Damien George 544c308c18 py/scheduler: Add option to wrap mp_sched_schedule in arbitrary attr.
So ports can put it in a special memory section if needed.
2020-04-30 23:47:11 +10:00
stijn 84fa3312cf all: Format code to add space after C++-style comment start.
Note: the uncrustify configuration is explicitly set to 'add' instead of
'force' in order not to alter the comments which use extra spaces after //
as a means of indenting text for clarity.
2020-04-23 11:24:25 +10:00
Damien George e292296d52 py/objexcept: Remove optional TimeoutError exception.
TimeoutError was added back in 077812b2ab for
the cc3200 port. In f522849a4d the cc3200
port enabled use of it in the socket module aliased to socket.timeout.  So
it was never added to the builtins.  Then it was replaced by
OSError(ETIMEDOUT) in 047af9b10b.

The esp32 port enables this exception, since the very beginning of that
port, but it could never be accessed because it's not in builtins.

It's being removed: 1) to not encourage its use; 2) because there are a lot
of other OSError subclasses which are not defined at all, and having
TimeoutError is a bit inconsistent.

Note that ports can add anything to the builtins via MICROPY_PORT_BUILTINS.
And they can also define their own exceptions using the
MP_DEFINE_EXCEPTION() macro.
2020-04-09 16:09:38 +10:00
Damien George bc009fdd62 extmod/uasyncio: Add optional implementation of core uasyncio in C.
Implements Task and TaskQueue classes in C, using a pairing-heap data
structure.  Using this reduces RAM use of each Task, and improves overall
performance of the uasyncio scheduler.
2020-03-26 01:25:45 +11:00
Damien George f8fc78691d py/mpconfig.h: Enable MICROPY_MODULE_GETATTR by default.
To enable lazy loading of submodules (among other things), which is very
useful for MicroPython libraries that want to have optional subcomponents.

Disabled explicitly on minimal ports.
2020-03-26 01:21:04 +11:00
Damien George feb2577585 all: Remove spaces between nested paren and inside function arg paren.
Using new options enabled in the uncrustify configuration.
2020-03-25 00:39:46 +11:00
Andrew Leech 86bfabec11 py/modmicropython: Add heap_locked function to test state of heap.
This commit adds micropython.heap_locked() which returns the current
lock-depth of the heap, and can be used by Python code to check if the heap
is locked or not.  This new function is configured via
MICROPY_PY_MICROPYTHON_HEAP_LOCKED and is disabled by default.

This commit also changes the return value of micropython.heap_unlock() so
it returns the current lock-depth as well.
2020-03-11 16:54:16 +11:00
Damien George 69661f3343 all: Reformat C and Python source code with tools/codeformat.py.
This is run with uncrustify 0.70.1, and black 19.10b0.
2020-02-28 10:33:03 +11:00
Damien George 8fb5c8fdd5 py/scheduler: Allow a port to specify attrs for mp_keyboard_interrupt.
Functions like mp_keyboard_interrupt() may need to be called from an IRQ
handler and may need to be in a special memory section, so provide a
generic wrapping macro for a port to do this.  The macro name is chosen to
be MICROPY_WRAP_<function name in uppercase> so that (in the future with
more wrappers) each function could potentially be handled separately.
2020-02-07 16:08:29 +11:00
Yonatan Goldschmidt 1c849d63a8 py/mpconfig.h: Define BITS_PER_BYTE only if not already defined.
It's a common macro that is possibly defined in headers of systems/SDKs
MicroPython is embedded into.
2020-01-12 20:57:01 +02:00
Damien George d96cfd13e3 py/obj: Add MICROPY_OBJ_IMMEDIATE_OBJS option to reduce code size.
This option (enabled by default for object representation A, B, C) makes
None/False/True objects immediate objects, ie they are no longer a concrete
object in ROM but are rather just values, eg None=0x6 for representation A.

Doing this saves a considerable amount of code size, due to these objects
being widely used:

   bare-arm:  -392 -0.591%
minimal x86:  -252 -0.170% [incl +52(data)]
   unix x64:  -624 -0.125% [incl -128(data)]
unix nanbox:    +0 +0.000%
      stm32: -1940 -0.510% PYBV10
     cc3200: -1216 -0.659%
    esp8266:  -404 -0.062% GENERIC
      esp32:  -732 -0.064% GENERIC[incl +48(data)]
        nrf:  -988 -0.675% pca10040
       samd:  -564 -0.556% ADAFRUIT_ITSYBITSY_M4_EXPRESS

Thanks go to @Jongy aka Yonatan Goldschmidt for the idea.
2020-01-13 01:01:45 +11:00
Damien George 6f0c83f6e1 py/obj.h: Redefine qstr object encoding to add immediate obj encoding.
This commit adjusts the definition of qstr encoding in all object
representations by taking a single bit from the qstr space and using it to
distinguish between qstrs and a new kind of literal object: immediate
objects.  In other words, the qstr space is divided in two pieces, one half
for qstrs and the other half for immediate objects.

There is still enough room for qstr values (29 bits in representation A on
a 32-bit architecture, and 19 bits in representation C) and the new
immediate objects can be used for things like None, False and True.
2020-01-13 01:01:45 +11:00
Yonatan Goldschmidt 853aaa06f2 lib/mp-readline: Add word-based move/delete EMACS key sequences.
This commit adds backward-word, backward-kill-word, forward-word,
forward-kill-word sequences for the REPL, with bindings to Alt+F, Alt+B,
Alt+D and Alt+Backspace respectively.  It is disabled by default and can be
enabled via MICROPY_REPL_EMACS_WORDS_MOVE.

Further enabling MICROPY_REPL_EMACS_EXTRA_WORDS_MOVE adds extra bindings
for these new sequences: Ctrl+Right, Ctrl+Left and Ctrl+W.

The features are enabled on unix micropython-coverage and micropython-dev.
2020-01-12 13:09:27 +11:00
Nicko van Someren 4c93955b7b py/objslice: Add support for indices() method on slice objects.
Instances of the slice class are passed to __getitem__() on objects when
the user indexes them with a slice.  In practice the majority of the time
(other than passing it on untouched) is to work out what the slice means in
the context of an array dimension of a particular length.  Since Python 2.3
there has been a method on the slice class, indices(), that takes a
dimension length and returns the real start, stop and step, accounting for
missing or negative values in the slice spec.  This commit implements such
a indices() method on the slice class.

It is configurable at compile-time via MICROPY_PY_BUILTINS_SLICE_INDICES,
disabled by default, enabled on unix, stm32 and esp32 ports.

This commit also adds new tests for slice indices and for slicing unicode
strings.
2019-12-28 23:55:15 +11:00
Yonatan Goldschmidt 61d2b40ad5 lib/utils/pyexec: Introduce MICROPY_REPL_INFO, wrap debug prints in it.
For the 3 ports that already make use of this feature (stm32, nrf and
teensy) this doesn't make any difference, it just allows to disable it from
now on.

For other ports that use pyexec, this decreases code size because the debug
printing code is dead (it can't be enabled) but the compiler can't deduce
that, so code is still emitted.
2019-12-28 00:05:39 +11:00
Damien George 1f37194730 all: Bump version to 1.12. 2019-12-20 16:58:17 +11:00
Damien George d2384efa80 py: Automatically provide weak links from "foo" to "ufoo" module name.
This commit implements automatic module weak links for all built-in
modules, by searching for "ufoo" in the built-in module list if "foo"
cannot be found.  This means that all modules named "ufoo" are always
available as "foo".  Also, a port can no longer add any other weak links,
which makes strict the definition of a weak link.

It saves some code size (about 100-200 bytes) on ports that previously had
lots of weak links.

Some changes from the previous behaviour:
- It doesn't intern the non-u module names (eg "foo" is not interned),
  which saves code size, but will mean that "import foo" creates a new qstr
  (namely "foo") in RAM (unless the importing module is frozen).
- help('modules') no longer lists non-u module names, only the u-variants;
  this reduces duplication in the help listing.

Weak links are effectively the same as having a set of symbolic links on
the filesystem that is searched last.  So an "import foo" will search
built-in modules first, then all paths in sys.path, then weak links last,
importing "ufoo" if it exists.  Thus a file called "foo.py" somewhere in
sys.path will still have precedence over the weak link of "foo" to "ufoo".

See issues: #1740, #4449, #5229, #5241.
2019-10-22 15:30:52 +11:00
Damien George 9adedce42e py: Add new Xtensa-Windowed arch for native emitter.
Enabled via the configuration MICROPY_EMIT_XTENSAWIN.
2019-10-05 13:44:53 +10:00
Milan Rossa 310b3d1b81 py: Integrate sys.settrace feature into the VM and runtime.
This commit adds support for sys.settrace, allowing to install Python
handlers to trace execution of Python code.  The interface follows CPython
as closely as possible.  The feature is disabled by default and can be
enabled via MICROPY_PY_SYS_SETTRACE.
2019-08-30 16:44:12 +10:00
Damien George 7d851a27f1 extmod/modure: Make regex dump-code debugging feature optional.
Enabled via MICROPY_PY_URE_DEBUG, disabled by default (but enabled on unix
coverage build).  This is a rarely used feature that costs a lot of code
(500-800 bytes flash).  Debugging of regular expressions can be done
offline with other tools.
2019-08-19 16:43:00 +10:00
Damien George 0c80cb39af py: Introduce MP_UNREACHABLE macro to annotate unreachable code.
And use it to replace the same pattern at the end of nlrthumb.c:nlr_jump.
2019-08-19 16:14:33 +10:00
stijn af5c998f37 py/modmath: Implement math.isclose() for non-complex numbers.
As per PEP 485, this function appeared in for Python 3.5.  Configured via
MICROPY_PY_MATH_ISCLOSE which is disabled by default, but enabled for the
ports which already have MICROPY_PY_MATH_SPECIAL_FUNCTIONS enabled.
2019-08-17 23:23:17 +10:00
Milan Rossa cb3647004f py: Implement new sys.atexit feature.
This patch implements a new sys.atexit function which registers a function
that is later executed when the main script ends.  It is configurable via
MICROPY_PY_SYS_ATEXIT, disabled by default.

This is not compliant with CPython, rather it can be used to implement a
CPython compatible "atexit" module if desired (similar to how
sys.print_exception can be used to implement functionality of the
"traceback" module).
2019-08-15 17:30:50 +10:00
Jun Wu b152bbddd1 py: Define EMIT_MACHINE_CODE as EMIT_NATIVE || EMIT_INLINE_ASM.
The combination MICROPY_EMIT_NATIVE || MICROPY_EMIT_INLINE_ASM is used in
many places, so define a new macro for it.
2019-06-28 13:54:45 +10:00
Damien George a4f1d82757 py/nativeglue: Remove dependency on mp_fun_table in dyn-compiler mode.
mpy-cross uses MICROPY_DYNAMIC_COMPILER and MICROPY_EMIT_NATIVE but does
not actually need to execute native functions, and does not need
mp_fun_table.  This commit makes it so mp_fun_table and all its entries are
not built when MICROPY_DYNAMIC_COMPILER is enabled, significantly reducing
the size of the mpy-cross executable and allowing it to be built on more
machines/OS's.
2019-05-29 21:17:29 +10:00
Damien George 6f75c4f3cd all: Bump version to 1.11. 2019-05-29 16:38:10 +10:00
stijn 90fae9172a py/objarray: Add support for memoryview.itemsize attribute.
This allows figuring out the number of bytes in the memoryview object as
len(memview) * memview.itemsize.

The feature is enabled via MICROPY_PY_BUILTINS_MEMORYVIEW_ITEMSIZE and is
disabled by default.
2019-05-14 17:15:17 +10:00
Yonatan Goldschmidt ef9843653b extmod/moducryptolib: Add AES-CTR support.
Selectable at compile time via MICROPY_PY_UCRYPTOLIB_CTR.  Disabled by
default.
2019-05-06 18:09:48 +10:00
Damien George 02cc288edb py: Add independent config for debugging sentinel object values.
The new compile-time option is MICROPY_DEBUG_MP_OBJ_SENTINELS, disabled by
default.  This is to allow finer control of whether this debugging feature
is enabled or not (because, for example, this setting must be the same for
mpy-cross and the MicroPython main code when using native code generation).
2019-03-08 15:53:04 +11:00
Damien George 0779693c23 py/compile: Add optimisation to compile OrderedDict inplace.
This optimisation eliminates the need to create a temporary normal dict.
The optimisation is enabled via MICROPY_COMP_CONST_LITERAL which is enabled
by default (although only has an effect if OrderdDict is enabled).

Thanks to @pfalcon for the initial idea and implementation.
2019-03-01 15:22:46 +11:00
Yonatan Goldschmidt bc4f8b438b extmod/moduwebsocket: Refactor `websocket` to `uwebsocket`.
As mentioned in #4450, `websocket` was experimental with a single intended
user, `webrepl`. Therefore, we'll make this change without a weak
link `websocket` -> `uwebsocket`.
2019-02-14 00:35:45 +11:00
Yonatan Goldschmidt 343401c6df py/mpconfig.h: Fix comments mentioning dangling file and variable names. 2019-02-06 00:25:30 +11:00
Paul Sokolovsky 2f5d113fad py/warning: Support categories for warnings.
Python defines warnings as belonging to categories, where category is a
warning type (descending from exception type). This is useful, as e.g.
allows to disable warnings selectively and provide user-defined warning
types.  So, implement this in MicroPython, except that categories are
represented just with strings.  However, enough hooks are left to implement
categories differently per-port (e.g. as types), without need to patch each
and every usage.
2019-01-31 16:48:30 +11:00
stijn 42863830be py: Add optional support for 2-argument version of built-in next().
Configurable via MICROPY_PY_BUILTINS_NEXT2, disabled by default.
2019-01-27 13:01:28 +11:00
Sean Burton e33bc59712 py: Remove calls to file reader functions when these are disabled.
If MICROPY_PERSISTENT_CODE_LOAD or MICROPY_ENABLE_COMPILER are enabled then
code gets enabled that calls file reading functions which may be disabled
if no readers have been implemented.

To fix this, introduce a MICROPY_HAS_FILE_READER variable, which is
automatically set if MICROPY_READER_POSIX or MICROPY_READER_VFS is set but
can also be manually set if a custom reader is being implemented.  Then
disable the file reading calls if this is not set.
2019-01-27 11:08:25 +11:00
Damien George 3e25d611ef all: Bump version to 1.10. 2019-01-26 00:56:48 +11:00
Damien George aba83e66d7 py/mpconfig.h: Remove parentheses from MICROPY_VERSION_xxx macros.
Otherwise MICROPY_VERSION_STRING includes these parentheses in the string.
2019-01-26 00:44:35 +11:00
Damien George 6d19934463 py: Get optional VM stack overflow check compiling and working again.
Changes to the layout of the bytecode header meant that this debug code was
no longer compiling.  This is now fixed and a new compile-time option is
introduced, MICROPY_DEBUG_VM_STACK_OVERFLOW, to turn on this feature (which
is disabled by default).  This option is needed because more than one file
needs to cooperate to make this check work.
2019-01-04 17:09:41 +11:00