Commit Graph

3510 Commits

Author SHA1 Message Date
Damien George 870e900d02 py: Introduce and use constants for multi-opcode sizes. 2019-09-26 15:27:11 +10:00
Damien George 1f7202d122 py/bc: Replace big opcode format table with simple macro. 2019-09-26 15:27:11 +10:00
Damien George 5889cf58db py/bc0: Order opcodes into groups based on their size and format. 2019-09-26 15:27:10 +10:00
Damien George 2069c563f9 py: Add support for matmul operator @ as per PEP 465.
To make progress towards MicroPython supporting Python 3.5, adding the
matmul operator is important because it's a really "low level" part of the
language, being a new token and modifications to the grammar.

It doesn't make sense to make it configurable because 1) it would make the
grammar and lexer complicated/messy; 2) no other operators are
configurable; 3) it's not a feature that can be "dynamically plugged in"
via an import.

And matmul can be useful as a general purpose user-defined operator, it
doesn't have to be just for numpy use.

Based on work done by Jim Mussared.
2019-09-26 15:12:39 +10:00
Damien George 14e203282a py/compile: Use calculation instead of switch to convert token to op. 2019-09-26 14:37:26 +10:00
Damien George 9bf2feba63 py/parse: Use calculation instead of table to convert token to operator. 2019-09-26 14:37:26 +10:00
Damien George 6ce7c051e8 py/lexer: Reorder operator tokens to match corresponding binary ops. 2019-09-26 14:37:26 +10:00
stijn 22131a6738 py/persistentcode: Enable persistent code saving for Windows ports. 2019-09-18 22:15:36 +10:00
Damien George 6e07fde895 py/mkrules.mk: Add QSTR_GLOBAL_REQUIREMENTS variable for qstr auto-gen. 2019-09-12 18:16:16 +10:00
Damien George ea060a42e9 py/vm: Factor cached map lookup code to inline function.
To reduce code duplication and allow to more easily modify this function.
2019-09-10 11:23:52 +10:00
Damien George fb73bdae62 py/mkenv.mk: Add GDB variable. 2019-09-05 22:59:06 +10:00
Damien George b29fae0c56 py/bc: Fix size calculation of UNWIND_JUMP opcode in mp_opcode_format.
Prior to this patch mp_opcode_format would calculate the incorrect size of
the MP_BC_UNWIND_JUMP opcode, missing the additional byte.  But, because
opcodes below 0x10 are unused and treated as bytes in the .mpy load/save
and freezing code, this bug did not show any symptoms, since nested unwind
jumps would rarely (if ever) reach a depth of 16 (so the extra byte of this
opcode would be between 0x01 and 0x0f and be correctly loaded/saved/frozen
simply as an undefined opcode).

This patch fixes this bug by correctly accounting for the additional byte.
        .
2019-09-02 13:30:16 +10:00
Damien George c348e79187 py/binary: Change mp_uint_t to size_t for index, size, align args.
Reduces code size for nan-box builds, otherwise changes nothing.
2019-09-02 13:14:27 +10:00
Damien George 24c3e9b283 py/modstruct: Fix struct.pack_into with unaligned offset of native type.
Following the same fix for unpack.
2019-09-02 13:14:16 +10:00
Tom McDermott 1022f9cc35 py/modstruct: Fix struct.unpack with unaligned offset of native type.
With this patch alignment is done relative to the start of the buffer that
is being unpacked, not the raw pointer value, as per CPython.

Fixes issue #3314.
2019-09-02 13:10:55 +10:00
Jeff Epler 12f13ee634 py/objtuple: Allow compatible subclasses of tuple in mp_obj_tuple_get.
As part of this patch a private macro mp_obj_is_tuple_compatible is
introduced to encapsulate the check, which is used in two locations.

Fixes #5005.
2019-08-31 00:14:11 +10:00
Milan Rossa b295df4b08 py/profile: Add debugging for sys.settrace feature. 2019-08-30 16:48:47 +10:00
Milan Rossa 310b3d1b81 py: Integrate sys.settrace feature into the VM and runtime.
This commit adds support for sys.settrace, allowing to install Python
handlers to trace execution of Python code.  The interface follows CPython
as closely as possible.  The feature is disabled by default and can be
enabled via MICROPY_PY_SYS_SETTRACE.
2019-08-30 16:44:12 +10:00
Milan Rossa c96aedad46 py/profile: Add initial implementation of sys.settrace feature. 2019-08-30 16:43:56 +10:00
Damien George dbf35d3da3 py/bc: Factor out code to get bytecode line number info into new func. 2019-08-30 16:43:46 +10:00
Damien George c7c6703950 py/compile: Improve the line numbering precision for lambdas.
Prior to this patch the line number for a lambda would be "line 1" if the
body of the lambda contained only a simple expression (with no line number
stored in the parse node).  Now the line number is always reported
correctly.
2019-08-30 16:43:46 +10:00
Damien George af20c2ead3 py: Add global default_emit_opt variable to make emit kind persistent.
mp_compile no longer takes an emit_opt argument, rather this setting is now
provided by the global default_emit_opt variable.

Now, when -X emit=native is passed as a command-line option, the emitter
will be set for all compiled modules (included imports), not just the
top-level script.

In the future there could be a way to also set this variable from a script.

Fixes issue #4267.
2019-08-28 12:47:58 +10:00
Damien George 08c1fe5569 py/vm: Don't add traceback info for exceptions that are re-raised.
With this patch exceptions that are re-raised have improved tracebacks
(less confusing, match CPython), and it makes re-raise slightly more
efficient (in time and RAM) because they no longer need to add a traceback.
Also general VM performance is not measurably affected.

Partially fixes issue #2928.
2019-08-28 12:31:53 +10:00
Damien George 16f6169c88 py/vm: Don't add traceback info for exc's propagated through a finally.
With this patch exception tracebacks that go through a finally are improved
(less confusing, match CPython), and it makes finally's slightly more
efficient (in time and RAM) because they no longer need to add a traceback.

Partially fixes issue #2928.
2019-08-28 12:31:49 +10:00
Damien George 3d7455a0bb py/py.mk: Remove trailing spaces at end of line. 2019-08-27 16:37:43 +10:00
Damien George 2fca0d7f18 py/vm: Shorten error message for not-implemented opcode.
It's really an opcode that's not implemented, so use "opcode" instead of
"byte code".  And remove the redundant "not implemented" text because that
is already implied by the exception type.  There's no need to have a long
error message for an exception that is almost never encountered.  Saves
about 20 bytes of code size on most ports.
2019-08-22 16:13:05 +10:00
Damien George 53527138a9 py/bc0.h: Add comment that MP_BC_MAKE_CLOSURE/_DEFARGS take extra byte. 2019-08-22 16:07:28 +10:00
Damien George 973c87d8fa py/objgenerator: Move defn of mp_const_GeneratorExit_obj here.
Because the mp_obj_exception_t type is now globally available.
2019-08-22 16:05:05 +10:00
Damien George bc9b656f35 py/runtime: Remove obsolete comment about mp_parse_compile_execute.
mp_locals_get/set and mp_globals_get/set are now static-inline functions so
this comment is no longer correct.
2019-08-22 15:59:14 +10:00
Damien George 8e7745eb31 py/emitbc: Make all emit_write_bytecode_* funcs take a stack_adj arg.
This factoring of code gives significant code-size savings:

   bare-arm:  -456 -0.682%
minimal x86:  -844 -0.547%
   unix x64:  -472 -0.095%
unix nanbox: -1348 -0.303%
      stm32:  -472 -0.130% PYBV10
     cc3200:  -448 -0.242%
    esp8266:  -708 -0.108%
      esp32:  -400 -0.036% GENERIC
        nrf:  -520 -0.356% pca10040
       samd:  -456 -0.448% ADAFRUIT_ITSYBITSY_M4_EXPRESS
2019-08-22 15:32:26 +10:00
Damien George 3d9bd80447 py/emitbc: Rewrite switch in load_const_tok to reduce code size. 2019-08-22 15:10:25 +10:00
Milan Rossa ae6fe8b43c py/compile: Improve the line numbering precision for comprehensions.
The line number for comprehensions is now always reported as the correct
global location in the script, instead of just "line 1".
2019-08-19 23:50:30 +10:00
Damien George 7d851a27f1 extmod/modure: Make regex dump-code debugging feature optional.
Enabled via MICROPY_PY_URE_DEBUG, disabled by default (but enabled on unix
coverage build).  This is a rarely used feature that costs a lot of code
(500-800 bytes flash).  Debugging of regular expressions can be done
offline with other tools.
2019-08-19 16:43:00 +10:00
Damien George 11ecdf2ec6 py/nlr: Use MP_UNREACHABLE at the end of arch-specific nlr_jump funcs.
Recent versions of gcc perform optimisations which can lead to the
following code from the MP_NLR_JUMP_HEAD macro being omitted:

    top->ret_val = val; \
    MP_NLR_RESTORE_PYSTACK(top); \
    *_top_ptr = top->prev; \

This is noticeable (at least) in the unix coverage on x86-64 built with gcc
9.1.0.  This is because the nlr_jump function is marked as no-return, so
gcc deduces that the above code has no effect.

Adding MP_UNREACHABLE tells the compiler that the asm code may branch
elsewhere, and so it cannot optimise away the code.
2019-08-19 16:14:57 +10:00
Damien George 0c80cb39af py: Introduce MP_UNREACHABLE macro to annotate unreachable code.
And use it to replace the same pattern at the end of nlrthumb.c:nlr_jump.
2019-08-19 16:14:33 +10:00
stijn af5c998f37 py/modmath: Implement math.isclose() for non-complex numbers.
As per PEP 485, this function appeared in for Python 3.5.  Configured via
MICROPY_PY_MATH_ISCLOSE which is disabled by default, but enabled for the
ports which already have MICROPY_PY_MATH_SPECIAL_FUNCTIONS enabled.
2019-08-17 23:23:17 +10:00
Damien George acfbb9febd py/objarray: Fix amount of free space in array when doing slice assign.
Prior to this patch the amount of free space in an array (including
bytearray) was not being maintained correctly for the case of slice
assignment which changed the size of the array.  Under certain cases (as
encoded in the new test) it was possible that the array could grow beyond
its allocated memory block and corrupt the heap.

Fixes issue #4127.
2019-08-15 23:02:04 +10:00
Milan Rossa cb3647004f py: Implement new sys.atexit feature.
This patch implements a new sys.atexit function which registers a function
that is later executed when the main script ends.  It is configurable via
MICROPY_PY_SYS_ATEXIT, disabled by default.

This is not compliant with CPython, rather it can be used to implement a
CPython compatible "atexit" module if desired (similar to how
sys.print_exception can be used to implement functionality of the
"traceback" module).
2019-08-15 17:30:50 +10:00
Milan Rossa efdcd6baa7 py/showbc: Fix off-by-one when showing address of unknown opcode. 2019-08-06 16:08:39 +10:00
Damien George cd35dd9d9a py: Allow to pass in read-only buffers to viper and inline-asm funcs.
Fixes #4936.
2019-08-06 15:58:23 +10:00
Paul m. p. P f60229e261 py/modio: Call mp_import_name to do resource stream import.
So code is not duplicated and it can take advantage of __import__ being
overridden.
2019-07-31 22:37:44 +10:00
Paul m. p. P 60f1063797 py/runtime: Allow to override builtins.__import__ with Python func.
This patch adds a simple but powerful hook into the import system, in a
CPython compatible way, by allowing to override builtins.__import__.

This does introduce some overhead to all imports but it's minor:
- the dict lookup of __import__ is bypassed if there are no modifications
  to the builtins module (which is the case at start up);
- imports are not performance critical, usually done just at the start of a
  script;
- compared to how much work is done in an import, looking up a value in a
  dict is a relatively small additional piece of work.
2019-07-31 22:36:00 +10:00
Paul m. p. P a8e3201b37 py/builtinimport: Populate __file__ when importing frozen or mpy files.
Note that bytecode already includes the source filename as a qstr so there
is no additional memory used by the interning operation here.
2019-07-31 17:00:11 +10:00
Eric Poulsen 01054f2092 py/objdict: Quote non-string types when used as keys in JSON output.
JSON requires that keys of objects be strings.  CPython will therefore
automatically quote simple types (NoneType, bool, int, float) when they are
used directly as keys in JSON output.  To prevent subtle bugs and emit
compliant JSON, MicroPython should at least test for such keys so they
aren't silently let through.  Then doing the actual quoting is a similar
cost to raising an exception, so that's what is implemented by this patch.

Fixes issue #4790.
2019-07-30 16:34:27 +10:00
Yonatan Goldschmidt e9593d5075 py/sequence: Fix grammar in comment about equality. 2019-07-25 16:37:25 +10:00
Paul m. p. P 3b3a4749ce py/objstringio: Guard bytesio_stream_p struct w/ MICROPY_PY_IO_BYTESIO.
It's static and can lead to a compilation warning/error when
MICROPY_PY_IO_BYTESIO is disabled.
2019-07-17 16:12:19 +10:00
Jim Mussared bc66fe9064 py/scheduler: Rename sched_stack to sched_queue.
Behaviour was changed from stack to queue in
8977c7eb58, and this updates variable names
to match.  Also updates other references (docs, error messages).
2019-07-17 16:09:32 +10:00
Paul m. p. P 154062d9cb py/makeqstrdata.py: Allow using \r\n as a qstr if a port requires it. 2019-07-12 17:10:11 +10:00
David Lechner 82dc9856b0 py/asmarm: Use __builtin___clear_cache instead of __clear_cache.
__clear_cache causes a compile error when using clang.  Instead use
__builtin___clear_cache which is available under both gcc and clang.

Also replace tabs with spaces in this section of code (introduced by a
previous commit).
2019-07-12 16:48:37 +10:00
Laurens Valk a73859d5af py/objgenerator: Add missing #if guard for PY_GENERATOR_PEND_THROW.
Without it, gen_instance_pend_throw_obj is defined but not used when
MICROPY_PY_GENERATOR_PEND_THROW is set to 0.
2019-07-09 13:24:23 +10:00