Commit Graph

249 Commits

Author SHA1 Message Date
Damien George 0c650d4276 py/vm: Simplify stack sentinel values for unwind return and jump.
This patch simplifies how sentinel values are stored on the stack when
doing an unwind return or jump.  Instead of storing two values on the stack
for an unwind jump it now stores only one: a negative small integer means
unwind-return and a non-negative small integer means unwind-jump with the
value being the number of exceptions to unwind.  The savings in code size
are:

   bare-arm:   -56
minimal x86:   -68
   unix x64:   -80
unix nanbox:    -4
      stm32:   -56
     cc3200:   -64
    esp8266:   -76
      esp32:  -156
2018-02-08 13:30:33 +11:00
Damien George 30fd8484eb py/runtime: Use the Python stack when building *arg and **kwarg state.
With MICROPY_ENABLE_PYSTACK enabled the following language constructs no
longer allocate on the heap: f(*arg), f(**kwarg).
2017-12-11 13:49:09 +11:00
Damien George 1e5a33df41 py: Convert all uses of alloca() to use new scoped allocation API. 2017-12-11 13:49:09 +11:00
Damien George 5b8998da6d py/runtime: Move mp_exc_recursion_depth to runtime and rename to raise.
For consistency this helper function is renamed to match the other
exception helpers, and moved to their location in runtime.c.
2017-12-11 13:49:09 +11:00
Paul Sokolovsky d72370def7 py/objfun, vm: Add comments on codestate allocation in stackless mode. 2017-12-09 11:01:34 +02:00
Damien George a3dc1b1957 all: Remove inclusion of internal py header files.
Header files that are considered internal to the py core and should not
normally be included directly are:
    py/nlr.h - internal nlr configuration and declarations
    py/bc0.h - contains bytecode macro definitions
    py/runtime0.h - contains basic runtime enums

Instead, the top-level header files to include are one of:
    py/obj.h - includes runtime0.h and defines everything to use the
        mp_obj_t type
    py/runtime.h - includes mpstate.h and hence nlr.h, obj.h, runtime0.h,
        and defines everything to use the general runtime support functions

Additional, specific headers (eg py/objlist.h) can be included if needed.
2017-10-04 12:37:50 +11:00
Damien George e2ba45c35f py/vm: Use lowercase letter at start of exception message.
For consistency with all the other exception messages.
2017-09-22 11:28:45 +10:00
Alexander Steffen 55f33240f3 all: Use the name MicroPython consistently in comments
There were several different spellings of MicroPython present in comments,
when there should be only one.
2017-07-31 18:35:40 +10:00
Damien George 016325dd0a py/vm: Make n_state variable local to just set-up part of VM.
It's not used anywhere else in the VM loop, and clashes with (is shadowed
by) the n_state variable that's redeclared towards the end of the
mp_execute_bytecode function.  Code size is unchanged.
2017-07-18 16:17:23 +10:00
Damien George 9ed5e80eea py/vm: Make "if" control flow more obvious in YIELD_FROM opcode. 2017-07-04 02:14:25 +10:00
Damien George a8a5d1e8c8 py: Provide mp_decode_uint_skip() to help reduce stack usage.
Taking the address of a local variable leads to increased stack usage, so
the mp_decode_uint_skip() function is added to reduce the need for taking
addresses.  The changes in this patch reduce stack usage of a Python call
by 8 bytes on ARM Thumb, by 16 bytes on non-windowing Xtensa archs, and by
16 bytes on x86-64.  Code size is also slightly reduced on most archs by
around 32 bytes.
2017-06-09 13:36:33 +10:00
Ville Skyttä ca16c38210 various: Spelling fixes 2017-05-29 11:36:05 +03:00
Damien George 04d05db27e py/vm: Fix bug with unwind jump popping the iterator from a for loop.
This patch fixes a regression introduced by
088740ecc4
2017-05-25 20:39:08 +10:00
Damien George 68e71eacb8 py/vm: Fix bug with stackless mode and unwinding of exceptions.
This patch fixes a regression introduced by
71a3d6ec3b

Previous to this patch the n_state variable was referring to that computed
at the very start of the mp_execute_bytecode function.  This patch fixes it
so that n_state is recomputed when the code_state changes.
2017-05-25 20:35:57 +10:00
Damien George dd11af209d py: Add LOAD_SUPER_METHOD bytecode to allow heap-free super meth calls.
This patch allows the following code to run without allocating on the heap:

    super().foo(...)

Before this patch such a call would allocate a super object on the heap and
then load the foo method and call it right away.  The super object is only
needed to perform the lookup of the method and not needed after that.  This
patch makes an optimisation to allocate the super object on the C stack and
discard it right after use.

Changes in code size due to this patch are:

   bare-arm: +128
    minimal: +232
   unix x64: +416
unix nanbox: +364
     stmhal: +184
    esp8266: +340
     cc3200: +128
2017-04-22 23:39:20 +10:00
Damien George a0973b09ce py/vm: Fix VM opcode tracing to print correct stack pointer.
Also const_table is now moved to the code_state->fun_bc structure.
2017-03-27 10:52:04 +11:00
Damien George 707f16b05c py: Use mp_locals/mp_globals accessor funcs instead of MP_STATE_CTX.
To improve maintainability of the code.
2017-03-24 18:41:11 +11:00
Damien George 60656eaea4 py: Define and use MP_OBJ_ITER_BUF_NSLOTS to get size of stack iter buf.
It improves readability of code and reduces the chance to make a mistake.

This patch also fixes a bug with nan-boxing builds by rounding up the
calculation of the new NSLOTS variable, giving the correct number of slots
(being 4) even if mp_obj_t is larger than the native machine size.
2017-03-23 16:36:08 +11:00
Damien George 1a5c8d1053 py/vm: Don't release the GIL if the scheduler is locked.
The scheduler being locked general means we are running a scheduled
function, and switching to another thread violates that, so don't switch in
such a case (even though we technically could).

And if we are running a scheduled function then we want to finish it ASAP,
so we shouldn't switch to another thread.

Furthermore, ports with threading enabled will lock the scheduler during a
hard IRQ, and this patch to the VM will make sure that threads are not
switched during a hard IRQ (which would crash the VM).
2017-03-20 18:42:27 +11:00
Damien George 6e74d24f30 py: Add micropython.schedule() function and associated runtime code. 2017-03-20 15:20:26 +11:00
Damien George 5640e6dacd py: Provide mp_decode_uint_value to help optimise stack usage.
This has a noticeable improvement on x86-64 and Thumb2 archs, where stack
usage is reduced by 2 machine words in the VM.
2017-03-17 16:50:19 +11:00
Damien George 71a3d6ec3b py: Reduce size of mp_code_state_t structure.
Instead of caching data that is constant (code_info, const_table and
n_state), store just a pointer to the underlying function object from which
this data can be derived.

This helps reduce stack usage for the case when the mp_code_state_t
structure is stored on the stack, as well as heap usage when it's stored
on the heap.

The downside is that the VM becomes a little more complex because it now
needs to derive the data from the underlying function object.  But this
doesn't impact the performance by much (if at all) because most of the
decoding of data is done outside the main opcode loop.  Measurements using
pystone show that little to no performance is lost.

This patch also fixes a nasty bug whereby the bytecode can be reclaimed by
the GC during execution.  With this patch there is always a pointer to the
function object held by the VM during execution, since it's stored in the
mp_code_state_t structure.
2017-03-17 16:39:13 +11:00
Damien George 088740ecc4 py: Optimise storage of iterator so it takes only 4 slots on Py stack. 2017-02-16 18:38:06 +11:00
Damien George 6e769da0da py: Make FOR_ITER opcode pop 1+4 slots from the stack when finished.
The extra 4 slots correspond to the iterator object stored on the stack.
2017-02-16 18:38:06 +11:00
Damien George f4df3aaa72 py: Allow bytecode/native to put iter_buf on stack for simple for loops.
So that the "for x in it: ..." statement can now work without using the
heap (so long as the iterator argument fits in an iter_buf structure).
2017-02-16 18:38:06 +11:00
Damien George ae8d867586 py: Add iter_buf to getiter type method.
Allows to iterate over the following without allocating on the heap:
- tuple
- list
- string, bytes
- bytearray, array
- dict (not dict.keys, dict.values, dict.items)
- set, frozenset

Allows to call the following without heap memory:
- all, any, min, max, sum

TODO: still need to allocate stack memory in bytecode for iter_buf.
2017-02-16 18:38:06 +11:00
Damien George 101886f529 py/vm: Convert mp_uint_t to size_t where appropriate. 2017-02-16 16:51:17 +11:00
Damien George f6c22a0679 py/vm: Add MICROPY_PY_THREAD_GIL_VM_DIVISOR option.
This improves efficiency of GIL release within the VM, by only doing the
release after a fixed number of jump-opcodes have executed in the current
thread.
2017-02-15 11:28:15 +11:00
Damien George cc4c1adf6e py/showbc: Make sure to set the const_table before printing bytecode. 2017-01-27 12:34:09 +11:00
Damien George 71fec076dc py/vm: Use MP_OBJ_FROM_PTR to cast a type to an object. 2016-09-27 13:21:23 +10:00
Damien George f040685b0c py: Only store the exception instance on Py stack in bytecode try block.
When an exception is raised and is to be handled by the VM, it is stored
on the Python value stack so the bytecode can access it.  CPython stores
3 objects on the stack for each exception: exc type, exc instance and
traceback.  uPy followed this approach, but it turns out not to be
necessary.  Instead, it is enough to store just the exception instance on
the Python value stack.  The only place where the 3 values are needed
explicitly is for the __exit__ handler of a with-statement context, but
for these cases the 3 values can be extracted from the single exception
instance.

This patch removes the need to store 3 values on the stack, and instead
just stores the exception instance.

Code size is reduced by about 50-100 bytes, the compiler and VM are
slightly simpler, generate bytecode is smaller (by 2 bytes for each try
block), and the Python value stack is reduced in size for functions that
handle exceptions.
2016-09-27 12:37:21 +10:00
Damien George adaf0d865c py: Combine 3 comprehension opcodes (list/dict/set) into 1.
With the previous patch combining 3 emit functions into 1, it now makes
sense to also combine the corresponding VM opcodes, which is what this
patch does.  This eliminates 2 opcodes which simplifies the VM and reduces
code size, in bytes: bare-arm:44, minimal:64, unix(NDEBUG,x86-64):272,
stmhal:92, esp8266:200.  Profiling (with a simple script that creates many
list/dict/set comprehensions) shows no measurable change in performance.
2016-09-19 12:28:03 +10:00
Damien George 581a59a456 py: Rename struct mp_code_state to mp_code_state_t.
Also at _t to mp_exc_stack pre-declaration in struct typedef.
2016-08-27 23:21:00 +10:00
Damien George 4cec63a9db py: Implement a simple global interpreter lock.
This makes the VM/runtime thread safe, at the cost of not being able to
run code in parallel.
2016-06-28 11:28:50 +01:00
Paul Sokolovsky eff85bb1dc py/vm: "yield from" didn't handle MP_OBJ_STOP_ITERATION optimization.
E.g. crashed when yielding from already stopped generators.
2016-04-28 02:08:43 +03:00
Damien George 40d8430ee3 py/vm: Add macros to hook into various points in the VM.
These can be used to insert arbitrary checks, polling, etc into the VM.
They are left general because the VM is a highly tuned loop and it should
be up to a given port how that port wants to modify the VM internals.

One common use would be to insert a polling check, but only done after
a certain number of opcodes were executed, so as not to slow down the VM
too much.  For example:

 #define MICROPY_VM_HOOK_COUNT (30)
 #define MICROPY_VM_HOOK_INIT static uint vm_hook_divisor = MICROPY_VM_HOOK_COUNT
 #define MICROPY_VM_HOOK_POLL if (--vm_hook_divisor == 0) { \
     vm_hook_divisor = MICROPY_VM_HOOK_COUNT;
     extern void vm_hook_function(void);
     vm_hook_function();
 }
 #define MICROPY_VM_HOOK_LOOP MICROPY_VM_HOOK_POLL
 #define MICROPY_VM_HOOK_RETURN MICROPY_VM_HOOK_POLL
2016-02-17 09:02:19 +00:00
Damien George 93bb7dffd2 py/vm: Fix popping of exception block in UNWIND_JUMP opcode.
Fixes issue #1812.
2016-02-01 16:07:21 +00:00
Damien George 3d2daa2d03 py: Change exception traceback data to use size_t instead of mp_uint_t.
The traceback array stores qstrs and line numbers.  qstrs are typed as
size_t, and line numbers should safely fit in size_t as well.
2016-01-02 22:04:12 +00:00
Damien George 8047340d75 py: Handle case of return within the finally block of try-finally.
Addresses issue #1636.
2015-12-24 12:47:39 +00:00
Damien George 7a30e87d2b py: Fix MICROPY_STACKLESS mode to compile with MICROPY_OBJ_REPR_D. 2015-12-17 12:32:41 +00:00
Damien George bdbe8c9ae2 py: Make UNARY_OP_NOT a first-class op, to agree with Py not semantics.
Fixes #1684 and makes "not" match Python semantics.  The code is also
simplified (the separate MP_BC_NOT opcode is removed) and the patch saves
68 bytes for bare-arm/ and 52 bytes for minimal/.

Previously "not x" was implemented as !mp_unary_op(x, MP_UNARY_OP_BOOL),
so any given object only needs to implement MP_UNARY_OP_BOOL (and the VM
had a special opcode to do the ! bit).

With this patch "not x" is implemented as mp_unary_op(x, MP_UNARY_OP_NOT),
but this operation is caught at the start of mp_unary_op and dispatched as
!mp_obj_is_true(x).  mp_obj_is_true has special logic to test for
truthness, and is the correct way to handle the not operation.
2015-12-10 22:19:48 +00:00
Damien George 999cedb90f py: Wrap all obj-ptr conversions in MP_OBJ_TO_PTR/MP_OBJ_FROM_PTR.
This allows the mp_obj_t type to be configured to something other than a
pointer-sized primitive type.

This patch also includes additional changes to allow the code to compile
when sizeof(mp_uint_t) != sizeof(void*), such as using size_t instead of
mp_uint_t, and various casts.
2015-11-29 14:25:35 +00:00
Damien George c8e9c0d89a py: Add MICROPY_PERSISTENT_CODE so code can persist beyond the runtime.
Main changes when MICROPY_PERSISTENT_CODE is enabled are:

- qstrs are encoded as 2-byte fixed width in the bytecode
- all pointers are removed from bytecode and put in const_table (this
  includes const objects and raw code pointers)

Ultimately this option will enable persistence for not just bytecode but
also native code.
2015-11-13 12:49:18 +00:00
Damien George 4bf3f2d3c0 py: Fix with+for+return bug by popping for-iter when unwinding exc stack.
Addresses issue #1182.
2015-10-15 17:48:28 +01:00
Paul Sokolovsky 2ff2ea5f3b vm: Handle "raise X from Y" statements the best way we can.
By issuing a warning that exception chaining is not supported, and ignoring
"from Y" argument.
2015-09-01 10:39:04 +03:00
Damien George 59fba2d6ea py: Remove mp_load_const_bytes and instead load precreated bytes object.
Previous to this patch each time a bytes object was referenced a new
instance (with the same data) was created.  With this patch a single
bytes object is created in the compiler and is loaded directly at execute
time as a true constant (similar to loading bignum and float objects).
This saves on allocating RAM and means that bytes objects can now be
used when the memory manager is locked (eg in interrupts).

The MP_BC_LOAD_CONST_BYTES bytecode was removed as part of this.

Generated bytecode is slightly larger due to storing a pointer to the
bytes object instead of the qstr identifier.

Code size is reduced by about 60 bytes on Thumb2 architectures.
2015-06-25 14:42:13 +00:00
Damien George ed570e4b2a py: Remove mp_load_const_str and replace uses with inlined version. 2015-06-25 13:58:41 +00:00
Damien George c5029bcbf3 py: Add MP_BINARY_OP_DIVMOD to simplify and consolidate divmod builtin. 2015-06-13 23:36:30 +01:00
Damien George c2a4e4effc py: Convert hash API to use MP_UNARY_OP_HASH instead of ad-hoc function.
Hashing is now done using mp_unary_op function with MP_UNARY_OP_HASH as
the operator argument.  Hashing for int, str and bytes still go via
fast-path in mp_unary_op since they are the most common objects which
need to be hashed.

This lead to quite a bit of code cleanup, and should be more efficient
if anything.  It saves 176 bytes code space on Thumb2, and 360 bytes on
x86.

The only loss is that the error message "unhashable type" is now the
more generic "unsupported type for __hash__".
2015-05-12 22:46:02 +01:00
Paul Sokolovsky 6738c1dded vm: Properly handle StopIteration raised in user instance iterator.
I.e. in bytecode Python functions.
2015-05-11 23:57:42 +01:00