micropython/py
Damien George a3de776486 py/emitnative: Optimise and improve exception handling in native code.
Prior to this patch, native code would use a full nlr_buf_t for each
exception handler (try-except, try-finally, with).  For nested exception
handlers this would use a lot of C stack and be rather inefficient.

This patch changes how exceptions are handled in native code by setting up
only a single nlr_buf_t context for the entire function, and then manages a
state machine (using the PC) to work out which exception handler to run
when an exception is raised by an nlr_jump.  This keeps the C stack usage
at a constant level regardless of the depth of Python exception blocks.

The patch also fixes an existing bug when local variables are written to
within an exception handler, then their value was incorrectly restored if
an exception was raised (since the nlr_jump would restore register values,
back to the point of the nlr_push).

And it also gets nested try-finally+with working with the viper emitter.

Broadly speaking, efficiency of executing native code that doesn't use
any exception blocks is unchanged, and emitted code size is only slightly
increased for such function.  C stack usage of all native functions is
either equal or less than before.  Emitted code size for native functions
that use exception blocks is increased by roughly 10% (due in part to
fixing of above-mentioned bugs).

But, most importantly, this patch allows to implement more Python features
in native code, like unwind jumps and yielding from within nested exception
blocks.
2018-08-16 13:56:36 +10:00
..
argcheck.c py/argcheck: Remove #if guard around terse error message helper func. 2017-10-19 18:57:26 +11:00
asmarm.c py/asm*: Support assembling code to jump to a register, and get PC+off. 2018-08-16 13:45:24 +10:00
asmarm.h py/asm*: Support assembling code to jump to a register, and get PC+off. 2018-08-16 13:45:24 +10:00
asmbase.c py/asmbase: Revert removal of clearing of label offsets for native emit. 2017-12-08 19:07:00 +11:00
asmbase.h py/asm: Remove need for dummy_data when doing initial assembler passes. 2016-12-09 22:50:58 +11:00
asmthumb.c py/asm*: Support assembling code to jump to a register, and get PC+off. 2018-08-16 13:45:24 +10:00
asmthumb.h py/asm*: Support assembling code to jump to a register, and get PC+off. 2018-08-16 13:45:24 +10:00
asmx64.c py/asm*: Support assembling code to jump to a register, and get PC+off. 2018-08-16 13:45:24 +10:00
asmx64.h py/asm*: Support assembling code to jump to a register, and get PC+off. 2018-08-16 13:45:24 +10:00
asmx86.c py/asm*: Support assembling code to jump to a register, and get PC+off. 2018-08-16 13:45:24 +10:00
asmx86.h py/asm*: Support assembling code to jump to a register, and get PC+off. 2018-08-16 13:45:24 +10:00
asmxtensa.c py/asm*: Support assembling code to jump to a register, and get PC+off. 2018-08-16 13:45:24 +10:00
asmxtensa.h py/asm*: Support assembling code to jump to a register, and get PC+off. 2018-08-16 13:45:24 +10:00
bc.c py/bc: Update opcode_format_table to match the bytecode. 2017-10-10 10:37:38 +11:00
bc.h all: Remove inclusion of internal py header files. 2017-10-04 12:37:50 +11:00
bc0.h py: Clean up unary and binary enum list to keep groups together. 2017-10-05 10:49:44 +11:00
binary.c py/objstr: Remove "make_qstr_if_not_already" arg from mp_obj_new_str. 2017-11-16 13:17:51 +11:00
binary.h py/binary: Change internal bytearray typecode from 0 to 1. 2017-08-17 16:19:35 +10:00
builtin.h extmod/moducryptolib: Add ucryptolib module with crypto functions. 2018-06-27 14:54:40 +10:00
builtinevex.c all: Remove inclusion of internal py header files. 2017-10-04 12:37:50 +11:00
builtinhelp.c py: Simplify some cases of accessing the map of module and type dict. 2018-07-08 21:31:09 +10:00
builtinimport.c py/builtinimport: Add compile-time option to disable external imports. 2018-02-20 18:00:44 +11:00
compile.c py/emitnative: Optimise and improve exception handling in native code. 2018-08-16 13:56:36 +10:00
compile.h all: Use the name MicroPython consistently in comments 2017-07-31 18:35:40 +10:00
emit.h py/emitnative: Optimise and improve exception handling in native code. 2018-08-16 13:56:36 +10:00
emitbc.c py/emit: Combine setup with/except/finally into one emit function. 2018-05-23 00:35:16 +10:00
emitcommon.c py/emit: Combine name and global into one func for load/store/delete. 2018-05-23 00:22:47 +10:00
emitglue.c py: Fix compiling with debug enabled and make more use of DEBUG_printf. 2018-08-02 14:17:24 +10:00
emitglue.h py/emitglue: When assigning bytecode only pass bytecode len if needed. 2018-02-14 18:41:17 +11:00
emitinlinethumb.c all: Use the name MicroPython consistently in comments 2017-07-31 18:35:40 +10:00
emitinlinextensa.c py/emitinline: Move common code for end of final pass to compiler. 2016-12-09 21:23:17 +11:00
emitnarm.c py/emitnative: Optimise and improve exception handling in native code. 2018-08-16 13:56:36 +10:00
emitnative.c py/emitnative: Optimise and improve exception handling in native code. 2018-08-16 13:56:36 +10:00
emitnthumb.c py/emitnative: Optimise and improve exception handling in native code. 2018-08-16 13:56:36 +10:00
emitnx64.c py/emitnative: Optimise and improve exception handling in native code. 2018-08-16 13:56:36 +10:00
emitnx86.c py/emitnative: Optimise and improve exception handling in native code. 2018-08-16 13:56:36 +10:00
emitnxtensa.c py/emitnative: Optimise and improve exception handling in native code. 2018-08-16 13:56:36 +10:00
formatfloat.c py/formatfloat: Fix case where floats could render with negative digits. 2018-03-01 17:00:02 +11:00
formatfloat.h all: Use the name MicroPython consistently in comments 2017-07-31 18:35:40 +10:00
frozenmod.c all: Use the name MicroPython consistently in comments 2017-07-31 18:35:40 +10:00
frozenmod.h all: Use the name MicroPython consistently in comments 2017-07-31 18:35:40 +10:00
gc.c py/gc: In gc_alloc, reset n_free var right before search for free mem. 2018-08-14 16:11:21 +10:00
gc.h py/gc: Add gc_sweep_all() function to run all remaining finalisers. 2018-06-12 11:55:29 +10:00
grammar.h py/compile: Combine expr, xor_expr and and_expr into one function. 2018-06-22 17:00:29 +10:00
lexer.c py/lexer: Add support for underscores in numeric literals. 2018-06-12 12:17:43 +10:00
lexer.h all: Use the name MicroPython consistently in comments 2017-07-31 18:35:40 +10:00
makeqstrdata.py py/modbuiltins: Make built-in dir support the __dir__ special method. 2018-05-10 23:14:23 +10:00
makeqstrdefs.py py/makeqstrdefs.py: Optimise by using compiled re's so it runs faster. 2018-03-16 23:54:06 +11:00
makeversionhdr.py py: Use "GEN" consistently for describing files generated in the build. 2018-02-22 12:48:51 +11:00
malloc.c py/malloc: Give a compile warning if using finaliser without GC. 2018-07-09 14:40:02 +10:00
map.c py: Fix compiling with debug enabled and make more use of DEBUG_printf. 2018-08-02 14:17:24 +10:00
misc.h py/misc.h: Add MP_STATIC_ASSERT macro to do static assertions. 2018-05-18 23:31:00 +10:00
mkenv.mk py/mkenv.mk: Use $(PYTHON) consistently when calling Python tools. 2017-11-15 11:56:58 +11:00
mkrules.mk py/mkrules.mk: Regenerate all qstrs when config files change. 2018-06-12 13:53:43 +10:00
modarray.c all: Use the name MicroPython consistently in comments 2017-07-31 18:35:40 +10:00
modbuiltins.c py: Add checks for stream objects in print() and sys.print_exception(). 2018-06-20 15:57:10 +10:00
modcmath.c py: Change obsolete "///" comment formatting to normal comments. 2017-08-30 21:02:00 +10:00
modcollections.c py/objdeque: Implement ucollections.deque type with fixed size. 2018-02-21 22:39:25 +11:00
modgc.c py: Change obsolete "///" comment formatting to normal comments. 2017-08-30 21:02:00 +10:00
modio.c py/modio: Add uio.IOBase class to allow to define user streams. 2018-06-12 12:29:26 +10:00
modmath.c py/modmath: Convert log2 macro into a function. 2017-10-10 16:01:04 +11:00
modmicropython.c py: Don't include mp_optimise_value or opt_level() if compiler disabled. 2018-04-04 14:24:03 +10:00
modstruct.c py/modstruct: Check and prevent buffer-write overflow in struct packing. 2017-09-01 11:11:09 +10:00
modsys.c py: Add checks for stream objects in print() and sys.print_exception(). 2018-06-20 15:57:10 +10:00
modthread.c py: Introduce a Python stack for scoped allocation. 2017-12-11 13:49:09 +11:00
moduerrno.c py/moduerrno: Make list of errno codes configurable. 2017-02-22 12:58:11 +11:00
mpconfig.h py/mpconfig.h: Introduce MICROPY_DEBUG_PRINTER for debugging output. 2018-08-02 14:04:44 +10:00
mperrno.h py/mperrno: Define MP_EWOULDBLOCK as EWOULDBLOCK, not EAGAIN. 2018-05-01 15:53:25 +10:00
mphal.h all: Use the name MicroPython consistently in comments 2017-07-31 18:35:40 +10:00
mpprint.c py/mpprint: Fix "%x" vs "%X" regression introduced in previous commit. 2017-12-07 10:31:14 +02:00
mpprint.h all: Use the name MicroPython consistently in comments 2017-07-31 18:35:40 +10:00
mpstate.c all: Use the name MicroPython consistently in comments 2017-07-31 18:35:40 +10:00
mpstate.h extmod/uos_dupterm: Use native C stream methods on dupterm object. 2018-06-12 15:06:11 +10:00
mpthread.h all: Unify header guard usage. 2017-07-18 11:57:39 +10:00
mpz.c py/mpz: Avoid undefined behavior at integer overflow in mpz_hash. 2018-05-21 12:48:26 +10:00
mpz.h py/mpz: Simplify handling of borrow and quo adjustment in mpn_div. 2017-12-29 14:05:48 +11:00
nativeglue.c py/emit: Merge build set/slice into existing build emit function. 2018-05-23 00:23:36 +10:00
nlr.c py/nlr: Fix missing trailing characters in comments in nlr.c 2017-12-29 22:24:53 +11:00
nlr.h py/nlr: Fix nlr functions for 64bit ports built with gcc on Windows 2017-12-29 22:24:46 +11:00
nlrsetjmp.c py/nlr: Factor out common NLR code to macro and generic funcs in nlr.c. 2017-12-28 16:46:30 +11:00
nlrthumb.c py/nlrthumb: Fix Clang support wrt use of "return 0". 2018-04-27 15:10:42 +10:00
nlrx64.c py/nlr: Fix nlr functions for 64bit ports built with gcc on Windows 2017-12-29 22:24:46 +11:00
nlrx86.c py/nlrx86: Use naked attribute on nlr_push for gcc 8.0 and higher. 2018-05-15 11:17:28 +10:00
nlrxtensa.c py/nlr: Factor out common NLR code to macro and generic funcs in nlr.c. 2017-12-28 16:46:30 +11:00
obj.c all: Remove inclusion of internal py header files. 2017-10-04 12:37:50 +11:00
obj.h py/obj.h: Give compile error if using obj repr D with single-prec float. 2018-07-08 23:45:05 +10:00
objarray.c py/objarray: Allow to build again when bytearray is disabled. 2018-08-14 16:23:21 +10:00
objarray.h py/objarray: Replace 0x80 with new MP_OBJ_ARRAY_TYPECODE_FLAG_RW macro. 2018-06-18 13:40:53 +10:00
objattrtuple.c all: Use the name MicroPython consistently in comments 2017-07-31 18:35:40 +10:00
objbool.c all: Remove inclusion of internal py header files. 2017-10-04 12:37:50 +11:00
objboundmeth.c py: Convert all uses of alloca() to use new scoped allocation API. 2017-12-11 13:49:09 +11:00
objcell.c all: Use the name MicroPython consistently in comments 2017-07-31 18:35:40 +10:00
objclosure.c all: Use the name MicroPython consistently in comments 2017-07-31 18:35:40 +10:00
objcomplex.c all: Remove inclusion of internal py header files. 2017-10-04 12:37:50 +11:00
objdeque.c py/objdeque: Fix sign extension bug when computing len of deque object. 2018-05-11 13:44:50 +10:00
objdict.c py/objdict: Make mp_obj_dict_get_map an inline function. 2018-07-08 22:27:05 +10:00
objenumerate.c all: Use the name MicroPython consistently in comments 2017-07-31 18:35:40 +10:00
objexcept.c py/objexcept: Make MP_DEFINE_EXCEPTION public so ports can define excs. 2018-03-17 00:31:40 +11:00
objexcept.h py/objexcept: Make MP_DEFINE_EXCEPTION public so ports can define excs. 2018-03-17 00:31:40 +11:00
objfilter.c all: Use the name MicroPython consistently in comments 2017-07-31 18:35:40 +10:00
objfloat.c py/objfloat: Fix undefined integer behavior hashing negative zero. 2018-05-21 12:49:56 +10:00
objfun.c py: Fix compiling with debug enabled and make more use of DEBUG_printf. 2018-08-02 14:17:24 +10:00
objfun.h py/objgenerator: Implement __name__ with normal fun attr accessor code. 2018-07-10 16:33:57 +10:00
objgenerator.c py/objgenerator: Implement __name__ with normal fun attr accessor code. 2018-07-10 16:33:57 +10:00
objgenerator.h all: Use the name MicroPython consistently in comments 2017-07-31 18:35:40 +10:00
objgetitemiter.c all: Remove inclusion of internal py header files. 2017-10-04 12:37:50 +11:00
objint.c py/objint: Remove unreachable code checking for int type in format func. 2018-03-02 11:01:24 +11:00
objint.h py/modbuiltins: Implement abs() by dispatching to MP_UNARY_OP_ABS. 2017-09-18 00:06:43 +03:00
objint_longlong.c py/objint: Simplify LHS arg type checking in int binary op functions. 2018-04-05 01:11:26 +10:00
objint_mpz.c py/objint: Simplify LHS arg type checking in int binary op functions. 2018-04-05 01:11:26 +10:00
objlist.c all: Use NULL instead of "" when calling mp_raise exception helpers. 2017-10-24 22:39:36 +11:00
objlist.h py/obj.h: Move declaration of mp_obj_list_init to objlist.h. 2018-03-13 14:03:15 +11:00
objmap.c all: Use the name MicroPython consistently in comments 2017-07-31 18:35:40 +10:00
objmodule.c py/objmodule: Make mp_obj_module_get_globals an inline function. 2018-07-08 22:27:39 +10:00
objmodule.h py/objmodule: Factor common code for calling __init__ on builtin module. 2018-02-20 17:56:58 +11:00
objnamedtuple.c py/objnamedtuple: Allow to reuse namedtuple basic functionality. 2017-11-20 09:30:06 +02:00
objnamedtuple.h py/objnamedtuple: Allow to reuse namedtuple basic functionality. 2017-11-20 09:30:06 +02:00
objnone.c all: Remove inclusion of internal py header files. 2017-10-04 12:37:50 +11:00
objobject.c py/objtype: Refactor object's handling of __new__ to not create 2 objs. 2017-12-12 16:53:44 +11:00
objpolyiter.c all: Remove inclusion of internal py header files. 2017-10-04 12:37:50 +11:00
objproperty.c all: Remove inclusion of internal py header files. 2017-10-04 12:37:50 +11:00
objrange.c py/objrange: Implement (in)equality comparison between range objects. 2018-02-14 23:17:06 +11:00
objreversed.c all: Remove inclusion of internal py header files. 2017-10-04 12:37:50 +11:00
objset.c py/objset: Remove unneeded check from set_equal. 2017-12-19 14:01:19 +11:00
objsingleton.c all: Remove inclusion of internal py header files. 2017-10-04 12:37:50 +11:00
objslice.c all: Remove inclusion of internal py header files. 2017-10-04 12:37:50 +11:00
objstr.c py/objstr: In format error message, use common string with %s for type. 2018-07-30 12:46:47 +10:00
objstr.h py/objstr: Make mp_obj_new_str_of_type check for existing interned qstr. 2017-11-16 13:53:04 +11:00
objstringio.c py/stream: Switch stream close operation from method to ioctl. 2018-04-10 13:41:32 +10:00
objstringio.h py/objstringio: If created from immutable object, follow copy on write policy. 2017-06-09 17:33:01 +03:00
objstrunicode.c py/unicode: Clean up utf8 funcs and provide non-utf8 inline versions. 2018-02-14 18:19:22 +11:00
objtuple.c all: Remove inclusion of internal py header files. 2017-10-04 12:37:50 +11:00
objtuple.h all: Convert mp_uint_t to mp_unary_op_t/mp_binary_op_t where appropriate 2017-08-29 13:16:30 +10:00
objtype.c py: Fix compiling with debug enabled and make more use of DEBUG_printf. 2018-08-02 14:17:24 +10:00
objtype.h py/objtype: Don't expose mp_obj_instance_attr(). 2018-06-08 11:48:25 +10:00
objzip.c all: Use the name MicroPython consistently in comments 2017-07-31 18:35:40 +10:00
opmethods.c py/runtime: Add MP_BINARY_OP_CONTAINS as reverse of MP_BINARY_OP_IN. 2017-11-24 14:48:23 +11:00
parse.c py/parse: Fix macro evaluation by avoiding empty __VA_ARGS__. 2017-12-29 13:44:26 +11:00
parse.h all: Use the name MicroPython consistently in comments 2017-07-31 18:35:40 +10:00
parsenum.c py/lexer: Add support for underscores in numeric literals. 2018-06-12 12:17:43 +10:00
parsenum.h all: Use the name MicroPython consistently in comments 2017-07-31 18:35:40 +10:00
parsenumbase.c all: Use the name MicroPython consistently in comments 2017-07-31 18:35:40 +10:00
parsenumbase.h all: Use the name MicroPython consistently in comments 2017-07-31 18:35:40 +10:00
persistentcode.c py/emitglue: When assigning bytecode only pass bytecode len if needed. 2018-02-14 18:41:17 +11:00
persistentcode.h py: Factor out persistent-code reader into separate files. 2016-11-16 18:13:50 +11:00
py.mk extmod/modbtree: Update to work with new mp_stream_posix_XXX signatures. 2018-08-14 17:41:23 +10:00
pystack.c py/pystack: Use "pystack exhausted" as error msg for out of pystack mem. 2018-02-19 00:26:14 +11:00
pystack.h py: Introduce a Python stack for scoped allocation. 2017-12-11 13:49:09 +11:00
qstr.c py/qstr: Rewrite find_qstr to make manifest that it returns a valid ptr. 2017-11-29 17:01:39 +11:00
qstr.h py/qstr: Add QSTR_TOTAL() macro to get number of qstrs. 2018-02-19 16:12:44 +11:00
qstrdefs.h py/pystack: Use "pystack exhausted" as error msg for out of pystack mem. 2018-02-19 00:26:14 +11:00
reader.c py/reader: Allow MICROPY_VFS_POSIX to work with MICROPY_READER_POSIX. 2018-06-06 14:28:23 +10:00
reader.h py: Allow lexer to raise exceptions during construction. 2017-03-14 11:52:05 +11:00
repl.c py/repl: Fix handling of unmatched brackets and unfinished quotes. 2018-05-18 15:23:02 +10:00
repl.h all: Use the name MicroPython consistently in comments 2017-07-31 18:35:40 +10:00
ringbuf.h all: Unify header guard usage. 2017-07-18 11:57:39 +10:00
runtime.c py/runtime: Use mp_obj_new_int_from_ll when return int is not small. 2018-07-14 23:05:25 +10:00
runtime.h py/runtime: Add mp_load_method_protected helper which catches exceptions 2018-05-10 23:00:04 +10:00
runtime0.h py/emit: Merge build set/slice into existing build emit function. 2018-05-23 00:23:36 +10:00
runtime_utils.c py: mp_call_function_*_protected(): Pass-thru return value if possible. 2017-12-05 00:38:41 +02:00
scheduler.c py: Add micropython.schedule() function and associated runtime code. 2017-03-20 15:20:26 +11:00
scope.c all: Use the name MicroPython consistently in comments 2017-07-31 18:35:40 +10:00
scope.h all: Use the name MicroPython consistently in comments 2017-07-31 18:35:40 +10:00
sequence.c all: Remove inclusion of internal py header files. 2017-10-04 12:37:50 +11:00
showbc.c py: Clean up unary and binary enum list to keep groups together. 2017-10-05 10:49:44 +11:00
smallint.c all: Use the name MicroPython consistently in comments 2017-07-31 18:35:40 +10:00
smallint.h py: Extend nan-boxing config to have 47-bit small integers. 2017-12-11 22:39:12 +11:00
stackctrl.c py/runtime: Move mp_exc_recursion_depth to runtime and rename to raise. 2017-12-11 13:49:09 +11:00
stackctrl.h all: Use the name MicroPython consistently in comments 2017-07-31 18:35:40 +10:00
stream.c py/stream: Adjust mp_stream_posix_XXX to take void*, not mp_obj_t. 2018-08-14 17:36:08 +10:00
stream.h py/stream: Adjust mp_stream_posix_XXX to take void*, not mp_obj_t. 2018-08-14 17:36:08 +10:00
unicode.c py/unicode: Clean up utf8 funcs and provide non-utf8 inline versions. 2018-02-14 18:19:22 +11:00
unicode.h py/objstr: Add check for valid UTF-8 when making a str from bytes. 2017-09-06 16:43:09 +10:00
vm.c py/objtype: Don't expose mp_obj_instance_attr(). 2018-06-08 11:48:25 +10:00
vmentrytable.h py: Clean up unary and binary enum list to keep groups together. 2017-10-05 10:49:44 +11:00
vstr.c py/vstr: Raise a RuntimeError if fixed vstr buffer overflows. 2017-09-21 20:29:41 +10:00
warning.c py: Add config option to print warnings/errors to stderr. 2017-09-26 11:59:11 +10:00