mirrors
/
micropython
mirror of https://github.com/micropython/micropython.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
micropython/py/showbc.c

Ignoring revisions in .git-blame-ignore-revs. Click here to bypass and see the normal blame view.

542 lines
17 KiB
C
Raw Normal View History

Add license header to (almost) all files. Blanket wide to all .c and .h files. Some files originating from ST are difficult to deal with (license wise) so it was left out of those. Also merged modpyb.h, modos.h, modstm.h and modtime.h in stmhal/.
2014-05-03 23:27:38 +01:00
/*
all: Use the name MicroPython consistently in comments There were several different spellings of MicroPython present in comments, when there should be only one.
2017-06-30 08:22:17 +01:00
* This file is part of the MicroPython project, http://micropython.org/
Add license header to (almost) all files. Blanket wide to all .c and .h files. Some files originating from ST are difficult to deal with (license wise) so it was left out of those. Also merged modpyb.h, modos.h, modstm.h and modtime.h in stmhal/.
2014-05-03 23:27:38 +01:00
*
* The MIT License (MIT)
*
* Copyright (c) 2013, 2014 Damien P. George
*
* Permission is hereby granted, free of charge, to any person obtaining a copy
* of this software and associated documentation files (the "Software"), to deal
* in the Software without restriction, including without limitation the rights
* to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
* copies of the Software, and to permit persons to whom the Software is
* furnished to do so, subject to the following conditions:
*
* The above copyright notice and this permission notice shall be included in
* all copies or substantial portions of the Software.
*
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
* FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
* AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
* LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
* OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
* THE SOFTWARE.
*/
Add function to decode and show byte code.
2013-11-17 13:19:33 +00:00
#include <stdio.h>
#include <assert.h>
py: Move to guarded includes, everywhere in py/ core. Addresses issue #1022.
2015-01-01 20:27:54 +00:00
#include "py/bc0.h"
#include "py/bc.h"
py: Add module/function/class name to exceptions. Exceptions know source file, line and block name. Also tidy up some debug printing functions and provide a global flag to enable/disable them.
2014-01-19 11:48:48 +00:00
#if MICROPY_DEBUG_PRINTERS
Factor and simplify Makefile's and mpconfig, part 2.
2014-01-07 15:20:33 +00:00
showbc: Update for recent int varlen storage refactor. TODO: De-duplicate DECODE_UINT, etc. definitions.
2014-02-21 00:31:05 +00:00
#define DECODE_UINT { \
unum = 0; \
do { \
unum = (unum << 7) + (*ip & 0x7f); \
} while ((*ip++ & 0x80) != 0); \
}
Add function to decode and show byte code.
2013-11-17 13:19:33 +00:00
#define DECODE_ULABEL do { unum = (ip[0] | (ip[1] << 8)); ip += 2; } while (0)
#define DECODE_SLABEL do { unum = (ip[0] | (ip[1] << 8)) - 0x8000; ip += 2; } while (0)
py: Add MICROPY_PERSISTENT_CODE so code can persist beyond the runtime. Main changes when MICROPY_PERSISTENT_CODE is enabled are: - qstrs are encoded as 2-byte fixed width in the bytecode - all pointers are removed from bytecode and put in const_table (this includes const objects and raw code pointers) Ultimately this option will enable persistence for not just bytecode but also native code.
2015-11-02 17:27:18 +00:00
py: Rework bytecode and .mpy file format to be mostly static data. Background: .mpy files are precompiled .py files, built using mpy-cross, that contain compiled bytecode functions (and can also contain machine code). The benefit of using an .mpy file over a .py file is that they are faster to import and take less memory when importing. They are also smaller on disk. But the real benefit of .mpy files comes when they are frozen into the firmware. This is done by loading the .mpy file during compilation of the firmware and turning it into a set of big C data structures (the job of mpy-tool.py), which are then compiled and downloaded into the ROM of a device. These C data structures can be executed in-place, ie directly from ROM. This makes importing even faster because there is very little to do, and also means such frozen modules take up much less RAM (because their bytecode stays in ROM). The downside of frozen code is that it requires recompiling and reflashing the entire firmware. This can be a big barrier to entry, slows down development time, and makes it harder to do OTA updates of frozen code (because the whole firmware must be updated). This commit attempts to solve this problem by providing a solution that sits between loading .mpy files into RAM and freezing them into the firmware. The .mpy file format has been reworked so that it consists of data and bytecode which is mostly static and ready to run in-place. If these new .mpy files are located in flash/ROM which is memory addressable, the .mpy file can be executed (mostly) in-place. With this approach there is still a small amount of unpacking and linking of the .mpy file that needs to be done when it's imported, but it's still much better than loading an .mpy from disk into RAM (although not as good as freezing .mpy files into the firmware). The main trick to make static .mpy files is to adjust the bytecode so any qstrs that it references now go through a lookup table to convert from local qstr number in the module to global qstr number in the firmware. That means the bytecode does not need linking/rewriting of qstrs when it's loaded. Instead only a small qstr table needs to be built (and put in RAM) at import time. This means the bytecode itself is static/constant and can be used directly if it's in addressable memory. Also the qstr string data in the .mpy file, and some constant object data, can be used directly. Note that the qstr table is global to the module (ie not per function). In more detail, in the VM what used to be (schematically): qst = DECODE_QSTR_VALUE; is now (schematically): idx = DECODE_QSTR_INDEX; qst = qstr_table[idx]; That allows the bytecode to be fixed at compile time and not need relinking/rewriting of the qstr values. Only qstr_table needs to be linked when the .mpy is loaded. Incidentally, this helps to reduce the size of bytecode because what used to be 2-byte qstr values in the bytecode are now (mostly) 1-byte indices. If the module uses the same qstr more than two times then the bytecode is smaller than before. The following changes are measured for this commit compared to the previous (the baseline): - average 7%-9% reduction in size of .mpy files - frozen code size is reduced by about 5%-7% - importing .py files uses about 5% less RAM in total - importing .mpy files uses about 4% less RAM in total - importing .py and .mpy files takes about the same time as before The qstr indirection in the bytecode has only a small impact on VM performance. For stm32 on PYBv1.0 the performance change of this commit is: diff of scores (higher is better) N=100 M=100 baseline -> this-commit diff diff% (error%) bm_chaos.py 371.07 -> 357.39 : -13.68 = -3.687% (+/-0.02%) bm_fannkuch.py 78.72 -> 77.49 : -1.23 = -1.563% (+/-0.01%) bm_fft.py 2591.73 -> 2539.28 : -52.45 = -2.024% (+/-0.00%) bm_float.py 6034.93 -> 5908.30 : -126.63 = -2.098% (+/-0.01%) bm_hexiom.py 48.96 -> 47.93 : -1.03 = -2.104% (+/-0.00%) bm_nqueens.py 4510.63 -> 4459.94 : -50.69 = -1.124% (+/-0.00%) bm_pidigits.py 650.28 -> 644.96 : -5.32 = -0.818% (+/-0.23%) core_import_mpy_multi.py 564.77 -> 581.49 : +16.72 = +2.960% (+/-0.01%) core_import_mpy_single.py 68.67 -> 67.16 : -1.51 = -2.199% (+/-0.01%) core_qstr.py 64.16 -> 64.12 : -0.04 = -0.062% (+/-0.00%) core_yield_from.py 362.58 -> 354.50 : -8.08 = -2.228% (+/-0.00%) misc_aes.py 429.69 -> 405.59 : -24.10 = -5.609% (+/-0.01%) misc_mandel.py 3485.13 -> 3416.51 : -68.62 = -1.969% (+/-0.00%) misc_pystone.py 2496.53 -> 2405.56 : -90.97 = -3.644% (+/-0.01%) misc_raytrace.py 381.47 -> 374.01 : -7.46 = -1.956% (+/-0.01%) viper_call0.py 576.73 -> 572.49 : -4.24 = -0.735% (+/-0.04%) viper_call1a.py 550.37 -> 546.21 : -4.16 = -0.756% (+/-0.09%) viper_call1b.py 438.23 -> 435.68 : -2.55 = -0.582% (+/-0.06%) viper_call1c.py 442.84 -> 440.04 : -2.80 = -0.632% (+/-0.08%) viper_call2a.py 536.31 -> 532.35 : -3.96 = -0.738% (+/-0.06%) viper_call2b.py 382.34 -> 377.07 : -5.27 = -1.378% (+/-0.03%) And for unix on x64: diff of scores (higher is better) N=2000 M=2000 baseline -> this-commit diff diff% (error%) bm_chaos.py 13594.20 -> 13073.84 : -520.36 = -3.828% (+/-5.44%) bm_fannkuch.py 60.63 -> 59.58 : -1.05 = -1.732% (+/-3.01%) bm_fft.py 112009.15 -> 111603.32 : -405.83 = -0.362% (+/-4.03%) bm_float.py 246202.55 -> 247923.81 : +1721.26 = +0.699% (+/-2.79%) bm_hexiom.py 615.65 -> 617.21 : +1.56 = +0.253% (+/-1.64%) bm_nqueens.py 215807.95 -> 215600.96 : -206.99 = -0.096% (+/-3.52%) bm_pidigits.py 8246.74 -> 8422.82 : +176.08 = +2.135% (+/-3.64%) misc_aes.py 16133.00 -> 16452.74 : +319.74 = +1.982% (+/-1.50%) misc_mandel.py 128146.69 -> 130796.43 : +2649.74 = +2.068% (+/-3.18%) misc_pystone.py 83811.49 -> 83124.85 : -686.64 = -0.819% (+/-1.03%) misc_raytrace.py 21688.02 -> 21385.10 : -302.92 = -1.397% (+/-3.20%) The code size change is (firmware with a lot of frozen code benefits the most): bare-arm: +396 +0.697% minimal x86: +1595 +0.979% [incl +32(data)] unix x64: +2408 +0.470% [incl +800(data)] unix nanbox: +1396 +0.309% [incl -96(data)] stm32: -1256 -0.318% PYBV10 cc3200: +288 +0.157% esp8266: -260 -0.037% GENERIC esp32: -216 -0.014% GENERIC[incl -1072(data)] nrf: +116 +0.067% pca10040 rp2: -664 -0.135% PICO samd: +844 +0.607% ADAFRUIT_ITSYBITSY_M4_EXPRESS As part of this change the .mpy file format version is bumped to version 6. And mpy-tool.py has been improved to provide a good visualisation of the contents of .mpy files. In summary: this commit changes the bytecode to use qstr indirection, and reworks the .mpy file format to be simpler and allow .mpy files to be executed in-place. Performance is not impacted too much. Eventually it will be possible to store such .mpy files in a linear, read-only, memory- mappable filesystem so they can be executed from flash/ROM. This will essentially be able to replace frozen code for most applications. Signed-off-by: Damien George <damien@micropython.org>
2021-10-22 12:22:47 +01:00
#if MICROPY_EMIT_BYTECODE_USES_QSTR_TABLE
py: Add MICROPY_PERSISTENT_CODE so code can persist beyond the runtime. Main changes when MICROPY_PERSISTENT_CODE is enabled are: - qstrs are encoded as 2-byte fixed width in the bytecode - all pointers are removed from bytecode and put in const_table (this includes const objects and raw code pointers) Ultimately this option will enable persistence for not just bytecode but also native code.
2015-11-02 17:27:18 +00:00
#define DECODE_QSTR \
py: Wrap all obj-ptr conversions in MP_OBJ_TO_PTR/MP_OBJ_FROM_PTR. This allows the mp_obj_t type to be configured to something other than a pointer-sized primitive type. This patch also includes additional changes to allow the code to compile when sizeof(mp_uint_t) != sizeof(void*), such as using size_t instead of mp_uint_t, and various casts.
2015-11-27 17:01:44 +00:00
DECODE_UINT; \
py: Rework bytecode and .mpy file format to be mostly static data. Background: .mpy files are precompiled .py files, built using mpy-cross, that contain compiled bytecode functions (and can also contain machine code). The benefit of using an .mpy file over a .py file is that they are faster to import and take less memory when importing. They are also smaller on disk. But the real benefit of .mpy files comes when they are frozen into the firmware. This is done by loading the .mpy file during compilation of the firmware and turning it into a set of big C data structures (the job of mpy-tool.py), which are then compiled and downloaded into the ROM of a device. These C data structures can be executed in-place, ie directly from ROM. This makes importing even faster because there is very little to do, and also means such frozen modules take up much less RAM (because their bytecode stays in ROM). The downside of frozen code is that it requires recompiling and reflashing the entire firmware. This can be a big barrier to entry, slows down development time, and makes it harder to do OTA updates of frozen code (because the whole firmware must be updated). This commit attempts to solve this problem by providing a solution that sits between loading .mpy files into RAM and freezing them into the firmware. The .mpy file format has been reworked so that it consists of data and bytecode which is mostly static and ready to run in-place. If these new .mpy files are located in flash/ROM which is memory addressable, the .mpy file can be executed (mostly) in-place. With this approach there is still a small amount of unpacking and linking of the .mpy file that needs to be done when it's imported, but it's still much better than loading an .mpy from disk into RAM (although not as good as freezing .mpy files into the firmware). The main trick to make static .mpy files is to adjust the bytecode so any qstrs that it references now go through a lookup table to convert from local qstr number in the module to global qstr number in the firmware. That means the bytecode does not need linking/rewriting of qstrs when it's loaded. Instead only a small qstr table needs to be built (and put in RAM) at import time. This means the bytecode itself is static/constant and can be used directly if it's in addressable memory. Also the qstr string data in the .mpy file, and some constant object data, can be used directly. Note that the qstr table is global to the module (ie not per function). In more detail, in the VM what used to be (schematically): qst = DECODE_QSTR_VALUE; is now (schematically): idx = DECODE_QSTR_INDEX; qst = qstr_table[idx]; That allows the bytecode to be fixed at compile time and not need relinking/rewriting of the qstr values. Only qstr_table needs to be linked when the .mpy is loaded. Incidentally, this helps to reduce the size of bytecode because what used to be 2-byte qstr values in the bytecode are now (mostly) 1-byte indices. If the module uses the same qstr more than two times then the bytecode is smaller than before. The following changes are measured for this commit compared to the previous (the baseline): - average 7%-9% reduction in size of .mpy files - frozen code size is reduced by about 5%-7% - importing .py files uses about 5% less RAM in total - importing .mpy files uses about 4% less RAM in total - importing .py and .mpy files takes about the same time as before The qstr indirection in the bytecode has only a small impact on VM performance. For stm32 on PYBv1.0 the performance change of this commit is: diff of scores (higher is better) N=100 M=100 baseline -> this-commit diff diff% (error%) bm_chaos.py 371.07 -> 357.39 : -13.68 = -3.687% (+/-0.02%) bm_fannkuch.py 78.72 -> 77.49 : -1.23 = -1.563% (+/-0.01%) bm_fft.py 2591.73 -> 2539.28 : -52.45 = -2.024% (+/-0.00%) bm_float.py 6034.93 -> 5908.30 : -126.63 = -2.098% (+/-0.01%) bm_hexiom.py 48.96 -> 47.93 : -1.03 = -2.104% (+/-0.00%) bm_nqueens.py 4510.63 -> 4459.94 : -50.69 = -1.124% (+/-0.00%) bm_pidigits.py 650.28 -> 644.96 : -5.32 = -0.818% (+/-0.23%) core_import_mpy_multi.py 564.77 -> 581.49 : +16.72 = +2.960% (+/-0.01%) core_import_mpy_single.py 68.67 -> 67.16 : -1.51 = -2.199% (+/-0.01%) core_qstr.py 64.16 -> 64.12 : -0.04 = -0.062% (+/-0.00%) core_yield_from.py 362.58 -> 354.50 : -8.08 = -2.228% (+/-0.00%) misc_aes.py 429.69 -> 405.59 : -24.10 = -5.609% (+/-0.01%) misc_mandel.py 3485.13 -> 3416.51 : -68.62 = -1.969% (+/-0.00%) misc_pystone.py 2496.53 -> 2405.56 : -90.97 = -3.644% (+/-0.01%) misc_raytrace.py 381.47 -> 374.01 : -7.46 = -1.956% (+/-0.01%) viper_call0.py 576.73 -> 572.49 : -4.24 = -0.735% (+/-0.04%) viper_call1a.py 550.37 -> 546.21 : -4.16 = -0.756% (+/-0.09%) viper_call1b.py 438.23 -> 435.68 : -2.55 = -0.582% (+/-0.06%) viper_call1c.py 442.84 -> 440.04 : -2.80 = -0.632% (+/-0.08%) viper_call2a.py 536.31 -> 532.35 : -3.96 = -0.738% (+/-0.06%) viper_call2b.py 382.34 -> 377.07 : -5.27 = -1.378% (+/-0.03%) And for unix on x64: diff of scores (higher is better) N=2000 M=2000 baseline -> this-commit diff diff% (error%) bm_chaos.py 13594.20 -> 13073.84 : -520.36 = -3.828% (+/-5.44%) bm_fannkuch.py 60.63 -> 59.58 : -1.05 = -1.732% (+/-3.01%) bm_fft.py 112009.15 -> 111603.32 : -405.83 = -0.362% (+/-4.03%) bm_float.py 246202.55 -> 247923.81 : +1721.26 = +0.699% (+/-2.79%) bm_hexiom.py 615.65 -> 617.21 : +1.56 = +0.253% (+/-1.64%) bm_nqueens.py 215807.95 -> 215600.96 : -206.99 = -0.096% (+/-3.52%) bm_pidigits.py 8246.74 -> 8422.82 : +176.08 = +2.135% (+/-3.64%) misc_aes.py 16133.00 -> 16452.74 : +319.74 = +1.982% (+/-1.50%) misc_mandel.py 128146.69 -> 130796.43 : +2649.74 = +2.068% (+/-3.18%) misc_pystone.py 83811.49 -> 83124.85 : -686.64 = -0.819% (+/-1.03%) misc_raytrace.py 21688.02 -> 21385.10 : -302.92 = -1.397% (+/-3.20%) The code size change is (firmware with a lot of frozen code benefits the most): bare-arm: +396 +0.697% minimal x86: +1595 +0.979% [incl +32(data)] unix x64: +2408 +0.470% [incl +800(data)] unix nanbox: +1396 +0.309% [incl -96(data)] stm32: -1256 -0.318% PYBV10 cc3200: +288 +0.157% esp8266: -260 -0.037% GENERIC esp32: -216 -0.014% GENERIC[incl -1072(data)] nrf: +116 +0.067% pca10040 rp2: -664 -0.135% PICO samd: +844 +0.607% ADAFRUIT_ITSYBITSY_M4_EXPRESS As part of this change the .mpy file format version is bumped to version 6. And mpy-tool.py has been improved to provide a good visualisation of the contents of .mpy files. In summary: this commit changes the bytecode to use qstr indirection, and reworks the .mpy file format to be simpler and allow .mpy files to be executed in-place. Performance is not impacted too much. Eventually it will be possible to store such .mpy files in a linear, read-only, memory- mappable filesystem so they can be executed from flash/ROM. This will essentially be able to replace frozen code for most applications. Signed-off-by: Damien George <damien@micropython.org>
2021-10-22 12:22:47 +01:00
qst = mp_showbc_constants->qstr_table[unum]
py: Add MICROPY_PERSISTENT_CODE so code can persist beyond the runtime. Main changes when MICROPY_PERSISTENT_CODE is enabled are: - qstrs are encoded as 2-byte fixed width in the bytecode - all pointers are removed from bytecode and put in const_table (this includes const objects and raw code pointers) Ultimately this option will enable persistence for not just bytecode but also native code.
2015-11-02 17:27:18 +00:00
#else
py: Rework bytecode and .mpy file format to be mostly static data. Background: .mpy files are precompiled .py files, built using mpy-cross, that contain compiled bytecode functions (and can also contain machine code). The benefit of using an .mpy file over a .py file is that they are faster to import and take less memory when importing. They are also smaller on disk. But the real benefit of .mpy files comes when they are frozen into the firmware. This is done by loading the .mpy file during compilation of the firmware and turning it into a set of big C data structures (the job of mpy-tool.py), which are then compiled and downloaded into the ROM of a device. These C data structures can be executed in-place, ie directly from ROM. This makes importing even faster because there is very little to do, and also means such frozen modules take up much less RAM (because their bytecode stays in ROM). The downside of frozen code is that it requires recompiling and reflashing the entire firmware. This can be a big barrier to entry, slows down development time, and makes it harder to do OTA updates of frozen code (because the whole firmware must be updated). This commit attempts to solve this problem by providing a solution that sits between loading .mpy files into RAM and freezing them into the firmware. The .mpy file format has been reworked so that it consists of data and bytecode which is mostly static and ready to run in-place. If these new .mpy files are located in flash/ROM which is memory addressable, the .mpy file can be executed (mostly) in-place. With this approach there is still a small amount of unpacking and linking of the .mpy file that needs to be done when it's imported, but it's still much better than loading an .mpy from disk into RAM (although not as good as freezing .mpy files into the firmware). The main trick to make static .mpy files is to adjust the bytecode so any qstrs that it references now go through a lookup table to convert from local qstr number in the module to global qstr number in the firmware. That means the bytecode does not need linking/rewriting of qstrs when it's loaded. Instead only a small qstr table needs to be built (and put in RAM) at import time. This means the bytecode itself is static/constant and can be used directly if it's in addressable memory. Also the qstr string data in the .mpy file, and some constant object data, can be used directly. Note that the qstr table is global to the module (ie not per function). In more detail, in the VM what used to be (schematically): qst = DECODE_QSTR_VALUE; is now (schematically): idx = DECODE_QSTR_INDEX; qst = qstr_table[idx]; That allows the bytecode to be fixed at compile time and not need relinking/rewriting of the qstr values. Only qstr_table needs to be linked when the .mpy is loaded. Incidentally, this helps to reduce the size of bytecode because what used to be 2-byte qstr values in the bytecode are now (mostly) 1-byte indices. If the module uses the same qstr more than two times then the bytecode is smaller than before. The following changes are measured for this commit compared to the previous (the baseline): - average 7%-9% reduction in size of .mpy files - frozen code size is reduced by about 5%-7% - importing .py files uses about 5% less RAM in total - importing .mpy files uses about 4% less RAM in total - importing .py and .mpy files takes about the same time as before The qstr indirection in the bytecode has only a small impact on VM performance. For stm32 on PYBv1.0 the performance change of this commit is: diff of scores (higher is better) N=100 M=100 baseline -> this-commit diff diff% (error%) bm_chaos.py 371.07 -> 357.39 : -13.68 = -3.687% (+/-0.02%) bm_fannkuch.py 78.72 -> 77.49 : -1.23 = -1.563% (+/-0.01%) bm_fft.py 2591.73 -> 2539.28 : -52.45 = -2.024% (+/-0.00%) bm_float.py 6034.93 -> 5908.30 : -126.63 = -2.098% (+/-0.01%) bm_hexiom.py 48.96 -> 47.93 : -1.03 = -2.104% (+/-0.00%) bm_nqueens.py 4510.63 -> 4459.94 : -50.69 = -1.124% (+/-0.00%) bm_pidigits.py 650.28 -> 644.96 : -5.32 = -0.818% (+/-0.23%) core_import_mpy_multi.py 564.77 -> 581.49 : +16.72 = +2.960% (+/-0.01%) core_import_mpy_single.py 68.67 -> 67.16 : -1.51 = -2.199% (+/-0.01%) core_qstr.py 64.16 -> 64.12 : -0.04 = -0.062% (+/-0.00%) core_yield_from.py 362.58 -> 354.50 : -8.08 = -2.228% (+/-0.00%) misc_aes.py 429.69 -> 405.59 : -24.10 = -5.609% (+/-0.01%) misc_mandel.py 3485.13 -> 3416.51 : -68.62 = -1.969% (+/-0.00%) misc_pystone.py 2496.53 -> 2405.56 : -90.97 = -3.644% (+/-0.01%) misc_raytrace.py 381.47 -> 374.01 : -7.46 = -1.956% (+/-0.01%) viper_call0.py 576.73 -> 572.49 : -4.24 = -0.735% (+/-0.04%) viper_call1a.py 550.37 -> 546.21 : -4.16 = -0.756% (+/-0.09%) viper_call1b.py 438.23 -> 435.68 : -2.55 = -0.582% (+/-0.06%) viper_call1c.py 442.84 -> 440.04 : -2.80 = -0.632% (+/-0.08%) viper_call2a.py 536.31 -> 532.35 : -3.96 = -0.738% (+/-0.06%) viper_call2b.py 382.34 -> 377.07 : -5.27 = -1.378% (+/-0.03%) And for unix on x64: diff of scores (higher is better) N=2000 M=2000 baseline -> this-commit diff diff% (error%) bm_chaos.py 13594.20 -> 13073.84 : -520.36 = -3.828% (+/-5.44%) bm_fannkuch.py 60.63 -> 59.58 : -1.05 = -1.732% (+/-3.01%) bm_fft.py 112009.15 -> 111603.32 : -405.83 = -0.362% (+/-4.03%) bm_float.py 246202.55 -> 247923.81 : +1721.26 = +0.699% (+/-2.79%) bm_hexiom.py 615.65 -> 617.21 : +1.56 = +0.253% (+/-1.64%) bm_nqueens.py 215807.95 -> 215600.96 : -206.99 = -0.096% (+/-3.52%) bm_pidigits.py 8246.74 -> 8422.82 : +176.08 = +2.135% (+/-3.64%) misc_aes.py 16133.00 -> 16452.74 : +319.74 = +1.982% (+/-1.50%) misc_mandel.py 128146.69 -> 130796.43 : +2649.74 = +2.068% (+/-3.18%) misc_pystone.py 83811.49 -> 83124.85 : -686.64 = -0.819% (+/-1.03%) misc_raytrace.py 21688.02 -> 21385.10 : -302.92 = -1.397% (+/-3.20%) The code size change is (firmware with a lot of frozen code benefits the most): bare-arm: +396 +0.697% minimal x86: +1595 +0.979% [incl +32(data)] unix x64: +2408 +0.470% [incl +800(data)] unix nanbox: +1396 +0.309% [incl -96(data)] stm32: -1256 -0.318% PYBV10 cc3200: +288 +0.157% esp8266: -260 -0.037% GENERIC esp32: -216 -0.014% GENERIC[incl -1072(data)] nrf: +116 +0.067% pca10040 rp2: -664 -0.135% PICO samd: +844 +0.607% ADAFRUIT_ITSYBITSY_M4_EXPRESS As part of this change the .mpy file format version is bumped to version 6. And mpy-tool.py has been improved to provide a good visualisation of the contents of .mpy files. In summary: this commit changes the bytecode to use qstr indirection, and reworks the .mpy file format to be simpler and allow .mpy files to be executed in-place. Performance is not impacted too much. Eventually it will be possible to store such .mpy files in a linear, read-only, memory- mappable filesystem so they can be executed from flash/ROM. This will essentially be able to replace frozen code for most applications. Signed-off-by: Damien George <damien@micropython.org>
2021-10-22 12:22:47 +01:00
#define DECODE_QSTR \
DECODE_UINT; \
qst = unum;
Add function to decode and show byte code.
2013-11-17 13:19:33 +00:00
py: Add MICROPY_PERSISTENT_CODE so code can persist beyond the runtime. Main changes when MICROPY_PERSISTENT_CODE is enabled are: - qstrs are encoded as 2-byte fixed width in the bytecode - all pointers are removed from bytecode and put in const_table (this includes const objects and raw code pointers) Ultimately this option will enable persistence for not just bytecode but also native code.
2015-11-02 17:27:18 +00:00
#endif
py: Rework bytecode and .mpy file format to be mostly static data. Background: .mpy files are precompiled .py files, built using mpy-cross, that contain compiled bytecode functions (and can also contain machine code). The benefit of using an .mpy file over a .py file is that they are faster to import and take less memory when importing. They are also smaller on disk. But the real benefit of .mpy files comes when they are frozen into the firmware. This is done by loading the .mpy file during compilation of the firmware and turning it into a set of big C data structures (the job of mpy-tool.py), which are then compiled and downloaded into the ROM of a device. These C data structures can be executed in-place, ie directly from ROM. This makes importing even faster because there is very little to do, and also means such frozen modules take up much less RAM (because their bytecode stays in ROM). The downside of frozen code is that it requires recompiling and reflashing the entire firmware. This can be a big barrier to entry, slows down development time, and makes it harder to do OTA updates of frozen code (because the whole firmware must be updated). This commit attempts to solve this problem by providing a solution that sits between loading .mpy files into RAM and freezing them into the firmware. The .mpy file format has been reworked so that it consists of data and bytecode which is mostly static and ready to run in-place. If these new .mpy files are located in flash/ROM which is memory addressable, the .mpy file can be executed (mostly) in-place. With this approach there is still a small amount of unpacking and linking of the .mpy file that needs to be done when it's imported, but it's still much better than loading an .mpy from disk into RAM (although not as good as freezing .mpy files into the firmware). The main trick to make static .mpy files is to adjust the bytecode so any qstrs that it references now go through a lookup table to convert from local qstr number in the module to global qstr number in the firmware. That means the bytecode does not need linking/rewriting of qstrs when it's loaded. Instead only a small qstr table needs to be built (and put in RAM) at import time. This means the bytecode itself is static/constant and can be used directly if it's in addressable memory. Also the qstr string data in the .mpy file, and some constant object data, can be used directly. Note that the qstr table is global to the module (ie not per function). In more detail, in the VM what used to be (schematically): qst = DECODE_QSTR_VALUE; is now (schematically): idx = DECODE_QSTR_INDEX; qst = qstr_table[idx]; That allows the bytecode to be fixed at compile time and not need relinking/rewriting of the qstr values. Only qstr_table needs to be linked when the .mpy is loaded. Incidentally, this helps to reduce the size of bytecode because what used to be 2-byte qstr values in the bytecode are now (mostly) 1-byte indices. If the module uses the same qstr more than two times then the bytecode is smaller than before. The following changes are measured for this commit compared to the previous (the baseline): - average 7%-9% reduction in size of .mpy files - frozen code size is reduced by about 5%-7% - importing .py files uses about 5% less RAM in total - importing .mpy files uses about 4% less RAM in total - importing .py and .mpy files takes about the same time as before The qstr indirection in the bytecode has only a small impact on VM performance. For stm32 on PYBv1.0 the performance change of this commit is: diff of scores (higher is better) N=100 M=100 baseline -> this-commit diff diff% (error%) bm_chaos.py 371.07 -> 357.39 : -13.68 = -3.687% (+/-0.02%) bm_fannkuch.py 78.72 -> 77.49 : -1.23 = -1.563% (+/-0.01%) bm_fft.py 2591.73 -> 2539.28 : -52.45 = -2.024% (+/-0.00%) bm_float.py 6034.93 -> 5908.30 : -126.63 = -2.098% (+/-0.01%) bm_hexiom.py 48.96 -> 47.93 : -1.03 = -2.104% (+/-0.00%) bm_nqueens.py 4510.63 -> 4459.94 : -50.69 = -1.124% (+/-0.00%) bm_pidigits.py 650.28 -> 644.96 : -5.32 = -0.818% (+/-0.23%) core_import_mpy_multi.py 564.77 -> 581.49 : +16.72 = +2.960% (+/-0.01%) core_import_mpy_single.py 68.67 -> 67.16 : -1.51 = -2.199% (+/-0.01%) core_qstr.py 64.16 -> 64.12 : -0.04 = -0.062% (+/-0.00%) core_yield_from.py 362.58 -> 354.50 : -8.08 = -2.228% (+/-0.00%) misc_aes.py 429.69 -> 405.59 : -24.10 = -5.609% (+/-0.01%) misc_mandel.py 3485.13 -> 3416.51 : -68.62 = -1.969% (+/-0.00%) misc_pystone.py 2496.53 -> 2405.56 : -90.97 = -3.644% (+/-0.01%) misc_raytrace.py 381.47 -> 374.01 : -7.46 = -1.956% (+/-0.01%) viper_call0.py 576.73 -> 572.49 : -4.24 = -0.735% (+/-0.04%) viper_call1a.py 550.37 -> 546.21 : -4.16 = -0.756% (+/-0.09%) viper_call1b.py 438.23 -> 435.68 : -2.55 = -0.582% (+/-0.06%) viper_call1c.py 442.84 -> 440.04 : -2.80 = -0.632% (+/-0.08%) viper_call2a.py 536.31 -> 532.35 : -3.96 = -0.738% (+/-0.06%) viper_call2b.py 382.34 -> 377.07 : -5.27 = -1.378% (+/-0.03%) And for unix on x64: diff of scores (higher is better) N=2000 M=2000 baseline -> this-commit diff diff% (error%) bm_chaos.py 13594.20 -> 13073.84 : -520.36 = -3.828% (+/-5.44%) bm_fannkuch.py 60.63 -> 59.58 : -1.05 = -1.732% (+/-3.01%) bm_fft.py 112009.15 -> 111603.32 : -405.83 = -0.362% (+/-4.03%) bm_float.py 246202.55 -> 247923.81 : +1721.26 = +0.699% (+/-2.79%) bm_hexiom.py 615.65 -> 617.21 : +1.56 = +0.253% (+/-1.64%) bm_nqueens.py 215807.95 -> 215600.96 : -206.99 = -0.096% (+/-3.52%) bm_pidigits.py 8246.74 -> 8422.82 : +176.08 = +2.135% (+/-3.64%) misc_aes.py 16133.00 -> 16452.74 : +319.74 = +1.982% (+/-1.50%) misc_mandel.py 128146.69 -> 130796.43 : +2649.74 = +2.068% (+/-3.18%) misc_pystone.py 83811.49 -> 83124.85 : -686.64 = -0.819% (+/-1.03%) misc_raytrace.py 21688.02 -> 21385.10 : -302.92 = -1.397% (+/-3.20%) The code size change is (firmware with a lot of frozen code benefits the most): bare-arm: +396 +0.697% minimal x86: +1595 +0.979% [incl +32(data)] unix x64: +2408 +0.470% [incl +800(data)] unix nanbox: +1396 +0.309% [incl -96(data)] stm32: -1256 -0.318% PYBV10 cc3200: +288 +0.157% esp8266: -260 -0.037% GENERIC esp32: -216 -0.014% GENERIC[incl -1072(data)] nrf: +116 +0.067% pca10040 rp2: -664 -0.135% PICO samd: +844 +0.607% ADAFRUIT_ITSYBITSY_M4_EXPRESS As part of this change the .mpy file format version is bumped to version 6. And mpy-tool.py has been improved to provide a good visualisation of the contents of .mpy files. In summary: this commit changes the bytecode to use qstr indirection, and reworks the .mpy file format to be simpler and allow .mpy files to be executed in-place. Performance is not impacted too much. Eventually it will be possible to store such .mpy files in a linear, read-only, memory- mappable filesystem so they can be executed from flash/ROM. This will essentially be able to replace frozen code for most applications. Signed-off-by: Damien George <damien@micropython.org>
2021-10-22 12:22:47 +01:00
#define DECODE_PTR \
DECODE_UINT;
#define DECODE_OBJ \
DECODE_UINT; \
unum = (mp_uint_t)mp_showbc_constants->obj_table[unum]
showbc: Refactor to allow inline instruction printing.
2014-12-27 03:00:08 +00:00
py: Rework bytecode and .mpy file format to be mostly static data. Background: .mpy files are precompiled .py files, built using mpy-cross, that contain compiled bytecode functions (and can also contain machine code). The benefit of using an .mpy file over a .py file is that they are faster to import and take less memory when importing. They are also smaller on disk. But the real benefit of .mpy files comes when they are frozen into the firmware. This is done by loading the .mpy file during compilation of the firmware and turning it into a set of big C data structures (the job of mpy-tool.py), which are then compiled and downloaded into the ROM of a device. These C data structures can be executed in-place, ie directly from ROM. This makes importing even faster because there is very little to do, and also means such frozen modules take up much less RAM (because their bytecode stays in ROM). The downside of frozen code is that it requires recompiling and reflashing the entire firmware. This can be a big barrier to entry, slows down development time, and makes it harder to do OTA updates of frozen code (because the whole firmware must be updated). This commit attempts to solve this problem by providing a solution that sits between loading .mpy files into RAM and freezing them into the firmware. The .mpy file format has been reworked so that it consists of data and bytecode which is mostly static and ready to run in-place. If these new .mpy files are located in flash/ROM which is memory addressable, the .mpy file can be executed (mostly) in-place. With this approach there is still a small amount of unpacking and linking of the .mpy file that needs to be done when it's imported, but it's still much better than loading an .mpy from disk into RAM (although not as good as freezing .mpy files into the firmware). The main trick to make static .mpy files is to adjust the bytecode so any qstrs that it references now go through a lookup table to convert from local qstr number in the module to global qstr number in the firmware. That means the bytecode does not need linking/rewriting of qstrs when it's loaded. Instead only a small qstr table needs to be built (and put in RAM) at import time. This means the bytecode itself is static/constant and can be used directly if it's in addressable memory. Also the qstr string data in the .mpy file, and some constant object data, can be used directly. Note that the qstr table is global to the module (ie not per function). In more detail, in the VM what used to be (schematically): qst = DECODE_QSTR_VALUE; is now (schematically): idx = DECODE_QSTR_INDEX; qst = qstr_table[idx]; That allows the bytecode to be fixed at compile time and not need relinking/rewriting of the qstr values. Only qstr_table needs to be linked when the .mpy is loaded. Incidentally, this helps to reduce the size of bytecode because what used to be 2-byte qstr values in the bytecode are now (mostly) 1-byte indices. If the module uses the same qstr more than two times then the bytecode is smaller than before. The following changes are measured for this commit compared to the previous (the baseline): - average 7%-9% reduction in size of .mpy files - frozen code size is reduced by about 5%-7% - importing .py files uses about 5% less RAM in total - importing .mpy files uses about 4% less RAM in total - importing .py and .mpy files takes about the same time as before The qstr indirection in the bytecode has only a small impact on VM performance. For stm32 on PYBv1.0 the performance change of this commit is: diff of scores (higher is better) N=100 M=100 baseline -> this-commit diff diff% (error%) bm_chaos.py 371.07 -> 357.39 : -13.68 = -3.687% (+/-0.02%) bm_fannkuch.py 78.72 -> 77.49 : -1.23 = -1.563% (+/-0.01%) bm_fft.py 2591.73 -> 2539.28 : -52.45 = -2.024% (+/-0.00%) bm_float.py 6034.93 -> 5908.30 : -126.63 = -2.098% (+/-0.01%) bm_hexiom.py 48.96 -> 47.93 : -1.03 = -2.104% (+/-0.00%) bm_nqueens.py 4510.63 -> 4459.94 : -50.69 = -1.124% (+/-0.00%) bm_pidigits.py 650.28 -> 644.96 : -5.32 = -0.818% (+/-0.23%) core_import_mpy_multi.py 564.77 -> 581.49 : +16.72 = +2.960% (+/-0.01%) core_import_mpy_single.py 68.67 -> 67.16 : -1.51 = -2.199% (+/-0.01%) core_qstr.py 64.16 -> 64.12 : -0.04 = -0.062% (+/-0.00%) core_yield_from.py 362.58 -> 354.50 : -8.08 = -2.228% (+/-0.00%) misc_aes.py 429.69 -> 405.59 : -24.10 = -5.609% (+/-0.01%) misc_mandel.py 3485.13 -> 3416.51 : -68.62 = -1.969% (+/-0.00%) misc_pystone.py 2496.53 -> 2405.56 : -90.97 = -3.644% (+/-0.01%) misc_raytrace.py 381.47 -> 374.01 : -7.46 = -1.956% (+/-0.01%) viper_call0.py 576.73 -> 572.49 : -4.24 = -0.735% (+/-0.04%) viper_call1a.py 550.37 -> 546.21 : -4.16 = -0.756% (+/-0.09%) viper_call1b.py 438.23 -> 435.68 : -2.55 = -0.582% (+/-0.06%) viper_call1c.py 442.84 -> 440.04 : -2.80 = -0.632% (+/-0.08%) viper_call2a.py 536.31 -> 532.35 : -3.96 = -0.738% (+/-0.06%) viper_call2b.py 382.34 -> 377.07 : -5.27 = -1.378% (+/-0.03%) And for unix on x64: diff of scores (higher is better) N=2000 M=2000 baseline -> this-commit diff diff% (error%) bm_chaos.py 13594.20 -> 13073.84 : -520.36 = -3.828% (+/-5.44%) bm_fannkuch.py 60.63 -> 59.58 : -1.05 = -1.732% (+/-3.01%) bm_fft.py 112009.15 -> 111603.32 : -405.83 = -0.362% (+/-4.03%) bm_float.py 246202.55 -> 247923.81 : +1721.26 = +0.699% (+/-2.79%) bm_hexiom.py 615.65 -> 617.21 : +1.56 = +0.253% (+/-1.64%) bm_nqueens.py 215807.95 -> 215600.96 : -206.99 = -0.096% (+/-3.52%) bm_pidigits.py 8246.74 -> 8422.82 : +176.08 = +2.135% (+/-3.64%) misc_aes.py 16133.00 -> 16452.74 : +319.74 = +1.982% (+/-1.50%) misc_mandel.py 128146.69 -> 130796.43 : +2649.74 = +2.068% (+/-3.18%) misc_pystone.py 83811.49 -> 83124.85 : -686.64 = -0.819% (+/-1.03%) misc_raytrace.py 21688.02 -> 21385.10 : -302.92 = -1.397% (+/-3.20%) The code size change is (firmware with a lot of frozen code benefits the most): bare-arm: +396 +0.697% minimal x86: +1595 +0.979% [incl +32(data)] unix x64: +2408 +0.470% [incl +800(data)] unix nanbox: +1396 +0.309% [incl -96(data)] stm32: -1256 -0.318% PYBV10 cc3200: +288 +0.157% esp8266: -260 -0.037% GENERIC esp32: -216 -0.014% GENERIC[incl -1072(data)] nrf: +116 +0.067% pca10040 rp2: -664 -0.135% PICO samd: +844 +0.607% ADAFRUIT_ITSYBITSY_M4_EXPRESS As part of this change the .mpy file format version is bumped to version 6. And mpy-tool.py has been improved to provide a good visualisation of the contents of .mpy files. In summary: this commit changes the bytecode to use qstr indirection, and reworks the .mpy file format to be simpler and allow .mpy files to be executed in-place. Performance is not impacted too much. Eventually it will be possible to store such .mpy files in a linear, read-only, memory- mappable filesystem so they can be executed from flash/ROM. This will essentially be able to replace frozen code for most applications. Signed-off-by: Damien George <damien@micropython.org>
2021-10-22 12:22:47 +01:00
const byte * mp_showbc_code_start;
const mp_module_constants_t *mp_showbc_constants;
#include "py/emitglue.h"
void mp_bytecode_print(const mp_print_t *print, const void *descr, const byte *ip, mp_uint_t len, const mp_module_constants_t *cm) {
showbc: Make code object start pointer semi-public. This allows to pring either absolute addresses or relative offsets in jumps and code references.
2014-12-28 19:34:45 +00:00
mp_showbc_code_start = ip;
py: make closures work.
2013-12-30 22:32:17 +00:00
py: Compress first part of bytecode prelude. The start of the bytecode prelude contains 6 numbers telling the amount of stack needed for the Python values and exceptions, and the signature of the function. Prior to this patch these numbers were all encoded one after the other (2x variable unsigned integers, then 4x bytes), but using so many bytes is unnecessary. An entropy analysis of around 150,000 bytecode functions from the CPython standard library showed that the optimal Shannon coding would need about 7.1 bits on average to encode these 6 numbers, compared to the existing 48 bits. This patch attempts to get close to this optimal value by packing the 6 numbers into a single, varible-length unsigned integer via bit-wise interleaving. The interleaving scheme is chosen to minimise the average number of bytes needed, and at the same time keep the scheme simple enough so it can be implemented without too much overhead in code size or speed. The scheme requires about 10.5 bits on average to store the 6 numbers. As a result most functions which originally took 6 bytes to encode these 6 numbers now need only 1 byte (in 80% of cases).
2019-09-16 13:12:59 +01:00
// Decode prelude
MP_BC_PRELUDE_SIG_DECODE(ip);
py: Rework and compress second part of bytecode prelude. This patch compresses the second part of the bytecode prelude which contains the source file name, function name, source-line-number mapping and cell closure information. This part of the prelude now begins with a single varible length unsigned integer which encodes 2 numbers, being the byte-size of the following 2 sections in the header: the "source info section" and the "closure section". After decoding this variable unsigned integer it's possible to skip over one or both of these sections very easily. This scheme saves about 2 bytes for most functions compared to the original format: one in the case that there are no closure cells, and one because padding was eliminated.
2019-09-25 06:45:47 +01:00
MP_BC_PRELUDE_SIZE_DECODE(ip);
py: Fix up source-line calculation. Should address issue #475.
2014-04-13 14:51:56 +01:00
const byte *code_info = ip;
Add source file name and line number to error messages. Byte code has a map from byte-code offset to source-code line number, used to give better error messages.
2014-01-18 23:24:36 +00:00
py: Use variable length encoded uints in more places in bytecode. Code-info size, block name, source name, n_state and n_exc_stack now use variable length encoded uints. This saves 7-9 bytes per bytecode function for most functions.
2014-09-04 14:44:01 +01:00
qstr block_name = mp_decode_uint(&code_info);
py: Rework bytecode and .mpy file format to be mostly static data. Background: .mpy files are precompiled .py files, built using mpy-cross, that contain compiled bytecode functions (and can also contain machine code). The benefit of using an .mpy file over a .py file is that they are faster to import and take less memory when importing. They are also smaller on disk. But the real benefit of .mpy files comes when they are frozen into the firmware. This is done by loading the .mpy file during compilation of the firmware and turning it into a set of big C data structures (the job of mpy-tool.py), which are then compiled and downloaded into the ROM of a device. These C data structures can be executed in-place, ie directly from ROM. This makes importing even faster because there is very little to do, and also means such frozen modules take up much less RAM (because their bytecode stays in ROM). The downside of frozen code is that it requires recompiling and reflashing the entire firmware. This can be a big barrier to entry, slows down development time, and makes it harder to do OTA updates of frozen code (because the whole firmware must be updated). This commit attempts to solve this problem by providing a solution that sits between loading .mpy files into RAM and freezing them into the firmware. The .mpy file format has been reworked so that it consists of data and bytecode which is mostly static and ready to run in-place. If these new .mpy files are located in flash/ROM which is memory addressable, the .mpy file can be executed (mostly) in-place. With this approach there is still a small amount of unpacking and linking of the .mpy file that needs to be done when it's imported, but it's still much better than loading an .mpy from disk into RAM (although not as good as freezing .mpy files into the firmware). The main trick to make static .mpy files is to adjust the bytecode so any qstrs that it references now go through a lookup table to convert from local qstr number in the module to global qstr number in the firmware. That means the bytecode does not need linking/rewriting of qstrs when it's loaded. Instead only a small qstr table needs to be built (and put in RAM) at import time. This means the bytecode itself is static/constant and can be used directly if it's in addressable memory. Also the qstr string data in the .mpy file, and some constant object data, can be used directly. Note that the qstr table is global to the module (ie not per function). In more detail, in the VM what used to be (schematically): qst = DECODE_QSTR_VALUE; is now (schematically): idx = DECODE_QSTR_INDEX; qst = qstr_table[idx]; That allows the bytecode to be fixed at compile time and not need relinking/rewriting of the qstr values. Only qstr_table needs to be linked when the .mpy is loaded. Incidentally, this helps to reduce the size of bytecode because what used to be 2-byte qstr values in the bytecode are now (mostly) 1-byte indices. If the module uses the same qstr more than two times then the bytecode is smaller than before. The following changes are measured for this commit compared to the previous (the baseline): - average 7%-9% reduction in size of .mpy files - frozen code size is reduced by about 5%-7% - importing .py files uses about 5% less RAM in total - importing .mpy files uses about 4% less RAM in total - importing .py and .mpy files takes about the same time as before The qstr indirection in the bytecode has only a small impact on VM performance. For stm32 on PYBv1.0 the performance change of this commit is: diff of scores (higher is better) N=100 M=100 baseline -> this-commit diff diff% (error%) bm_chaos.py 371.07 -> 357.39 : -13.68 = -3.687% (+/-0.02%) bm_fannkuch.py 78.72 -> 77.49 : -1.23 = -1.563% (+/-0.01%) bm_fft.py 2591.73 -> 2539.28 : -52.45 = -2.024% (+/-0.00%) bm_float.py 6034.93 -> 5908.30 : -126.63 = -2.098% (+/-0.01%) bm_hexiom.py 48.96 -> 47.93 : -1.03 = -2.104% (+/-0.00%) bm_nqueens.py 4510.63 -> 4459.94 : -50.69 = -1.124% (+/-0.00%) bm_pidigits.py 650.28 -> 644.96 : -5.32 = -0.818% (+/-0.23%) core_import_mpy_multi.py 564.77 -> 581.49 : +16.72 = +2.960% (+/-0.01%) core_import_mpy_single.py 68.67 -> 67.16 : -1.51 = -2.199% (+/-0.01%) core_qstr.py 64.16 -> 64.12 : -0.04 = -0.062% (+/-0.00%) core_yield_from.py 362.58 -> 354.50 : -8.08 = -2.228% (+/-0.00%) misc_aes.py 429.69 -> 405.59 : -24.10 = -5.609% (+/-0.01%) misc_mandel.py 3485.13 -> 3416.51 : -68.62 = -1.969% (+/-0.00%) misc_pystone.py 2496.53 -> 2405.56 : -90.97 = -3.644% (+/-0.01%) misc_raytrace.py 381.47 -> 374.01 : -7.46 = -1.956% (+/-0.01%) viper_call0.py 576.73 -> 572.49 : -4.24 = -0.735% (+/-0.04%) viper_call1a.py 550.37 -> 546.21 : -4.16 = -0.756% (+/-0.09%) viper_call1b.py 438.23 -> 435.68 : -2.55 = -0.582% (+/-0.06%) viper_call1c.py 442.84 -> 440.04 : -2.80 = -0.632% (+/-0.08%) viper_call2a.py 536.31 -> 532.35 : -3.96 = -0.738% (+/-0.06%) viper_call2b.py 382.34 -> 377.07 : -5.27 = -1.378% (+/-0.03%) And for unix on x64: diff of scores (higher is better) N=2000 M=2000 baseline -> this-commit diff diff% (error%) bm_chaos.py 13594.20 -> 13073.84 : -520.36 = -3.828% (+/-5.44%) bm_fannkuch.py 60.63 -> 59.58 : -1.05 = -1.732% (+/-3.01%) bm_fft.py 112009.15 -> 111603.32 : -405.83 = -0.362% (+/-4.03%) bm_float.py 246202.55 -> 247923.81 : +1721.26 = +0.699% (+/-2.79%) bm_hexiom.py 615.65 -> 617.21 : +1.56 = +0.253% (+/-1.64%) bm_nqueens.py 215807.95 -> 215600.96 : -206.99 = -0.096% (+/-3.52%) bm_pidigits.py 8246.74 -> 8422.82 : +176.08 = +2.135% (+/-3.64%) misc_aes.py 16133.00 -> 16452.74 : +319.74 = +1.982% (+/-1.50%) misc_mandel.py 128146.69 -> 130796.43 : +2649.74 = +2.068% (+/-3.18%) misc_pystone.py 83811.49 -> 83124.85 : -686.64 = -0.819% (+/-1.03%) misc_raytrace.py 21688.02 -> 21385.10 : -302.92 = -1.397% (+/-3.20%) The code size change is (firmware with a lot of frozen code benefits the most): bare-arm: +396 +0.697% minimal x86: +1595 +0.979% [incl +32(data)] unix x64: +2408 +0.470% [incl +800(data)] unix nanbox: +1396 +0.309% [incl -96(data)] stm32: -1256 -0.318% PYBV10 cc3200: +288 +0.157% esp8266: -260 -0.037% GENERIC esp32: -216 -0.014% GENERIC[incl -1072(data)] nrf: +116 +0.067% pca10040 rp2: -664 -0.135% PICO samd: +844 +0.607% ADAFRUIT_ITSYBITSY_M4_EXPRESS As part of this change the .mpy file format version is bumped to version 6. And mpy-tool.py has been improved to provide a good visualisation of the contents of .mpy files. In summary: this commit changes the bytecode to use qstr indirection, and reworks the .mpy file format to be simpler and allow .mpy files to be executed in-place. Performance is not impacted too much. Eventually it will be possible to store such .mpy files in a linear, read-only, memory- mappable filesystem so they can be executed from flash/ROM. This will essentially be able to replace frozen code for most applications. Signed-off-by: Damien George <damien@micropython.org>
2021-10-22 12:22:47 +01:00
#if MICROPY_EMIT_BYTECODE_USES_QSTR_TABLE
block_name = cm->qstr_table[block_name];
qstr source_file = cm->qstr_table[0];
#else
qstr source_file = cm->source_file;
py: Add MICROPY_PERSISTENT_CODE so code can persist beyond the runtime. Main changes when MICROPY_PERSISTENT_CODE is enabled are: - qstrs are encoded as 2-byte fixed width in the bytecode - all pointers are removed from bytecode and put in const_table (this includes const objects and raw code pointers) Ultimately this option will enable persistence for not just bytecode but also native code.
2015-11-02 17:27:18 +00:00
#endif
py: Rework bytecode and .mpy file format to be mostly static data. Background: .mpy files are precompiled .py files, built using mpy-cross, that contain compiled bytecode functions (and can also contain machine code). The benefit of using an .mpy file over a .py file is that they are faster to import and take less memory when importing. They are also smaller on disk. But the real benefit of .mpy files comes when they are frozen into the firmware. This is done by loading the .mpy file during compilation of the firmware and turning it into a set of big C data structures (the job of mpy-tool.py), which are then compiled and downloaded into the ROM of a device. These C data structures can be executed in-place, ie directly from ROM. This makes importing even faster because there is very little to do, and also means such frozen modules take up much less RAM (because their bytecode stays in ROM). The downside of frozen code is that it requires recompiling and reflashing the entire firmware. This can be a big barrier to entry, slows down development time, and makes it harder to do OTA updates of frozen code (because the whole firmware must be updated). This commit attempts to solve this problem by providing a solution that sits between loading .mpy files into RAM and freezing them into the firmware. The .mpy file format has been reworked so that it consists of data and bytecode which is mostly static and ready to run in-place. If these new .mpy files are located in flash/ROM which is memory addressable, the .mpy file can be executed (mostly) in-place. With this approach there is still a small amount of unpacking and linking of the .mpy file that needs to be done when it's imported, but it's still much better than loading an .mpy from disk into RAM (although not as good as freezing .mpy files into the firmware). The main trick to make static .mpy files is to adjust the bytecode so any qstrs that it references now go through a lookup table to convert from local qstr number in the module to global qstr number in the firmware. That means the bytecode does not need linking/rewriting of qstrs when it's loaded. Instead only a small qstr table needs to be built (and put in RAM) at import time. This means the bytecode itself is static/constant and can be used directly if it's in addressable memory. Also the qstr string data in the .mpy file, and some constant object data, can be used directly. Note that the qstr table is global to the module (ie not per function). In more detail, in the VM what used to be (schematically): qst = DECODE_QSTR_VALUE; is now (schematically): idx = DECODE_QSTR_INDEX; qst = qstr_table[idx]; That allows the bytecode to be fixed at compile time and not need relinking/rewriting of the qstr values. Only qstr_table needs to be linked when the .mpy is loaded. Incidentally, this helps to reduce the size of bytecode because what used to be 2-byte qstr values in the bytecode are now (mostly) 1-byte indices. If the module uses the same qstr more than two times then the bytecode is smaller than before. The following changes are measured for this commit compared to the previous (the baseline): - average 7%-9% reduction in size of .mpy files - frozen code size is reduced by about 5%-7% - importing .py files uses about 5% less RAM in total - importing .mpy files uses about 4% less RAM in total - importing .py and .mpy files takes about the same time as before The qstr indirection in the bytecode has only a small impact on VM performance. For stm32 on PYBv1.0 the performance change of this commit is: diff of scores (higher is better) N=100 M=100 baseline -> this-commit diff diff% (error%) bm_chaos.py 371.07 -> 357.39 : -13.68 = -3.687% (+/-0.02%) bm_fannkuch.py 78.72 -> 77.49 : -1.23 = -1.563% (+/-0.01%) bm_fft.py 2591.73 -> 2539.28 : -52.45 = -2.024% (+/-0.00%) bm_float.py 6034.93 -> 5908.30 : -126.63 = -2.098% (+/-0.01%) bm_hexiom.py 48.96 -> 47.93 : -1.03 = -2.104% (+/-0.00%) bm_nqueens.py 4510.63 -> 4459.94 : -50.69 = -1.124% (+/-0.00%) bm_pidigits.py 650.28 -> 644.96 : -5.32 = -0.818% (+/-0.23%) core_import_mpy_multi.py 564.77 -> 581.49 : +16.72 = +2.960% (+/-0.01%) core_import_mpy_single.py 68.67 -> 67.16 : -1.51 = -2.199% (+/-0.01%) core_qstr.py 64.16 -> 64.12 : -0.04 = -0.062% (+/-0.00%) core_yield_from.py 362.58 -> 354.50 : -8.08 = -2.228% (+/-0.00%) misc_aes.py 429.69 -> 405.59 : -24.10 = -5.609% (+/-0.01%) misc_mandel.py 3485.13 -> 3416.51 : -68.62 = -1.969% (+/-0.00%) misc_pystone.py 2496.53 -> 2405.56 : -90.97 = -3.644% (+/-0.01%) misc_raytrace.py 381.47 -> 374.01 : -7.46 = -1.956% (+/-0.01%) viper_call0.py 576.73 -> 572.49 : -4.24 = -0.735% (+/-0.04%) viper_call1a.py 550.37 -> 546.21 : -4.16 = -0.756% (+/-0.09%) viper_call1b.py 438.23 -> 435.68 : -2.55 = -0.582% (+/-0.06%) viper_call1c.py 442.84 -> 440.04 : -2.80 = -0.632% (+/-0.08%) viper_call2a.py 536.31 -> 532.35 : -3.96 = -0.738% (+/-0.06%) viper_call2b.py 382.34 -> 377.07 : -5.27 = -1.378% (+/-0.03%) And for unix on x64: diff of scores (higher is better) N=2000 M=2000 baseline -> this-commit diff diff% (error%) bm_chaos.py 13594.20 -> 13073.84 : -520.36 = -3.828% (+/-5.44%) bm_fannkuch.py 60.63 -> 59.58 : -1.05 = -1.732% (+/-3.01%) bm_fft.py 112009.15 -> 111603.32 : -405.83 = -0.362% (+/-4.03%) bm_float.py 246202.55 -> 247923.81 : +1721.26 = +0.699% (+/-2.79%) bm_hexiom.py 615.65 -> 617.21 : +1.56 = +0.253% (+/-1.64%) bm_nqueens.py 215807.95 -> 215600.96 : -206.99 = -0.096% (+/-3.52%) bm_pidigits.py 8246.74 -> 8422.82 : +176.08 = +2.135% (+/-3.64%) misc_aes.py 16133.00 -> 16452.74 : +319.74 = +1.982% (+/-1.50%) misc_mandel.py 128146.69 -> 130796.43 : +2649.74 = +2.068% (+/-3.18%) misc_pystone.py 83811.49 -> 83124.85 : -686.64 = -0.819% (+/-1.03%) misc_raytrace.py 21688.02 -> 21385.10 : -302.92 = -1.397% (+/-3.20%) The code size change is (firmware with a lot of frozen code benefits the most): bare-arm: +396 +0.697% minimal x86: +1595 +0.979% [incl +32(data)] unix x64: +2408 +0.470% [incl +800(data)] unix nanbox: +1396 +0.309% [incl -96(data)] stm32: -1256 -0.318% PYBV10 cc3200: +288 +0.157% esp8266: -260 -0.037% GENERIC esp32: -216 -0.014% GENERIC[incl -1072(data)] nrf: +116 +0.067% pca10040 rp2: -664 -0.135% PICO samd: +844 +0.607% ADAFRUIT_ITSYBITSY_M4_EXPRESS As part of this change the .mpy file format version is bumped to version 6. And mpy-tool.py has been improved to provide a good visualisation of the contents of .mpy files. In summary: this commit changes the bytecode to use qstr indirection, and reworks the .mpy file format to be simpler and allow .mpy files to be executed in-place. Performance is not impacted too much. Eventually it will be possible to store such .mpy files in a linear, read-only, memory- mappable filesystem so they can be executed from flash/ROM. This will essentially be able to replace frozen code for most applications. Signed-off-by: Damien George <damien@micropython.org>
2021-10-22 12:22:47 +01:00
mp_printf(print, "File %s, code block '%s' (descriptor: %p, bytecode @%p " UINT_FMT " bytes)\n",// rct=%p\n",
qstr_str(source_file), qstr_str(block_name), descr, mp_showbc_code_start, len);// , ((mp_raw_code_t*)descr)->children);
showbc: Print code block header at the beginning, not in the middle of dump. Also, dump code block in bytes.
2014-06-02 14:11:16 +01:00
py: Fix debug-printing of bytecode line numbers. Also move the raw bytecode printing code from emitglue to mp_bytecode_print.
2014-10-24 15:42:50 +01:00
// raw bytecode dump
py: Rework and compress second part of bytecode prelude. This patch compresses the second part of the bytecode prelude which contains the source file name, function name, source-line-number mapping and cell closure information. This part of the prelude now begins with a single varible length unsigned integer which encodes 2 numbers, being the byte-size of the following 2 sections in the header: the "source info section" and the "closure section". After decoding this variable unsigned integer it's possible to skip over one or both of these sections very easily. This scheme saves about 2 bytes for most functions compared to the original format: one in the case that there are no closure cells, and one because padding was eliminated.
2019-09-25 06:45:47 +01:00
size_t prelude_size = ip - mp_showbc_code_start + n_info + n_cell;
py/showbc: Fix printing of raw bytecode header on nanbox builds. Signed-off-by: Damien George <damien@micropython.org>
2021-12-15 05:54:47 +00:00
mp_printf(print, "Raw bytecode (code_info_size=%u, bytecode_size=%u):\n",
(unsigned)prelude_size, (unsigned)(len - prelude_size));
py: Fix debug-printing of bytecode line numbers. Also move the raw bytecode printing code from emitglue to mp_bytecode_print.
2014-10-24 15:42:50 +01:00
for (mp_uint_t i = 0; i < len; i++) {
if (i > 0 && i % 16 == 0) {
py/showbc: Pass in an mp_print_t struct to all bytecode-print functions. So the output can be redirected if needed. Signed-off-by: Damien George <damien@micropython.org>
2020-09-11 08:22:28 +01:00
mp_printf(print, "\n");
py: Fix debug-printing of bytecode line numbers. Also move the raw bytecode printing code from emitglue to mp_bytecode_print.
2014-10-24 15:42:50 +01:00
}
py/showbc: Pass in an mp_print_t struct to all bytecode-print functions. So the output can be redirected if needed. Signed-off-by: Damien George <damien@micropython.org>
2020-09-11 08:22:28 +01:00
mp_printf(print, " %02x", mp_showbc_code_start[i]);
py: Fix debug-printing of bytecode line numbers. Also move the raw bytecode printing code from emitglue to mp_bytecode_print.
2014-10-24 15:42:50 +01:00
}
py/showbc: Pass in an mp_print_t struct to all bytecode-print functions. So the output can be redirected if needed. Signed-off-by: Damien George <damien@micropython.org>
2020-09-11 08:22:28 +01:00
mp_printf(print, "\n");
py: Fix debug-printing of bytecode line numbers. Also move the raw bytecode printing code from emitglue to mp_bytecode_print.
2014-10-24 15:42:50 +01:00
py: Store bytecode arg names in bytecode (were in own array). This saves a lot of RAM for 2 reasons: 1. For functions that don't have default values, var args or var kw args (which is a large number of functions in the general case), the mp_obj_fun_bc_t type now fits in 1 GC block (previously needed 2 because of the extra pointer to point to the arg_names array). So this saves 16 bytes per function (32 bytes on 64-bit machines). 2. Combining separate memory regions generally saves RAM because the unused bytes at the end of the GC block are saved for 1 of the blocks (since that block doesn't exist on its own anymore). So generally this saves 8 bytes per function. Tested by importing lots of modules: - 64-bit Linux gave about an 8% RAM saving for 86k of used RAM. - pyboard gave about a 6% RAM saving for 31k of used RAM.
2014-10-25 15:07:02 +01:00
// bytecode prelude: arg names (as qstr objects)
py/showbc: Pass in an mp_print_t struct to all bytecode-print functions. So the output can be redirected if needed. Signed-off-by: Damien George <damien@micropython.org>
2020-09-11 08:22:28 +01:00
mp_printf(print, "arg names:");
py: Put all bytecode state (arg count, etc) in bytecode.
2015-10-22 23:45:37 +01:00
for (mp_uint_t i = 0; i < n_pos_args + n_kwonly_args; i++) {
py: Rework bytecode and .mpy file format to be mostly static data. Background: .mpy files are precompiled .py files, built using mpy-cross, that contain compiled bytecode functions (and can also contain machine code). The benefit of using an .mpy file over a .py file is that they are faster to import and take less memory when importing. They are also smaller on disk. But the real benefit of .mpy files comes when they are frozen into the firmware. This is done by loading the .mpy file during compilation of the firmware and turning it into a set of big C data structures (the job of mpy-tool.py), which are then compiled and downloaded into the ROM of a device. These C data structures can be executed in-place, ie directly from ROM. This makes importing even faster because there is very little to do, and also means such frozen modules take up much less RAM (because their bytecode stays in ROM). The downside of frozen code is that it requires recompiling and reflashing the entire firmware. This can be a big barrier to entry, slows down development time, and makes it harder to do OTA updates of frozen code (because the whole firmware must be updated). This commit attempts to solve this problem by providing a solution that sits between loading .mpy files into RAM and freezing them into the firmware. The .mpy file format has been reworked so that it consists of data and bytecode which is mostly static and ready to run in-place. If these new .mpy files are located in flash/ROM which is memory addressable, the .mpy file can be executed (mostly) in-place. With this approach there is still a small amount of unpacking and linking of the .mpy file that needs to be done when it's imported, but it's still much better than loading an .mpy from disk into RAM (although not as good as freezing .mpy files into the firmware). The main trick to make static .mpy files is to adjust the bytecode so any qstrs that it references now go through a lookup table to convert from local qstr number in the module to global qstr number in the firmware. That means the bytecode does not need linking/rewriting of qstrs when it's loaded. Instead only a small qstr table needs to be built (and put in RAM) at import time. This means the bytecode itself is static/constant and can be used directly if it's in addressable memory. Also the qstr string data in the .mpy file, and some constant object data, can be used directly. Note that the qstr table is global to the module (ie not per function). In more detail, in the VM what used to be (schematically): qst = DECODE_QSTR_VALUE; is now (schematically): idx = DECODE_QSTR_INDEX; qst = qstr_table[idx]; That allows the bytecode to be fixed at compile time and not need relinking/rewriting of the qstr values. Only qstr_table needs to be linked when the .mpy is loaded. Incidentally, this helps to reduce the size of bytecode because what used to be 2-byte qstr values in the bytecode are now (mostly) 1-byte indices. If the module uses the same qstr more than two times then the bytecode is smaller than before. The following changes are measured for this commit compared to the previous (the baseline): - average 7%-9% reduction in size of .mpy files - frozen code size is reduced by about 5%-7% - importing .py files uses about 5% less RAM in total - importing .mpy files uses about 4% less RAM in total - importing .py and .mpy files takes about the same time as before The qstr indirection in the bytecode has only a small impact on VM performance. For stm32 on PYBv1.0 the performance change of this commit is: diff of scores (higher is better) N=100 M=100 baseline -> this-commit diff diff% (error%) bm_chaos.py 371.07 -> 357.39 : -13.68 = -3.687% (+/-0.02%) bm_fannkuch.py 78.72 -> 77.49 : -1.23 = -1.563% (+/-0.01%) bm_fft.py 2591.73 -> 2539.28 : -52.45 = -2.024% (+/-0.00%) bm_float.py 6034.93 -> 5908.30 : -126.63 = -2.098% (+/-0.01%) bm_hexiom.py 48.96 -> 47.93 : -1.03 = -2.104% (+/-0.00%) bm_nqueens.py 4510.63 -> 4459.94 : -50.69 = -1.124% (+/-0.00%) bm_pidigits.py 650.28 -> 644.96 : -5.32 = -0.818% (+/-0.23%) core_import_mpy_multi.py 564.77 -> 581.49 : +16.72 = +2.960% (+/-0.01%) core_import_mpy_single.py 68.67 -> 67.16 : -1.51 = -2.199% (+/-0.01%) core_qstr.py 64.16 -> 64.12 : -0.04 = -0.062% (+/-0.00%) core_yield_from.py 362.58 -> 354.50 : -8.08 = -2.228% (+/-0.00%) misc_aes.py 429.69 -> 405.59 : -24.10 = -5.609% (+/-0.01%) misc_mandel.py 3485.13 -> 3416.51 : -68.62 = -1.969% (+/-0.00%) misc_pystone.py 2496.53 -> 2405.56 : -90.97 = -3.644% (+/-0.01%) misc_raytrace.py 381.47 -> 374.01 : -7.46 = -1.956% (+/-0.01%) viper_call0.py 576.73 -> 572.49 : -4.24 = -0.735% (+/-0.04%) viper_call1a.py 550.37 -> 546.21 : -4.16 = -0.756% (+/-0.09%) viper_call1b.py 438.23 -> 435.68 : -2.55 = -0.582% (+/-0.06%) viper_call1c.py 442.84 -> 440.04 : -2.80 = -0.632% (+/-0.08%) viper_call2a.py 536.31 -> 532.35 : -3.96 = -0.738% (+/-0.06%) viper_call2b.py 382.34 -> 377.07 : -5.27 = -1.378% (+/-0.03%) And for unix on x64: diff of scores (higher is better) N=2000 M=2000 baseline -> this-commit diff diff% (error%) bm_chaos.py 13594.20 -> 13073.84 : -520.36 = -3.828% (+/-5.44%) bm_fannkuch.py 60.63 -> 59.58 : -1.05 = -1.732% (+/-3.01%) bm_fft.py 112009.15 -> 111603.32 : -405.83 = -0.362% (+/-4.03%) bm_float.py 246202.55 -> 247923.81 : +1721.26 = +0.699% (+/-2.79%) bm_hexiom.py 615.65 -> 617.21 : +1.56 = +0.253% (+/-1.64%) bm_nqueens.py 215807.95 -> 215600.96 : -206.99 = -0.096% (+/-3.52%) bm_pidigits.py 8246.74 -> 8422.82 : +176.08 = +2.135% (+/-3.64%) misc_aes.py 16133.00 -> 16452.74 : +319.74 = +1.982% (+/-1.50%) misc_mandel.py 128146.69 -> 130796.43 : +2649.74 = +2.068% (+/-3.18%) misc_pystone.py 83811.49 -> 83124.85 : -686.64 = -0.819% (+/-1.03%) misc_raytrace.py 21688.02 -> 21385.10 : -302.92 = -1.397% (+/-3.20%) The code size change is (firmware with a lot of frozen code benefits the most): bare-arm: +396 +0.697% minimal x86: +1595 +0.979% [incl +32(data)] unix x64: +2408 +0.470% [incl +800(data)] unix nanbox: +1396 +0.309% [incl -96(data)] stm32: -1256 -0.318% PYBV10 cc3200: +288 +0.157% esp8266: -260 -0.037% GENERIC esp32: -216 -0.014% GENERIC[incl -1072(data)] nrf: +116 +0.067% pca10040 rp2: -664 -0.135% PICO samd: +844 +0.607% ADAFRUIT_ITSYBITSY_M4_EXPRESS As part of this change the .mpy file format version is bumped to version 6. And mpy-tool.py has been improved to provide a good visualisation of the contents of .mpy files. In summary: this commit changes the bytecode to use qstr indirection, and reworks the .mpy file format to be simpler and allow .mpy files to be executed in-place. Performance is not impacted too much. Eventually it will be possible to store such .mpy files in a linear, read-only, memory- mappable filesystem so they can be executed from flash/ROM. This will essentially be able to replace frozen code for most applications. Signed-off-by: Damien George <damien@micropython.org>
2021-10-22 12:22:47 +01:00
qstr qst = mp_decode_uint(&code_info);
#if MICROPY_EMIT_BYTECODE_USES_QSTR_TABLE
qst = cm->qstr_table[qst];
#endif
mp_printf(print, " %s", qstr_str(qst));
py: Store bytecode arg names in bytecode (were in own array). This saves a lot of RAM for 2 reasons: 1. For functions that don't have default values, var args or var kw args (which is a large number of functions in the general case), the mp_obj_fun_bc_t type now fits in 1 GC block (previously needed 2 because of the extra pointer to point to the arg_names array). So this saves 16 bytes per function (32 bytes on 64-bit machines). 2. Combining separate memory regions generally saves RAM because the unused bytes at the end of the GC block are saved for 1 of the blocks (since that block doesn't exist on its own anymore). So generally this saves 8 bytes per function. Tested by importing lots of modules: - 64-bit Linux gave about an 8% RAM saving for 86k of used RAM. - pyboard gave about a 6% RAM saving for 31k of used RAM.
2014-10-25 15:07:02 +01:00
}
py/showbc: Pass in an mp_print_t struct to all bytecode-print functions. So the output can be redirected if needed. Signed-off-by: Damien George <damien@micropython.org>
2020-09-11 08:22:28 +01:00
mp_printf(print, "\n");
py: Store bytecode arg names in bytecode (were in own array). This saves a lot of RAM for 2 reasons: 1. For functions that don't have default values, var args or var kw args (which is a large number of functions in the general case), the mp_obj_fun_bc_t type now fits in 1 GC block (previously needed 2 because of the extra pointer to point to the arg_names array). So this saves 16 bytes per function (32 bytes on 64-bit machines). 2. Combining separate memory regions generally saves RAM because the unused bytes at the end of the GC block are saved for 1 of the blocks (since that block doesn't exist on its own anymore). So generally this saves 8 bytes per function. Tested by importing lots of modules: - 64-bit Linux gave about an 8% RAM saving for 86k of used RAM. - pyboard gave about a 6% RAM saving for 31k of used RAM.
2014-10-25 15:07:02 +01:00
py/showbc: Pass in an mp_print_t struct to all bytecode-print functions. So the output can be redirected if needed. Signed-off-by: Damien George <damien@micropython.org>
2020-09-11 08:22:28 +01:00
mp_printf(print, "(N_STATE %u)\n", (unsigned)n_state);
mp_printf(print, "(N_EXC_STACK %u)\n", (unsigned)n_exc_stack);
py: Reorganise bytecode layout so it's more structured, easier to edit.
2015-03-18 17:47:47 +00:00
py: Rework and compress second part of bytecode prelude. This patch compresses the second part of the bytecode prelude which contains the source file name, function name, source-line-number mapping and cell closure information. This part of the prelude now begins with a single varible length unsigned integer which encodes 2 numbers, being the byte-size of the following 2 sections in the header: the "source info section" and the "closure section". After decoding this variable unsigned integer it's possible to skip over one or both of these sections very easily. This scheme saves about 2 bytes for most functions compared to the original format: one in the case that there are no closure cells, and one because padding was eliminated.
2019-09-25 06:45:47 +01:00
// skip over code_info
ip += n_info;
py: Rework bytecode and .mpy file format to be mostly static data. Background: .mpy files are precompiled .py files, built using mpy-cross, that contain compiled bytecode functions (and can also contain machine code). The benefit of using an .mpy file over a .py file is that they are faster to import and take less memory when importing. They are also smaller on disk. But the real benefit of .mpy files comes when they are frozen into the firmware. This is done by loading the .mpy file during compilation of the firmware and turning it into a set of big C data structures (the job of mpy-tool.py), which are then compiled and downloaded into the ROM of a device. These C data structures can be executed in-place, ie directly from ROM. This makes importing even faster because there is very little to do, and also means such frozen modules take up much less RAM (because their bytecode stays in ROM). The downside of frozen code is that it requires recompiling and reflashing the entire firmware. This can be a big barrier to entry, slows down development time, and makes it harder to do OTA updates of frozen code (because the whole firmware must be updated). This commit attempts to solve this problem by providing a solution that sits between loading .mpy files into RAM and freezing them into the firmware. The .mpy file format has been reworked so that it consists of data and bytecode which is mostly static and ready to run in-place. If these new .mpy files are located in flash/ROM which is memory addressable, the .mpy file can be executed (mostly) in-place. With this approach there is still a small amount of unpacking and linking of the .mpy file that needs to be done when it's imported, but it's still much better than loading an .mpy from disk into RAM (although not as good as freezing .mpy files into the firmware). The main trick to make static .mpy files is to adjust the bytecode so any qstrs that it references now go through a lookup table to convert from local qstr number in the module to global qstr number in the firmware. That means the bytecode does not need linking/rewriting of qstrs when it's loaded. Instead only a small qstr table needs to be built (and put in RAM) at import time. This means the bytecode itself is static/constant and can be used directly if it's in addressable memory. Also the qstr string data in the .mpy file, and some constant object data, can be used directly. Note that the qstr table is global to the module (ie not per function). In more detail, in the VM what used to be (schematically): qst = DECODE_QSTR_VALUE; is now (schematically): idx = DECODE_QSTR_INDEX; qst = qstr_table[idx]; That allows the bytecode to be fixed at compile time and not need relinking/rewriting of the qstr values. Only qstr_table needs to be linked when the .mpy is loaded. Incidentally, this helps to reduce the size of bytecode because what used to be 2-byte qstr values in the bytecode are now (mostly) 1-byte indices. If the module uses the same qstr more than two times then the bytecode is smaller than before. The following changes are measured for this commit compared to the previous (the baseline): - average 7%-9% reduction in size of .mpy files - frozen code size is reduced by about 5%-7% - importing .py files uses about 5% less RAM in total - importing .mpy files uses about 4% less RAM in total - importing .py and .mpy files takes about the same time as before The qstr indirection in the bytecode has only a small impact on VM performance. For stm32 on PYBv1.0 the performance change of this commit is: diff of scores (higher is better) N=100 M=100 baseline -> this-commit diff diff% (error%) bm_chaos.py 371.07 -> 357.39 : -13.68 = -3.687% (+/-0.02%) bm_fannkuch.py 78.72 -> 77.49 : -1.23 = -1.563% (+/-0.01%) bm_fft.py 2591.73 -> 2539.28 : -52.45 = -2.024% (+/-0.00%) bm_float.py 6034.93 -> 5908.30 : -126.63 = -2.098% (+/-0.01%) bm_hexiom.py 48.96 -> 47.93 : -1.03 = -2.104% (+/-0.00%) bm_nqueens.py 4510.63 -> 4459.94 : -50.69 = -1.124% (+/-0.00%) bm_pidigits.py 650.28 -> 644.96 : -5.32 = -0.818% (+/-0.23%) core_import_mpy_multi.py 564.77 -> 581.49 : +16.72 = +2.960% (+/-0.01%) core_import_mpy_single.py 68.67 -> 67.16 : -1.51 = -2.199% (+/-0.01%) core_qstr.py 64.16 -> 64.12 : -0.04 = -0.062% (+/-0.00%) core_yield_from.py 362.58 -> 354.50 : -8.08 = -2.228% (+/-0.00%) misc_aes.py 429.69 -> 405.59 : -24.10 = -5.609% (+/-0.01%) misc_mandel.py 3485.13 -> 3416.51 : -68.62 = -1.969% (+/-0.00%) misc_pystone.py 2496.53 -> 2405.56 : -90.97 = -3.644% (+/-0.01%) misc_raytrace.py 381.47 -> 374.01 : -7.46 = -1.956% (+/-0.01%) viper_call0.py 576.73 -> 572.49 : -4.24 = -0.735% (+/-0.04%) viper_call1a.py 550.37 -> 546.21 : -4.16 = -0.756% (+/-0.09%) viper_call1b.py 438.23 -> 435.68 : -2.55 = -0.582% (+/-0.06%) viper_call1c.py 442.84 -> 440.04 : -2.80 = -0.632% (+/-0.08%) viper_call2a.py 536.31 -> 532.35 : -3.96 = -0.738% (+/-0.06%) viper_call2b.py 382.34 -> 377.07 : -5.27 = -1.378% (+/-0.03%) And for unix on x64: diff of scores (higher is better) N=2000 M=2000 baseline -> this-commit diff diff% (error%) bm_chaos.py 13594.20 -> 13073.84 : -520.36 = -3.828% (+/-5.44%) bm_fannkuch.py 60.63 -> 59.58 : -1.05 = -1.732% (+/-3.01%) bm_fft.py 112009.15 -> 111603.32 : -405.83 = -0.362% (+/-4.03%) bm_float.py 246202.55 -> 247923.81 : +1721.26 = +0.699% (+/-2.79%) bm_hexiom.py 615.65 -> 617.21 : +1.56 = +0.253% (+/-1.64%) bm_nqueens.py 215807.95 -> 215600.96 : -206.99 = -0.096% (+/-3.52%) bm_pidigits.py 8246.74 -> 8422.82 : +176.08 = +2.135% (+/-3.64%) misc_aes.py 16133.00 -> 16452.74 : +319.74 = +1.982% (+/-1.50%) misc_mandel.py 128146.69 -> 130796.43 : +2649.74 = +2.068% (+/-3.18%) misc_pystone.py 83811.49 -> 83124.85 : -686.64 = -0.819% (+/-1.03%) misc_raytrace.py 21688.02 -> 21385.10 : -302.92 = -1.397% (+/-3.20%) The code size change is (firmware with a lot of frozen code benefits the most): bare-arm: +396 +0.697% minimal x86: +1595 +0.979% [incl +32(data)] unix x64: +2408 +0.470% [incl +800(data)] unix nanbox: +1396 +0.309% [incl -96(data)] stm32: -1256 -0.318% PYBV10 cc3200: +288 +0.157% esp8266: -260 -0.037% GENERIC esp32: -216 -0.014% GENERIC[incl -1072(data)] nrf: +116 +0.067% pca10040 rp2: -664 -0.135% PICO samd: +844 +0.607% ADAFRUIT_ITSYBITSY_M4_EXPRESS As part of this change the .mpy file format version is bumped to version 6. And mpy-tool.py has been improved to provide a good visualisation of the contents of .mpy files. In summary: this commit changes the bytecode to use qstr indirection, and reworks the .mpy file format to be simpler and allow .mpy files to be executed in-place. Performance is not impacted too much. Eventually it will be possible to store such .mpy files in a linear, read-only, memory- mappable filesystem so they can be executed from flash/ROM. This will essentially be able to replace frozen code for most applications. Signed-off-by: Damien George <damien@micropython.org>
2021-10-22 12:22:47 +01:00
const byte *line_info_top = ip;
py: Fix bugs with debugging output. show_bc now decodes the prelude correctly. Moved WRITE_FILE stuff from runtime.c to emitglue.c. Addresses issue #385.
2014-03-28 18:38:20 +00:00
// bytecode prelude: initialise closed over variables
py: Rework and compress second part of bytecode prelude. This patch compresses the second part of the bytecode prelude which contains the source file name, function name, source-line-number mapping and cell closure information. This part of the prelude now begins with a single varible length unsigned integer which encodes 2 numbers, being the byte-size of the following 2 sections in the header: the "source info section" and the "closure section". After decoding this variable unsigned integer it's possible to skip over one or both of these sections very easily. This scheme saves about 2 bytes for most functions compared to the original format: one in the case that there are no closure cells, and one because padding was eliminated.
2019-09-25 06:45:47 +01:00
for (size_t i = 0; i < n_cell; ++i) {
uint local_num = *ip++;
py/showbc: Pass in an mp_print_t struct to all bytecode-print functions. So the output can be redirected if needed. Signed-off-by: Damien George <damien@micropython.org>
2020-09-11 08:22:28 +01:00
mp_printf(print, "(INIT_CELL %u)\n", local_num);
py: make closures work.
2013-12-30 22:32:17 +00:00
}
py: Fix up source-line calculation. Should address issue #475.
2014-04-13 14:51:56 +01:00
// print out line number info
{
py: Rework and compress second part of bytecode prelude. This patch compresses the second part of the bytecode prelude which contains the source file name, function name, source-line-number mapping and cell closure information. This part of the prelude now begins with a single varible length unsigned integer which encodes 2 numbers, being the byte-size of the following 2 sections in the header: the "source info section" and the "closure section". After decoding this variable unsigned integer it's possible to skip over one or both of these sections very easily. This scheme saves about 2 bytes for most functions compared to the original format: one in the case that there are no closure cells, and one because padding was eliminated.
2019-09-25 06:45:47 +01:00
mp_int_t bc = 0;
Rename machine_(u)int_t to mp_(u)int_t. See discussion in issue #50.
2014-07-03 13:25:24 +01:00
mp_uint_t source_line = 1;
py/showbc: Pass in an mp_print_t struct to all bytecode-print functions. So the output can be redirected if needed. Signed-off-by: Damien George <damien@micropython.org>
2020-09-11 08:22:28 +01:00
mp_printf(print, " bc=" INT_FMT " line=" UINT_FMT "\n", bc, source_line);
py: Rework bytecode and .mpy file format to be mostly static data. Background: .mpy files are precompiled .py files, built using mpy-cross, that contain compiled bytecode functions (and can also contain machine code). The benefit of using an .mpy file over a .py file is that they are faster to import and take less memory when importing. They are also smaller on disk. But the real benefit of .mpy files comes when they are frozen into the firmware. This is done by loading the .mpy file during compilation of the firmware and turning it into a set of big C data structures (the job of mpy-tool.py), which are then compiled and downloaded into the ROM of a device. These C data structures can be executed in-place, ie directly from ROM. This makes importing even faster because there is very little to do, and also means such frozen modules take up much less RAM (because their bytecode stays in ROM). The downside of frozen code is that it requires recompiling and reflashing the entire firmware. This can be a big barrier to entry, slows down development time, and makes it harder to do OTA updates of frozen code (because the whole firmware must be updated). This commit attempts to solve this problem by providing a solution that sits between loading .mpy files into RAM and freezing them into the firmware. The .mpy file format has been reworked so that it consists of data and bytecode which is mostly static and ready to run in-place. If these new .mpy files are located in flash/ROM which is memory addressable, the .mpy file can be executed (mostly) in-place. With this approach there is still a small amount of unpacking and linking of the .mpy file that needs to be done when it's imported, but it's still much better than loading an .mpy from disk into RAM (although not as good as freezing .mpy files into the firmware). The main trick to make static .mpy files is to adjust the bytecode so any qstrs that it references now go through a lookup table to convert from local qstr number in the module to global qstr number in the firmware. That means the bytecode does not need linking/rewriting of qstrs when it's loaded. Instead only a small qstr table needs to be built (and put in RAM) at import time. This means the bytecode itself is static/constant and can be used directly if it's in addressable memory. Also the qstr string data in the .mpy file, and some constant object data, can be used directly. Note that the qstr table is global to the module (ie not per function). In more detail, in the VM what used to be (schematically): qst = DECODE_QSTR_VALUE; is now (schematically): idx = DECODE_QSTR_INDEX; qst = qstr_table[idx]; That allows the bytecode to be fixed at compile time and not need relinking/rewriting of the qstr values. Only qstr_table needs to be linked when the .mpy is loaded. Incidentally, this helps to reduce the size of bytecode because what used to be 2-byte qstr values in the bytecode are now (mostly) 1-byte indices. If the module uses the same qstr more than two times then the bytecode is smaller than before. The following changes are measured for this commit compared to the previous (the baseline): - average 7%-9% reduction in size of .mpy files - frozen code size is reduced by about 5%-7% - importing .py files uses about 5% less RAM in total - importing .mpy files uses about 4% less RAM in total - importing .py and .mpy files takes about the same time as before The qstr indirection in the bytecode has only a small impact on VM performance. For stm32 on PYBv1.0 the performance change of this commit is: diff of scores (higher is better) N=100 M=100 baseline -> this-commit diff diff% (error%) bm_chaos.py 371.07 -> 357.39 : -13.68 = -3.687% (+/-0.02%) bm_fannkuch.py 78.72 -> 77.49 : -1.23 = -1.563% (+/-0.01%) bm_fft.py 2591.73 -> 2539.28 : -52.45 = -2.024% (+/-0.00%) bm_float.py 6034.93 -> 5908.30 : -126.63 = -2.098% (+/-0.01%) bm_hexiom.py 48.96 -> 47.93 : -1.03 = -2.104% (+/-0.00%) bm_nqueens.py 4510.63 -> 4459.94 : -50.69 = -1.124% (+/-0.00%) bm_pidigits.py 650.28 -> 644.96 : -5.32 = -0.818% (+/-0.23%) core_import_mpy_multi.py 564.77 -> 581.49 : +16.72 = +2.960% (+/-0.01%) core_import_mpy_single.py 68.67 -> 67.16 : -1.51 = -2.199% (+/-0.01%) core_qstr.py 64.16 -> 64.12 : -0.04 = -0.062% (+/-0.00%) core_yield_from.py 362.58 -> 354.50 : -8.08 = -2.228% (+/-0.00%) misc_aes.py 429.69 -> 405.59 : -24.10 = -5.609% (+/-0.01%) misc_mandel.py 3485.13 -> 3416.51 : -68.62 = -1.969% (+/-0.00%) misc_pystone.py 2496.53 -> 2405.56 : -90.97 = -3.644% (+/-0.01%) misc_raytrace.py 381.47 -> 374.01 : -7.46 = -1.956% (+/-0.01%) viper_call0.py 576.73 -> 572.49 : -4.24 = -0.735% (+/-0.04%) viper_call1a.py 550.37 -> 546.21 : -4.16 = -0.756% (+/-0.09%) viper_call1b.py 438.23 -> 435.68 : -2.55 = -0.582% (+/-0.06%) viper_call1c.py 442.84 -> 440.04 : -2.80 = -0.632% (+/-0.08%) viper_call2a.py 536.31 -> 532.35 : -3.96 = -0.738% (+/-0.06%) viper_call2b.py 382.34 -> 377.07 : -5.27 = -1.378% (+/-0.03%) And for unix on x64: diff of scores (higher is better) N=2000 M=2000 baseline -> this-commit diff diff% (error%) bm_chaos.py 13594.20 -> 13073.84 : -520.36 = -3.828% (+/-5.44%) bm_fannkuch.py 60.63 -> 59.58 : -1.05 = -1.732% (+/-3.01%) bm_fft.py 112009.15 -> 111603.32 : -405.83 = -0.362% (+/-4.03%) bm_float.py 246202.55 -> 247923.81 : +1721.26 = +0.699% (+/-2.79%) bm_hexiom.py 615.65 -> 617.21 : +1.56 = +0.253% (+/-1.64%) bm_nqueens.py 215807.95 -> 215600.96 : -206.99 = -0.096% (+/-3.52%) bm_pidigits.py 8246.74 -> 8422.82 : +176.08 = +2.135% (+/-3.64%) misc_aes.py 16133.00 -> 16452.74 : +319.74 = +1.982% (+/-1.50%) misc_mandel.py 128146.69 -> 130796.43 : +2649.74 = +2.068% (+/-3.18%) misc_pystone.py 83811.49 -> 83124.85 : -686.64 = -0.819% (+/-1.03%) misc_raytrace.py 21688.02 -> 21385.10 : -302.92 = -1.397% (+/-3.20%) The code size change is (firmware with a lot of frozen code benefits the most): bare-arm: +396 +0.697% minimal x86: +1595 +0.979% [incl +32(data)] unix x64: +2408 +0.470% [incl +800(data)] unix nanbox: +1396 +0.309% [incl -96(data)] stm32: -1256 -0.318% PYBV10 cc3200: +288 +0.157% esp8266: -260 -0.037% GENERIC esp32: -216 -0.014% GENERIC[incl -1072(data)] nrf: +116 +0.067% pca10040 rp2: -664 -0.135% PICO samd: +844 +0.607% ADAFRUIT_ITSYBITSY_M4_EXPRESS As part of this change the .mpy file format version is bumped to version 6. And mpy-tool.py has been improved to provide a good visualisation of the contents of .mpy files. In summary: this commit changes the bytecode to use qstr indirection, and reworks the .mpy file format to be simpler and allow .mpy files to be executed in-place. Performance is not impacted too much. Eventually it will be possible to store such .mpy files in a linear, read-only, memory- mappable filesystem so they can be executed from flash/ROM. This will essentially be able to replace frozen code for most applications. Signed-off-by: Damien George <damien@micropython.org>
2021-10-22 12:22:47 +01:00
for (const byte *ci = code_info; ci < line_info_top;) {
py: Improve encoding scheme for line-number to bytecode map. Reduces by about a factor of 10 on average the amount of RAM needed to store the line-number to bytecode map in the bytecode prelude. Using CPython3.4's stdlib for statistics: previously, an average of 13 bytes were used per (bytecode offset, line-number offset) pair, and now with this improvement, that's down to 1.3 bytes on average. Large RAM usage before was due to some very large steps in line numbers, both from the start of the first line in a function way down in the file, and also functions that have big comments and/or big strings in them (both cases were significant). Although the savings are large on average for the CPython stdlib, it won't have such a big effect for small scripts used in embedded programming. Addresses issue #648.
2014-07-31 17:12:01 +01:00
if ((ci[0] & 0x80) == 0) {
// 0b0LLBBBBB encoding
bc += ci[0] & 0x1f;
source_line += ci[0] >> 5;
ci += 1;
} else {
// 0b1LLLBBBB 0bLLLLLLLL encoding (l's LSB in second byte)
bc += ci[0] & 0xf;
source_line += ((ci[0] << 4) & 0x700) | ci[1];
ci += 2;
}
py/showbc: Pass in an mp_print_t struct to all bytecode-print functions. So the output can be redirected if needed. Signed-off-by: Damien George <damien@micropython.org>
2020-09-11 08:22:28 +01:00
mp_printf(print, " bc=" INT_FMT " line=" UINT_FMT "\n", bc, source_line);
py: Fix up source-line calculation. Should address issue #475.
2014-04-13 14:51:56 +01:00
}
}
py: Rework bytecode and .mpy file format to be mostly static data. Background: .mpy files are precompiled .py files, built using mpy-cross, that contain compiled bytecode functions (and can also contain machine code). The benefit of using an .mpy file over a .py file is that they are faster to import and take less memory when importing. They are also smaller on disk. But the real benefit of .mpy files comes when they are frozen into the firmware. This is done by loading the .mpy file during compilation of the firmware and turning it into a set of big C data structures (the job of mpy-tool.py), which are then compiled and downloaded into the ROM of a device. These C data structures can be executed in-place, ie directly from ROM. This makes importing even faster because there is very little to do, and also means such frozen modules take up much less RAM (because their bytecode stays in ROM). The downside of frozen code is that it requires recompiling and reflashing the entire firmware. This can be a big barrier to entry, slows down development time, and makes it harder to do OTA updates of frozen code (because the whole firmware must be updated). This commit attempts to solve this problem by providing a solution that sits between loading .mpy files into RAM and freezing them into the firmware. The .mpy file format has been reworked so that it consists of data and bytecode which is mostly static and ready to run in-place. If these new .mpy files are located in flash/ROM which is memory addressable, the .mpy file can be executed (mostly) in-place. With this approach there is still a small amount of unpacking and linking of the .mpy file that needs to be done when it's imported, but it's still much better than loading an .mpy from disk into RAM (although not as good as freezing .mpy files into the firmware). The main trick to make static .mpy files is to adjust the bytecode so any qstrs that it references now go through a lookup table to convert from local qstr number in the module to global qstr number in the firmware. That means the bytecode does not need linking/rewriting of qstrs when it's loaded. Instead only a small qstr table needs to be built (and put in RAM) at import time. This means the bytecode itself is static/constant and can be used directly if it's in addressable memory. Also the qstr string data in the .mpy file, and some constant object data, can be used directly. Note that the qstr table is global to the module (ie not per function). In more detail, in the VM what used to be (schematically): qst = DECODE_QSTR_VALUE; is now (schematically): idx = DECODE_QSTR_INDEX; qst = qstr_table[idx]; That allows the bytecode to be fixed at compile time and not need relinking/rewriting of the qstr values. Only qstr_table needs to be linked when the .mpy is loaded. Incidentally, this helps to reduce the size of bytecode because what used to be 2-byte qstr values in the bytecode are now (mostly) 1-byte indices. If the module uses the same qstr more than two times then the bytecode is smaller than before. The following changes are measured for this commit compared to the previous (the baseline): - average 7%-9% reduction in size of .mpy files - frozen code size is reduced by about 5%-7% - importing .py files uses about 5% less RAM in total - importing .mpy files uses about 4% less RAM in total - importing .py and .mpy files takes about the same time as before The qstr indirection in the bytecode has only a small impact on VM performance. For stm32 on PYBv1.0 the performance change of this commit is: diff of scores (higher is better) N=100 M=100 baseline -> this-commit diff diff% (error%) bm_chaos.py 371.07 -> 357.39 : -13.68 = -3.687% (+/-0.02%) bm_fannkuch.py 78.72 -> 77.49 : -1.23 = -1.563% (+/-0.01%) bm_fft.py 2591.73 -> 2539.28 : -52.45 = -2.024% (+/-0.00%) bm_float.py 6034.93 -> 5908.30 : -126.63 = -2.098% (+/-0.01%) bm_hexiom.py 48.96 -> 47.93 : -1.03 = -2.104% (+/-0.00%) bm_nqueens.py 4510.63 -> 4459.94 : -50.69 = -1.124% (+/-0.00%) bm_pidigits.py 650.28 -> 644.96 : -5.32 = -0.818% (+/-0.23%) core_import_mpy_multi.py 564.77 -> 581.49 : +16.72 = +2.960% (+/-0.01%) core_import_mpy_single.py 68.67 -> 67.16 : -1.51 = -2.199% (+/-0.01%) core_qstr.py 64.16 -> 64.12 : -0.04 = -0.062% (+/-0.00%) core_yield_from.py 362.58 -> 354.50 : -8.08 = -2.228% (+/-0.00%) misc_aes.py 429.69 -> 405.59 : -24.10 = -5.609% (+/-0.01%) misc_mandel.py 3485.13 -> 3416.51 : -68.62 = -1.969% (+/-0.00%) misc_pystone.py 2496.53 -> 2405.56 : -90.97 = -3.644% (+/-0.01%) misc_raytrace.py 381.47 -> 374.01 : -7.46 = -1.956% (+/-0.01%) viper_call0.py 576.73 -> 572.49 : -4.24 = -0.735% (+/-0.04%) viper_call1a.py 550.37 -> 546.21 : -4.16 = -0.756% (+/-0.09%) viper_call1b.py 438.23 -> 435.68 : -2.55 = -0.582% (+/-0.06%) viper_call1c.py 442.84 -> 440.04 : -2.80 = -0.632% (+/-0.08%) viper_call2a.py 536.31 -> 532.35 : -3.96 = -0.738% (+/-0.06%) viper_call2b.py 382.34 -> 377.07 : -5.27 = -1.378% (+/-0.03%) And for unix on x64: diff of scores (higher is better) N=2000 M=2000 baseline -> this-commit diff diff% (error%) bm_chaos.py 13594.20 -> 13073.84 : -520.36 = -3.828% (+/-5.44%) bm_fannkuch.py 60.63 -> 59.58 : -1.05 = -1.732% (+/-3.01%) bm_fft.py 112009.15 -> 111603.32 : -405.83 = -0.362% (+/-4.03%) bm_float.py 246202.55 -> 247923.81 : +1721.26 = +0.699% (+/-2.79%) bm_hexiom.py 615.65 -> 617.21 : +1.56 = +0.253% (+/-1.64%) bm_nqueens.py 215807.95 -> 215600.96 : -206.99 = -0.096% (+/-3.52%) bm_pidigits.py 8246.74 -> 8422.82 : +176.08 = +2.135% (+/-3.64%) misc_aes.py 16133.00 -> 16452.74 : +319.74 = +1.982% (+/-1.50%) misc_mandel.py 128146.69 -> 130796.43 : +2649.74 = +2.068% (+/-3.18%) misc_pystone.py 83811.49 -> 83124.85 : -686.64 = -0.819% (+/-1.03%) misc_raytrace.py 21688.02 -> 21385.10 : -302.92 = -1.397% (+/-3.20%) The code size change is (firmware with a lot of frozen code benefits the most): bare-arm: +396 +0.697% minimal x86: +1595 +0.979% [incl +32(data)] unix x64: +2408 +0.470% [incl +800(data)] unix nanbox: +1396 +0.309% [incl -96(data)] stm32: -1256 -0.318% PYBV10 cc3200: +288 +0.157% esp8266: -260 -0.037% GENERIC esp32: -216 -0.014% GENERIC[incl -1072(data)] nrf: +116 +0.067% pca10040 rp2: -664 -0.135% PICO samd: +844 +0.607% ADAFRUIT_ITSYBITSY_M4_EXPRESS As part of this change the .mpy file format version is bumped to version 6. And mpy-tool.py has been improved to provide a good visualisation of the contents of .mpy files. In summary: this commit changes the bytecode to use qstr indirection, and reworks the .mpy file format to be simpler and allow .mpy files to be executed in-place. Performance is not impacted too much. Eventually it will be possible to store such .mpy files in a linear, read-only, memory- mappable filesystem so they can be executed from flash/ROM. This will essentially be able to replace frozen code for most applications. Signed-off-by: Damien George <damien@micropython.org>
2021-10-22 12:22:47 +01:00
mp_bytecode_print2(print, ip, len - prelude_size, cm);
vm: Add rudimentary bytecode execution tracing capability.
2014-04-23 01:40:24 +01:00
}
py: Fix up source-line calculation. Should address issue #475.
2014-04-13 14:51:56 +01:00
py/showbc: Pass in an mp_print_t struct to all bytecode-print functions. So the output can be redirected if needed. Signed-off-by: Damien George <damien@micropython.org>
2020-09-11 08:22:28 +01:00
const byte *mp_bytecode_print_str(const mp_print_t *print, const byte *ip) {
Rename machine_(u)int_t to mp_(u)int_t. See discussion in issue #50.
2014-07-03 13:25:24 +01:00
mp_uint_t unum;
py, unix, stmhal: Allow to compile with -Wshadow. See issue #699.
2015-01-20 11:55:10 +00:00
qstr qst;
showbc: Refactor to allow inline instruction printing.
2014-12-27 03:00:08 +00:00
switch (*ip++) {
case MP_BC_LOAD_CONST_FALSE:
py/showbc: Pass in an mp_print_t struct to all bytecode-print functions. So the output can be redirected if needed. Signed-off-by: Damien George <damien@micropython.org>
2020-09-11 08:22:28 +01:00
mp_printf(print, "LOAD_CONST_FALSE");
showbc: Refactor to allow inline instruction printing.
2014-12-27 03:00:08 +00:00
break;
case MP_BC_LOAD_CONST_NONE:
py/showbc: Pass in an mp_print_t struct to all bytecode-print functions. So the output can be redirected if needed. Signed-off-by: Damien George <damien@micropython.org>
2020-09-11 08:22:28 +01:00
mp_printf(print, "LOAD_CONST_NONE");
showbc: Refactor to allow inline instruction printing.
2014-12-27 03:00:08 +00:00
break;
case MP_BC_LOAD_CONST_TRUE:
py/showbc: Pass in an mp_print_t struct to all bytecode-print functions. So the output can be redirected if needed. Signed-off-by: Damien George <damien@micropython.org>
2020-09-11 08:22:28 +01:00
mp_printf(print, "LOAD_CONST_TRUE");
showbc: Refactor to allow inline instruction printing.
2014-12-27 03:00:08 +00:00
break;
case MP_BC_LOAD_CONST_SMALL_INT: {
mp_int_t num = 0;
if ((ip[0] & 0x40) != 0) {
// Number is negative
num--;
Bytecode int varlen encoding: support arbitrary values for signed ints too.
2014-02-19 13:47:59 +00:00
}
showbc: Refactor to allow inline instruction printing.
2014-12-27 03:00:08 +00:00
do {
all: Fix signed shifts and NULL access errors from -fsanitize=undefined. Fixes the following (the line numbers match commit 0e87459e2bfd07): ../../extmod/crypto-algorithms/sha256.c:49:19: runtime error: left shif... ../../extmod/moduasyncio.c:106:35: runtime error: member access within ... ../../py/binary.c:210:13: runtime error: left shift of negative value -... ../../py/mpz.c:744:16: runtime error: negation of -9223372036854775808 ... ../../py/objint.c:109:22: runtime error: left shift of 1 by 31 places c... ../../py/objint_mpz.c:374:9: runtime error: left shift of 4611686018427... ../../py/objint_mpz.c:374:9: runtime error: left shift of negative valu... ../../py/parsenum.c:106:14: runtime error: left shift of 46116860184273... ../../py/runtime.c:395:33: runtime error: left shift of negative value ... ../../py/showbc.c:177:28: runtime error: left shift of negative value -... ../../py/vm.c:321:36: runtime error: left shift of negative value -1``` Testing was done on an amd64 Debian Buster system using gcc-8.3 and these settings: CFLAGS += -g3 -Og -fsanitize=undefined LDFLAGS += -fsanitize=undefined The introduced TASK_PAIRHEAP macro's conditional (x ? &x->i : NULL) assembles (under amd64 gcc 8.3 -Os) to the same as &x->i, since i is the initial field of the struct. However, for the purposes of undefined behavior analysis the conditional is needed. Signed-off-by: Jeff Epler <jepler@gmail.com>
2021-06-08 13:45:56 +01:00
num = ((mp_uint_t)num << 7) | (*ip & 0x7f);
showbc: Refactor to allow inline instruction printing.
2014-12-27 03:00:08 +00:00
} while ((*ip++ & 0x80) != 0);
py/showbc: Pass in an mp_print_t struct to all bytecode-print functions. So the output can be redirected if needed. Signed-off-by: Damien George <damien@micropython.org>
2020-09-11 08:22:28 +01:00
mp_printf(print, "LOAD_CONST_SMALL_INT " INT_FMT, num);
showbc: Refactor to allow inline instruction printing.
2014-12-27 03:00:08 +00:00
break;
}
Add function to decode and show byte code.
2013-11-17 13:19:33 +00:00
showbc: Refactor to allow inline instruction printing.
2014-12-27 03:00:08 +00:00
case MP_BC_LOAD_CONST_STRING:
DECODE_QSTR;
py/showbc: Pass in an mp_print_t struct to all bytecode-print functions. So the output can be redirected if needed. Signed-off-by: Damien George <damien@micropython.org>
2020-09-11 08:22:28 +01:00
mp_printf(print, "LOAD_CONST_STRING '%s'", qstr_str(qst));
showbc: Refactor to allow inline instruction printing.
2014-12-27 03:00:08 +00:00
break;
py/showbc.c: Handle new LOAD_CONST_OBJ opcode, and opcodes with cache.
2015-01-13 23:08:47 +00:00
case MP_BC_LOAD_CONST_OBJ:
py: Wrap all obj-ptr conversions in MP_OBJ_TO_PTR/MP_OBJ_FROM_PTR. This allows the mp_obj_t type to be configured to something other than a pointer-sized primitive type. This patch also includes additional changes to allow the code to compile when sizeof(mp_uint_t) != sizeof(void*), such as using size_t instead of mp_uint_t, and various casts.
2015-11-27 17:01:44 +00:00
DECODE_OBJ;
py/showbc: Pass in an mp_print_t struct to all bytecode-print functions. So the output can be redirected if needed. Signed-off-by: Damien George <damien@micropython.org>
2020-09-11 08:22:28 +01:00
mp_printf(print, "LOAD_CONST_OBJ %p=", MP_OBJ_TO_PTR(unum));
mp_obj_print_helper(print, (mp_obj_t)unum, PRINT_REPR);
py/showbc.c: Handle new LOAD_CONST_OBJ opcode, and opcodes with cache.
2015-01-13 23:08:47 +00:00
break;
showbc: Refactor to allow inline instruction printing.
2014-12-27 03:00:08 +00:00
case MP_BC_LOAD_NULL:
py/showbc: Pass in an mp_print_t struct to all bytecode-print functions. So the output can be redirected if needed. Signed-off-by: Damien George <damien@micropython.org>
2020-09-11 08:22:28 +01:00
mp_printf(print, "LOAD_NULL");
showbc: Refactor to allow inline instruction printing.
2014-12-27 03:00:08 +00:00
break;
case MP_BC_LOAD_FAST_N:
DECODE_UINT;
py/showbc: Pass in an mp_print_t struct to all bytecode-print functions. So the output can be redirected if needed. Signed-off-by: Damien George <damien@micropython.org>
2020-09-11 08:22:28 +01:00
mp_printf(print, "LOAD_FAST_N " UINT_FMT, unum);
showbc: Refactor to allow inline instruction printing.
2014-12-27 03:00:08 +00:00
break;
case MP_BC_LOAD_DEREF:
DECODE_UINT;
py/showbc: Pass in an mp_print_t struct to all bytecode-print functions. So the output can be redirected if needed. Signed-off-by: Damien George <damien@micropython.org>
2020-09-11 08:22:28 +01:00
mp_printf(print, "LOAD_DEREF " UINT_FMT, unum);
showbc: Refactor to allow inline instruction printing.
2014-12-27 03:00:08 +00:00
break;
case MP_BC_LOAD_NAME:
DECODE_QSTR;
py/showbc: Pass in an mp_print_t struct to all bytecode-print functions. So the output can be redirected if needed. Signed-off-by: Damien George <damien@micropython.org>
2020-09-11 08:22:28 +01:00
mp_printf(print, "LOAD_NAME %s", qstr_str(qst));
showbc: Refactor to allow inline instruction printing.
2014-12-27 03:00:08 +00:00
break;
case MP_BC_LOAD_GLOBAL:
DECODE_QSTR;
py/showbc: Pass in an mp_print_t struct to all bytecode-print functions. So the output can be redirected if needed. Signed-off-by: Damien George <damien@micropython.org>
2020-09-11 08:22:28 +01:00
mp_printf(print, "LOAD_GLOBAL %s", qstr_str(qst));
showbc: Refactor to allow inline instruction printing.
2014-12-27 03:00:08 +00:00
break;
case MP_BC_LOAD_ATTR:
DECODE_QSTR;
py/showbc: Pass in an mp_print_t struct to all bytecode-print functions. So the output can be redirected if needed. Signed-off-by: Damien George <damien@micropython.org>
2020-09-11 08:22:28 +01:00
mp_printf(print, "LOAD_ATTR %s", qstr_str(qst));
showbc: Refactor to allow inline instruction printing.
2014-12-27 03:00:08 +00:00
break;
case MP_BC_LOAD_METHOD:
DECODE_QSTR;
py/showbc: Pass in an mp_print_t struct to all bytecode-print functions. So the output can be redirected if needed. Signed-off-by: Damien George <damien@micropython.org>
2020-09-11 08:22:28 +01:00
mp_printf(print, "LOAD_METHOD %s", qstr_str(qst));
showbc: Refactor to allow inline instruction printing.
2014-12-27 03:00:08 +00:00
break;
py: Add LOAD_SUPER_METHOD bytecode to allow heap-free super meth calls. This patch allows the following code to run without allocating on the heap: super().foo(...) Before this patch such a call would allocate a super object on the heap and then load the foo method and call it right away. The super object is only needed to perform the lookup of the method and not needed after that. This patch makes an optimisation to allocate the super object on the C stack and discard it right after use. Changes in code size due to this patch are: bare-arm: +128 minimal: +232 unix x64: +416 unix nanbox: +364 stmhal: +184 esp8266: +340 cc3200: +128
2017-04-19 00:45:59 +01:00
case MP_BC_LOAD_SUPER_METHOD:
DECODE_QSTR;
py/showbc: Pass in an mp_print_t struct to all bytecode-print functions. So the output can be redirected if needed. Signed-off-by: Damien George <damien@micropython.org>
2020-09-11 08:22:28 +01:00
mp_printf(print, "LOAD_SUPER_METHOD %s", qstr_str(qst));
py: Add LOAD_SUPER_METHOD bytecode to allow heap-free super meth calls. This patch allows the following code to run without allocating on the heap: super().foo(...) Before this patch such a call would allocate a super object on the heap and then load the foo method and call it right away. The super object is only needed to perform the lookup of the method and not needed after that. This patch makes an optimisation to allocate the super object on the C stack and discard it right after use. Changes in code size due to this patch are: bare-arm: +128 minimal: +232 unix x64: +416 unix nanbox: +364 stmhal: +184 esp8266: +340 cc3200: +128
2017-04-19 00:45:59 +01:00
break;
showbc: Refactor to allow inline instruction printing.
2014-12-27 03:00:08 +00:00
case MP_BC_LOAD_BUILD_CLASS:
py/showbc: Pass in an mp_print_t struct to all bytecode-print functions. So the output can be redirected if needed. Signed-off-by: Damien George <damien@micropython.org>
2020-09-11 08:22:28 +01:00
mp_printf(print, "LOAD_BUILD_CLASS");
showbc: Refactor to allow inline instruction printing.
2014-12-27 03:00:08 +00:00
break;
case MP_BC_LOAD_SUBSCR:
py/showbc: Pass in an mp_print_t struct to all bytecode-print functions. So the output can be redirected if needed. Signed-off-by: Damien George <damien@micropython.org>
2020-09-11 08:22:28 +01:00
mp_printf(print, "LOAD_SUBSCR");
showbc: Refactor to allow inline instruction printing.
2014-12-27 03:00:08 +00:00
break;
case MP_BC_STORE_FAST_N:
DECODE_UINT;
py/showbc: Pass in an mp_print_t struct to all bytecode-print functions. So the output can be redirected if needed. Signed-off-by: Damien George <damien@micropython.org>
2020-09-11 08:22:28 +01:00
mp_printf(print, "STORE_FAST_N " UINT_FMT, unum);
showbc: Refactor to allow inline instruction printing.
2014-12-27 03:00:08 +00:00
break;
case MP_BC_STORE_DEREF:
DECODE_UINT;
py/showbc: Pass in an mp_print_t struct to all bytecode-print functions. So the output can be redirected if needed. Signed-off-by: Damien George <damien@micropython.org>
2020-09-11 08:22:28 +01:00
mp_printf(print, "STORE_DEREF " UINT_FMT, unum);
showbc: Refactor to allow inline instruction printing.
2014-12-27 03:00:08 +00:00
break;
case MP_BC_STORE_NAME:
DECODE_QSTR;
py/showbc: Pass in an mp_print_t struct to all bytecode-print functions. So the output can be redirected if needed. Signed-off-by: Damien George <damien@micropython.org>
2020-09-11 08:22:28 +01:00
mp_printf(print, "STORE_NAME %s", qstr_str(qst));
showbc: Refactor to allow inline instruction printing.
2014-12-27 03:00:08 +00:00
break;
case MP_BC_STORE_GLOBAL:
DECODE_QSTR;
py/showbc: Pass in an mp_print_t struct to all bytecode-print functions. So the output can be redirected if needed. Signed-off-by: Damien George <damien@micropython.org>
2020-09-11 08:22:28 +01:00
mp_printf(print, "STORE_GLOBAL %s", qstr_str(qst));
showbc: Refactor to allow inline instruction printing.
2014-12-27 03:00:08 +00:00
break;
case MP_BC_STORE_ATTR:
DECODE_QSTR;
py/showbc: Pass in an mp_print_t struct to all bytecode-print functions. So the output can be redirected if needed. Signed-off-by: Damien George <damien@micropython.org>
2020-09-11 08:22:28 +01:00
mp_printf(print, "STORE_ATTR %s", qstr_str(qst));
showbc: Refactor to allow inline instruction printing.
2014-12-27 03:00:08 +00:00
break;
case MP_BC_STORE_SUBSCR:
py/showbc: Pass in an mp_print_t struct to all bytecode-print functions. So the output can be redirected if needed. Signed-off-by: Damien George <damien@micropython.org>
2020-09-11 08:22:28 +01:00
mp_printf(print, "STORE_SUBSCR");
showbc: Refactor to allow inline instruction printing.
2014-12-27 03:00:08 +00:00
break;
case MP_BC_DELETE_FAST:
DECODE_UINT;
py/showbc: Pass in an mp_print_t struct to all bytecode-print functions. So the output can be redirected if needed. Signed-off-by: Damien George <damien@micropython.org>
2020-09-11 08:22:28 +01:00
mp_printf(print, "DELETE_FAST " UINT_FMT, unum);
showbc: Refactor to allow inline instruction printing.
2014-12-27 03:00:08 +00:00
break;
case MP_BC_DELETE_DEREF:
DECODE_UINT;
py/showbc: Pass in an mp_print_t struct to all bytecode-print functions. So the output can be redirected if needed. Signed-off-by: Damien George <damien@micropython.org>
2020-09-11 08:22:28 +01:00
mp_printf(print, "DELETE_DEREF " UINT_FMT, unum);
showbc: Refactor to allow inline instruction printing.
2014-12-27 03:00:08 +00:00
break;
case MP_BC_DELETE_NAME:
DECODE_QSTR;
py/showbc: Pass in an mp_print_t struct to all bytecode-print functions. So the output can be redirected if needed. Signed-off-by: Damien George <damien@micropython.org>
2020-09-11 08:22:28 +01:00
mp_printf(print, "DELETE_NAME %s", qstr_str(qst));
showbc: Refactor to allow inline instruction printing.
2014-12-27 03:00:08 +00:00
break;
py: Implement DELETE_GLOBAL in showbc.c.
2015-03-20 17:12:09 +00:00
case MP_BC_DELETE_GLOBAL:
DECODE_QSTR;
py/showbc: Pass in an mp_print_t struct to all bytecode-print functions. So the output can be redirected if needed. Signed-off-by: Damien George <damien@micropython.org>
2020-09-11 08:22:28 +01:00
mp_printf(print, "DELETE_GLOBAL %s", qstr_str(qst));
py: Implement DELETE_GLOBAL in showbc.c.
2015-03-20 17:12:09 +00:00
break;
showbc: Refactor to allow inline instruction printing.
2014-12-27 03:00:08 +00:00
case MP_BC_DUP_TOP:
py/showbc: Pass in an mp_print_t struct to all bytecode-print functions. So the output can be redirected if needed. Signed-off-by: Damien George <damien@micropython.org>
2020-09-11 08:22:28 +01:00
mp_printf(print, "DUP_TOP");
showbc: Refactor to allow inline instruction printing.
2014-12-27 03:00:08 +00:00
break;
case MP_BC_DUP_TOP_TWO:
py/showbc: Pass in an mp_print_t struct to all bytecode-print functions. So the output can be redirected if needed. Signed-off-by: Damien George <damien@micropython.org>
2020-09-11 08:22:28 +01:00
mp_printf(print, "DUP_TOP_TWO");
showbc: Refactor to allow inline instruction printing.
2014-12-27 03:00:08 +00:00
break;
case MP_BC_POP_TOP:
py/showbc: Pass in an mp_print_t struct to all bytecode-print functions. So the output can be redirected if needed. Signed-off-by: Damien George <damien@micropython.org>
2020-09-11 08:22:28 +01:00
mp_printf(print, "POP_TOP");
showbc: Refactor to allow inline instruction printing.
2014-12-27 03:00:08 +00:00
break;
case MP_BC_ROT_TWO:
py/showbc: Pass in an mp_print_t struct to all bytecode-print functions. So the output can be redirected if needed. Signed-off-by: Damien George <damien@micropython.org>
2020-09-11 08:22:28 +01:00
mp_printf(print, "ROT_TWO");
showbc: Refactor to allow inline instruction printing.
2014-12-27 03:00:08 +00:00
break;
case MP_BC_ROT_THREE:
py/showbc: Pass in an mp_print_t struct to all bytecode-print functions. So the output can be redirected if needed. Signed-off-by: Damien George <damien@micropython.org>
2020-09-11 08:22:28 +01:00
mp_printf(print, "ROT_THREE");
showbc: Refactor to allow inline instruction printing.
2014-12-27 03:00:08 +00:00
break;
case MP_BC_JUMP:
DECODE_SLABEL;
py/showbc: Pass in an mp_print_t struct to all bytecode-print functions. So the output can be redirected if needed. Signed-off-by: Damien George <damien@micropython.org>
2020-09-11 08:22:28 +01:00
mp_printf(print, "JUMP " UINT_FMT, (mp_uint_t)(ip + unum - mp_showbc_code_start));
showbc: Refactor to allow inline instruction printing.
2014-12-27 03:00:08 +00:00
break;
case MP_BC_POP_JUMP_IF_TRUE:
DECODE_SLABEL;
py/showbc: Pass in an mp_print_t struct to all bytecode-print functions. So the output can be redirected if needed. Signed-off-by: Damien George <damien@micropython.org>
2020-09-11 08:22:28 +01:00
mp_printf(print, "POP_JUMP_IF_TRUE " UINT_FMT, (mp_uint_t)(ip + unum - mp_showbc_code_start));
showbc: Refactor to allow inline instruction printing.
2014-12-27 03:00:08 +00:00
break;
case MP_BC_POP_JUMP_IF_FALSE:
DECODE_SLABEL;
py/showbc: Pass in an mp_print_t struct to all bytecode-print functions. So the output can be redirected if needed. Signed-off-by: Damien George <damien@micropython.org>
2020-09-11 08:22:28 +01:00
mp_printf(print, "POP_JUMP_IF_FALSE " UINT_FMT, (mp_uint_t)(ip + unum - mp_showbc_code_start));
showbc: Refactor to allow inline instruction printing.
2014-12-27 03:00:08 +00:00
break;
case MP_BC_JUMP_IF_TRUE_OR_POP:
DECODE_SLABEL;
py/showbc: Pass in an mp_print_t struct to all bytecode-print functions. So the output can be redirected if needed. Signed-off-by: Damien George <damien@micropython.org>
2020-09-11 08:22:28 +01:00
mp_printf(print, "JUMP_IF_TRUE_OR_POP " UINT_FMT, (mp_uint_t)(ip + unum - mp_showbc_code_start));
showbc: Refactor to allow inline instruction printing.
2014-12-27 03:00:08 +00:00
break;
case MP_BC_JUMP_IF_FALSE_OR_POP:
DECODE_SLABEL;
py/showbc: Pass in an mp_print_t struct to all bytecode-print functions. So the output can be redirected if needed. Signed-off-by: Damien George <damien@micropython.org>
2020-09-11 08:22:28 +01:00
mp_printf(print, "JUMP_IF_FALSE_OR_POP " UINT_FMT, (mp_uint_t)(ip + unum - mp_showbc_code_start));
showbc: Refactor to allow inline instruction printing.
2014-12-27 03:00:08 +00:00
break;
case MP_BC_SETUP_WITH:
DECODE_ULABEL; // loop-like labels are always forward
py/showbc: Pass in an mp_print_t struct to all bytecode-print functions. So the output can be redirected if needed. Signed-off-by: Damien George <damien@micropython.org>
2020-09-11 08:22:28 +01:00
mp_printf(print, "SETUP_WITH " UINT_FMT, (mp_uint_t)(ip + unum - mp_showbc_code_start));
showbc: Refactor to allow inline instruction printing.
2014-12-27 03:00:08 +00:00
break;
case MP_BC_WITH_CLEANUP:
py/showbc: Pass in an mp_print_t struct to all bytecode-print functions. So the output can be redirected if needed. Signed-off-by: Damien George <damien@micropython.org>
2020-09-11 08:22:28 +01:00
mp_printf(print, "WITH_CLEANUP");
showbc: Refactor to allow inline instruction printing.
2014-12-27 03:00:08 +00:00
break;
case MP_BC_UNWIND_JUMP:
DECODE_SLABEL;
py/showbc: Pass in an mp_print_t struct to all bytecode-print functions. So the output can be redirected if needed. Signed-off-by: Damien George <damien@micropython.org>
2020-09-11 08:22:28 +01:00
mp_printf(print, "UNWIND_JUMP " UINT_FMT " %d", (mp_uint_t)(ip + unum - mp_showbc_code_start), *ip);
showbc: Refactor to allow inline instruction printing.
2014-12-27 03:00:08 +00:00
ip += 1;
break;
case MP_BC_SETUP_EXCEPT:
DECODE_ULABEL; // except labels are always forward
py/showbc: Pass in an mp_print_t struct to all bytecode-print functions. So the output can be redirected if needed. Signed-off-by: Damien George <damien@micropython.org>
2020-09-11 08:22:28 +01:00
mp_printf(print, "SETUP_EXCEPT " UINT_FMT, (mp_uint_t)(ip + unum - mp_showbc_code_start));
showbc: Refactor to allow inline instruction printing.
2014-12-27 03:00:08 +00:00
break;
case MP_BC_SETUP_FINALLY:
DECODE_ULABEL; // except labels are always forward
py/showbc: Pass in an mp_print_t struct to all bytecode-print functions. So the output can be redirected if needed. Signed-off-by: Damien George <damien@micropython.org>
2020-09-11 08:22:28 +01:00
mp_printf(print, "SETUP_FINALLY " UINT_FMT, (mp_uint_t)(ip + unum - mp_showbc_code_start));
showbc: Refactor to allow inline instruction printing.
2014-12-27 03:00:08 +00:00
break;
case MP_BC_END_FINALLY:
// if TOS is an exception, reraises the exception (3 values on TOS)
// if TOS is an integer, does something else
// if TOS is None, just pops it and continues
// else error
py/showbc: Pass in an mp_print_t struct to all bytecode-print functions. So the output can be redirected if needed. Signed-off-by: Damien George <damien@micropython.org>
2020-09-11 08:22:28 +01:00
mp_printf(print, "END_FINALLY");
showbc: Refactor to allow inline instruction printing.
2014-12-27 03:00:08 +00:00
break;
case MP_BC_GET_ITER:
py/showbc: Pass in an mp_print_t struct to all bytecode-print functions. So the output can be redirected if needed. Signed-off-by: Damien George <damien@micropython.org>
2020-09-11 08:22:28 +01:00
mp_printf(print, "GET_ITER");
showbc: Refactor to allow inline instruction printing.
2014-12-27 03:00:08 +00:00
break;
py: Allow bytecode/native to put iter_buf on stack for simple for loops. So that the "for x in it: ..." statement can now work without using the heap (so long as the iterator argument fits in an iter_buf structure).
2016-01-09 23:59:52 +00:00
case MP_BC_GET_ITER_STACK:
py/showbc: Pass in an mp_print_t struct to all bytecode-print functions. So the output can be redirected if needed. Signed-off-by: Damien George <damien@micropython.org>
2020-09-11 08:22:28 +01:00
mp_printf(print, "GET_ITER_STACK");
py: Allow bytecode/native to put iter_buf on stack for simple for loops. So that the "for x in it: ..." statement can now work without using the heap (so long as the iterator argument fits in an iter_buf structure).
2016-01-09 23:59:52 +00:00
break;
showbc: Refactor to allow inline instruction printing.
2014-12-27 03:00:08 +00:00
case MP_BC_FOR_ITER:
DECODE_ULABEL; // the jump offset if iteration finishes; for labels are always forward
py/showbc: Pass in an mp_print_t struct to all bytecode-print functions. So the output can be redirected if needed. Signed-off-by: Damien George <damien@micropython.org>
2020-09-11 08:22:28 +01:00
mp_printf(print, "FOR_ITER " UINT_FMT, (mp_uint_t)(ip + unum - mp_showbc_code_start));
showbc: Refactor to allow inline instruction printing.
2014-12-27 03:00:08 +00:00
break;
py: Replace POP_BLOCK and POP_EXCEPT opcodes with POP_EXCEPT_JUMP. POP_BLOCK and POP_EXCEPT are now the same, and are always followed by a JUMP. So this optimisation reduces code size, and RAM usage of bytecode by two bytes for each try-except handler.
2019-02-15 01:18:59 +00:00
case MP_BC_POP_EXCEPT_JUMP:
DECODE_ULABEL; // these labels are always forward
py/showbc: Pass in an mp_print_t struct to all bytecode-print functions. So the output can be redirected if needed. Signed-off-by: Damien George <damien@micropython.org>
2020-09-11 08:22:28 +01:00
mp_printf(print, "POP_EXCEPT_JUMP " UINT_FMT, (mp_uint_t)(ip + unum - mp_showbc_code_start));
showbc: Refactor to allow inline instruction printing.
2014-12-27 03:00:08 +00:00
break;
case MP_BC_BUILD_TUPLE:
DECODE_UINT;
py/showbc: Pass in an mp_print_t struct to all bytecode-print functions. So the output can be redirected if needed. Signed-off-by: Damien George <damien@micropython.org>
2020-09-11 08:22:28 +01:00
mp_printf(print, "BUILD_TUPLE " UINT_FMT, unum);
showbc: Refactor to allow inline instruction printing.
2014-12-27 03:00:08 +00:00
break;
case MP_BC_BUILD_LIST:
DECODE_UINT;
py/showbc: Pass in an mp_print_t struct to all bytecode-print functions. So the output can be redirected if needed. Signed-off-by: Damien George <damien@micropython.org>
2020-09-11 08:22:28 +01:00
mp_printf(print, "BUILD_LIST " UINT_FMT, unum);
showbc: Refactor to allow inline instruction printing.
2014-12-27 03:00:08 +00:00
break;
case MP_BC_BUILD_MAP:
DECODE_UINT;
py/showbc: Pass in an mp_print_t struct to all bytecode-print functions. So the output can be redirected if needed. Signed-off-by: Damien George <damien@micropython.org>
2020-09-11 08:22:28 +01:00
mp_printf(print, "BUILD_MAP " UINT_FMT, unum);
showbc: Refactor to allow inline instruction printing.
2014-12-27 03:00:08 +00:00
break;
case MP_BC_STORE_MAP:
py/showbc: Pass in an mp_print_t struct to all bytecode-print functions. So the output can be redirected if needed. Signed-off-by: Damien George <damien@micropython.org>
2020-09-11 08:22:28 +01:00
mp_printf(print, "STORE_MAP");
showbc: Refactor to allow inline instruction printing.
2014-12-27 03:00:08 +00:00
break;
case MP_BC_BUILD_SET:
DECODE_UINT;
py/showbc: Pass in an mp_print_t struct to all bytecode-print functions. So the output can be redirected if needed. Signed-off-by: Damien George <damien@micropython.org>
2020-09-11 08:22:28 +01:00
mp_printf(print, "BUILD_SET " UINT_FMT, unum);
showbc: Refactor to allow inline instruction printing.
2014-12-27 03:00:08 +00:00
break;
Rename bultins config variables to MICROPY_PY_BUILTINS_*. This renames: MICROPY_PY_FROZENSET -> MICROPY_PY_BUILTINS_FROZENSET MICROPY_PY_PROPERTY -> MICROPY_PY_BUILTINS_PROPERTY MICROPY_PY_SLICE -> MICROPY_PY_BUILTINS_SLICE MICROPY_ENABLE_FLOAT -> MICROPY_PY_BUILTINS_FLOAT See issue #35 for discussion.
2014-06-01 13:32:54 +01:00
#if MICROPY_PY_BUILTINS_SLICE
showbc: Refactor to allow inline instruction printing.
2014-12-27 03:00:08 +00:00
case MP_BC_BUILD_SLICE:
DECODE_UINT;
py/showbc: Pass in an mp_print_t struct to all bytecode-print functions. So the output can be redirected if needed. Signed-off-by: Damien George <damien@micropython.org>
2020-09-11 08:22:28 +01:00
mp_printf(print, "BUILD_SLICE " UINT_FMT, unum);
showbc: Refactor to allow inline instruction printing.
2014-12-27 03:00:08 +00:00
break;
Make VM stack grow upwards, and so no reversed args arrays. Change state layout in VM so the stack starts at state[0] and grows upwards. Locals are at the top end of the state and number downwards. This cleans up a lot of the interface connecting the VM to C: now all functions that take an array of Micro Python objects are in order (ie no longer in reverse). Also clean up C API with keyword arguments (call_n and call_n_kw replaced with single call method that takes keyword arguments). And now make_new takes keyword arguments. emitnative.c has not yet been changed to comply with the new order of stack layout.
2014-01-18 14:10:48 +00:00
#endif
py: Combine 3 comprehension opcodes (list/dict/set) into 1. With the previous patch combining 3 emit functions into 1, it now makes sense to also combine the corresponding VM opcodes, which is what this patch does. This eliminates 2 opcodes which simplifies the VM and reduces code size, in bytes: bare-arm:44, minimal:64, unix(NDEBUG,x86-64):272, stmhal:92, esp8266:200. Profiling (with a simple script that creates many list/dict/set comprehensions) shows no measurable change in performance.
2016-09-18 23:46:01 +01:00
case MP_BC_STORE_COMP:
DECODE_UINT;
py/showbc: Pass in an mp_print_t struct to all bytecode-print functions. So the output can be redirected if needed. Signed-off-by: Damien George <damien@micropython.org>
2020-09-11 08:22:28 +01:00
mp_printf(print, "STORE_COMP " UINT_FMT, unum);
py: Combine 3 comprehension opcodes (list/dict/set) into 1. With the previous patch combining 3 emit functions into 1, it now makes sense to also combine the corresponding VM opcodes, which is what this patch does. This eliminates 2 opcodes which simplifies the VM and reduces code size, in bytes: bare-arm:44, minimal:64, unix(NDEBUG,x86-64):272, stmhal:92, esp8266:200. Profiling (with a simple script that creates many list/dict/set comprehensions) shows no measurable change in performance.
2016-09-18 23:46:01 +01:00
break;
showbc: Refactor to allow inline instruction printing.
2014-12-27 03:00:08 +00:00
case MP_BC_UNPACK_SEQUENCE:
DECODE_UINT;
py/showbc: Pass in an mp_print_t struct to all bytecode-print functions. So the output can be redirected if needed. Signed-off-by: Damien George <damien@micropython.org>
2020-09-11 08:22:28 +01:00
mp_printf(print, "UNPACK_SEQUENCE " UINT_FMT, unum);
showbc: Refactor to allow inline instruction printing.
2014-12-27 03:00:08 +00:00
break;
py: Make showbc decode UNPACK_EX, and use correct range for unop/binop.
2015-06-18 16:12:17 +01:00
case MP_BC_UNPACK_EX:
DECODE_UINT;
py/showbc: Pass in an mp_print_t struct to all bytecode-print functions. So the output can be redirected if needed. Signed-off-by: Damien George <damien@micropython.org>
2020-09-11 08:22:28 +01:00
mp_printf(print, "UNPACK_EX " UINT_FMT, unum);
py: Make showbc decode UNPACK_EX, and use correct range for unop/binop.
2015-06-18 16:12:17 +01:00
break;
showbc: Refactor to allow inline instruction printing.
2014-12-27 03:00:08 +00:00
case MP_BC_MAKE_FUNCTION:
DECODE_PTR;
py/showbc: Pass in an mp_print_t struct to all bytecode-print functions. So the output can be redirected if needed. Signed-off-by: Damien George <damien@micropython.org>
2020-09-11 08:22:28 +01:00
mp_printf(print, "MAKE_FUNCTION %p", (void *)(uintptr_t)unum);
showbc: Refactor to allow inline instruction printing.
2014-12-27 03:00:08 +00:00
break;
case MP_BC_MAKE_FUNCTION_DEFARGS:
DECODE_PTR;
py/showbc: Pass in an mp_print_t struct to all bytecode-print functions. So the output can be redirected if needed. Signed-off-by: Damien George <damien@micropython.org>
2020-09-11 08:22:28 +01:00
mp_printf(print, "MAKE_FUNCTION_DEFARGS %p", (void *)(uintptr_t)unum);
showbc: Refactor to allow inline instruction printing.
2014-12-27 03:00:08 +00:00
break;
case MP_BC_MAKE_CLOSURE: {
DECODE_PTR;
mp_uint_t n_closed_over = *ip++;
py/showbc: Pass in an mp_print_t struct to all bytecode-print functions. So the output can be redirected if needed. Signed-off-by: Damien George <damien@micropython.org>
2020-09-11 08:22:28 +01:00
mp_printf(print, "MAKE_CLOSURE %p " UINT_FMT, (void *)(uintptr_t)unum, n_closed_over);
showbc: Refactor to allow inline instruction printing.
2014-12-27 03:00:08 +00:00
break;
}
case MP_BC_MAKE_CLOSURE_DEFARGS: {
DECODE_PTR;
mp_uint_t n_closed_over = *ip++;
py/showbc: Pass in an mp_print_t struct to all bytecode-print functions. So the output can be redirected if needed. Signed-off-by: Damien George <damien@micropython.org>
2020-09-11 08:22:28 +01:00
mp_printf(print, "MAKE_CLOSURE_DEFARGS %p " UINT_FMT, (void *)(uintptr_t)unum, n_closed_over);
showbc: Refactor to allow inline instruction printing.
2014-12-27 03:00:08 +00:00
break;
}
py: make closures work.
2013-12-30 22:32:17 +00:00
showbc: Refactor to allow inline instruction printing.
2014-12-27 03:00:08 +00:00
case MP_BC_CALL_FUNCTION:
DECODE_UINT;
py/showbc: Pass in an mp_print_t struct to all bytecode-print functions. So the output can be redirected if needed. Signed-off-by: Damien George <damien@micropython.org>
2020-09-11 08:22:28 +01:00
mp_printf(print, "CALL_FUNCTION n=" UINT_FMT " nkw=" UINT_FMT, unum & 0xff, (unum >> 8) & 0xff);
showbc: Refactor to allow inline instruction printing.
2014-12-27 03:00:08 +00:00
break;
case MP_BC_CALL_FUNCTION_VAR_KW:
DECODE_UINT;
py/showbc: Pass in an mp_print_t struct to all bytecode-print functions. So the output can be redirected if needed. Signed-off-by: Damien George <damien@micropython.org>
2020-09-11 08:22:28 +01:00
mp_printf(print, "CALL_FUNCTION_VAR_KW n=" UINT_FMT " nkw=" UINT_FMT, unum & 0xff, (unum >> 8) & 0xff);
showbc: Refactor to allow inline instruction printing.
2014-12-27 03:00:08 +00:00
break;
case MP_BC_CALL_METHOD:
DECODE_UINT;
py/showbc: Pass in an mp_print_t struct to all bytecode-print functions. So the output can be redirected if needed. Signed-off-by: Damien George <damien@micropython.org>
2020-09-11 08:22:28 +01:00
mp_printf(print, "CALL_METHOD n=" UINT_FMT " nkw=" UINT_FMT, unum & 0xff, (unum >> 8) & 0xff);
showbc: Refactor to allow inline instruction printing.
2014-12-27 03:00:08 +00:00
break;
case MP_BC_CALL_METHOD_VAR_KW:
DECODE_UINT;
py/showbc: Pass in an mp_print_t struct to all bytecode-print functions. So the output can be redirected if needed. Signed-off-by: Damien George <damien@micropython.org>
2020-09-11 08:22:28 +01:00
mp_printf(print, "CALL_METHOD_VAR_KW n=" UINT_FMT " nkw=" UINT_FMT, unum & 0xff, (unum >> 8) & 0xff);
showbc: Refactor to allow inline instruction printing.
2014-12-27 03:00:08 +00:00
break;
case MP_BC_RETURN_VALUE:
py/showbc: Pass in an mp_print_t struct to all bytecode-print functions. So the output can be redirected if needed. Signed-off-by: Damien George <damien@micropython.org>
2020-09-11 08:22:28 +01:00
mp_printf(print, "RETURN_VALUE");
showbc: Refactor to allow inline instruction printing.
2014-12-27 03:00:08 +00:00
break;
py: Split RAISE_VARARGS opcode into 3 separate ones. From the beginning of this project the RAISE_VARARGS opcode was named and implemented following CPython, where it has an argument (to the opcode) counting how many args the raise takes: raise # 0 args (re-raise previous exception) raise exc # 1 arg raise exc from exc2 # 2 args (chained raise) In the bytecode this operation therefore takes 2 bytes, one for RAISE_VARARGS and one for the number of args. This patch splits this opcode into 3, where each is now a single byte. This reduces bytecode size by 1 byte for each use of raise. Every byte counts! It also has the benefit of reducing code size (on all ports except nanbox).
2019-08-22 03:39:07 +01:00
case MP_BC_RAISE_LAST:
py/showbc: Pass in an mp_print_t struct to all bytecode-print functions. So the output can be redirected if needed. Signed-off-by: Damien George <damien@micropython.org>
2020-09-11 08:22:28 +01:00
mp_printf(print, "RAISE_LAST");
py: Split RAISE_VARARGS opcode into 3 separate ones. From the beginning of this project the RAISE_VARARGS opcode was named and implemented following CPython, where it has an argument (to the opcode) counting how many args the raise takes: raise # 0 args (re-raise previous exception) raise exc # 1 arg raise exc from exc2 # 2 args (chained raise) In the bytecode this operation therefore takes 2 bytes, one for RAISE_VARARGS and one for the number of args. This patch splits this opcode into 3, where each is now a single byte. This reduces bytecode size by 1 byte for each use of raise. Every byte counts! It also has the benefit of reducing code size (on all ports except nanbox).
2019-08-22 03:39:07 +01:00
break;
case MP_BC_RAISE_OBJ:
py/showbc: Pass in an mp_print_t struct to all bytecode-print functions. So the output can be redirected if needed. Signed-off-by: Damien George <damien@micropython.org>
2020-09-11 08:22:28 +01:00
mp_printf(print, "RAISE_OBJ");
py: Split RAISE_VARARGS opcode into 3 separate ones. From the beginning of this project the RAISE_VARARGS opcode was named and implemented following CPython, where it has an argument (to the opcode) counting how many args the raise takes: raise # 0 args (re-raise previous exception) raise exc # 1 arg raise exc from exc2 # 2 args (chained raise) In the bytecode this operation therefore takes 2 bytes, one for RAISE_VARARGS and one for the number of args. This patch splits this opcode into 3, where each is now a single byte. This reduces bytecode size by 1 byte for each use of raise. Every byte counts! It also has the benefit of reducing code size (on all ports except nanbox).
2019-08-22 03:39:07 +01:00
break;
case MP_BC_RAISE_FROM:
py/showbc: Pass in an mp_print_t struct to all bytecode-print functions. So the output can be redirected if needed. Signed-off-by: Damien George <damien@micropython.org>
2020-09-11 08:22:28 +01:00
mp_printf(print, "RAISE_FROM");
showbc: Refactor to allow inline instruction printing.
2014-12-27 03:00:08 +00:00
break;
case MP_BC_YIELD_VALUE:
py/showbc: Pass in an mp_print_t struct to all bytecode-print functions. So the output can be redirected if needed. Signed-off-by: Damien George <damien@micropython.org>
2020-09-11 08:22:28 +01:00
mp_printf(print, "YIELD_VALUE");
showbc: Refactor to allow inline instruction printing.
2014-12-27 03:00:08 +00:00
break;
case MP_BC_YIELD_FROM:
py/showbc: Pass in an mp_print_t struct to all bytecode-print functions. So the output can be redirected if needed. Signed-off-by: Damien George <damien@micropython.org>
2020-09-11 08:22:28 +01:00
mp_printf(print, "YIELD_FROM");
showbc: Refactor to allow inline instruction printing.
2014-12-27 03:00:08 +00:00
break;
case MP_BC_IMPORT_NAME:
DECODE_QSTR;
py/showbc: Pass in an mp_print_t struct to all bytecode-print functions. So the output can be redirected if needed. Signed-off-by: Damien George <damien@micropython.org>
2020-09-11 08:22:28 +01:00
mp_printf(print, "IMPORT_NAME '%s'", qstr_str(qst));
showbc: Refactor to allow inline instruction printing.
2014-12-27 03:00:08 +00:00
break;
case MP_BC_IMPORT_FROM:
DECODE_QSTR;
py/showbc: Pass in an mp_print_t struct to all bytecode-print functions. So the output can be redirected if needed. Signed-off-by: Damien George <damien@micropython.org>
2020-09-11 08:22:28 +01:00
mp_printf(print, "IMPORT_FROM '%s'", qstr_str(qst));
showbc: Refactor to allow inline instruction printing.
2014-12-27 03:00:08 +00:00
break;
case MP_BC_IMPORT_STAR:
py/showbc: Pass in an mp_print_t struct to all bytecode-print functions. So the output can be redirected if needed. Signed-off-by: Damien George <damien@micropython.org>
2020-09-11 08:22:28 +01:00
mp_printf(print, "IMPORT_STAR");
showbc: Refactor to allow inline instruction printing.
2014-12-27 03:00:08 +00:00
break;
default:
if (ip[-1] < MP_BC_LOAD_CONST_SMALL_INT_MULTI + 64) {
py/showbc: Pass in an mp_print_t struct to all bytecode-print functions. So the output can be redirected if needed. Signed-off-by: Damien George <damien@micropython.org>
2020-09-11 08:22:28 +01:00
mp_printf(print, "LOAD_CONST_SMALL_INT " INT_FMT, (mp_int_t)ip[-1] - MP_BC_LOAD_CONST_SMALL_INT_MULTI - 16);
showbc: Refactor to allow inline instruction printing.
2014-12-27 03:00:08 +00:00
} else if (ip[-1] < MP_BC_LOAD_FAST_MULTI + 16) {
py/showbc: Pass in an mp_print_t struct to all bytecode-print functions. So the output can be redirected if needed. Signed-off-by: Damien George <damien@micropython.org>
2020-09-11 08:22:28 +01:00
mp_printf(print, "LOAD_FAST " UINT_FMT, (mp_uint_t)ip[-1] - MP_BC_LOAD_FAST_MULTI);
showbc: Refactor to allow inline instruction printing.
2014-12-27 03:00:08 +00:00
} else if (ip[-1] < MP_BC_STORE_FAST_MULTI + 16) {
py/showbc: Pass in an mp_print_t struct to all bytecode-print functions. So the output can be redirected if needed. Signed-off-by: Damien George <damien@micropython.org>
2020-09-11 08:22:28 +01:00
mp_printf(print, "STORE_FAST " UINT_FMT, (mp_uint_t)ip[-1] - MP_BC_STORE_FAST_MULTI);
py: Clean up unary and binary enum list to keep groups together. 2 non-bytecode binary ops (NOT_IN and IN_NOT) are moved out of the bytecode group, so this change will change the bytecode format.
2017-10-03 13:34:28 +01:00
} else if (ip[-1] < MP_BC_UNARY_OP_MULTI + MP_UNARY_OP_NUM_BYTECODE) {
py/showbc: Print unary-op string when dumping bytecode. Signed-off-by: Damien George <damien@micropython.org>
2021-11-19 06:05:40 +00:00
mp_uint_t op = ip[-1] - MP_BC_UNARY_OP_MULTI;
mp_printf(print, "UNARY_OP " UINT_FMT " %s", op, qstr_str(mp_unary_op_method_name[op]));
py: Clean up unary and binary enum list to keep groups together. 2 non-bytecode binary ops (NOT_IN and IN_NOT) are moved out of the bytecode group, so this change will change the bytecode format.
2017-10-03 13:34:28 +01:00
} else if (ip[-1] < MP_BC_BINARY_OP_MULTI + MP_BINARY_OP_NUM_BYTECODE) {
showbc: Print operation mnemonic in BINARY_OP.
2014-12-28 19:41:58 +00:00
mp_uint_t op = ip[-1] - MP_BC_BINARY_OP_MULTI;
py/showbc: Pass in an mp_print_t struct to all bytecode-print functions. So the output can be redirected if needed. Signed-off-by: Damien George <damien@micropython.org>
2020-09-11 08:22:28 +01:00
mp_printf(print, "BINARY_OP " UINT_FMT " %s", op, qstr_str(mp_binary_op_method_name[op]));
showbc: Refactor to allow inline instruction printing.
2014-12-27 03:00:08 +00:00
} else {
py/showbc: Pass in an mp_print_t struct to all bytecode-print functions. So the output can be redirected if needed. Signed-off-by: Damien George <damien@micropython.org>
2020-09-11 08:22:28 +01:00
mp_printf(print, "code %p, byte code 0x%02x not implemented\n", ip - 1, ip[-1]);
showbc: Refactor to allow inline instruction printing.
2014-12-27 03:00:08 +00:00
assert(0);
return ip;
showbc: MAKE_CLOSURE*: Update for new closed-over encoding.
2014-04-23 01:22:10 +01:00
}
showbc: Refactor to allow inline instruction printing.
2014-12-27 03:00:08 +00:00
break;
}
py: Support closures with default args.
2014-03-26 21:14:59 +00:00
showbc: Refactor to allow inline instruction printing.
2014-12-27 03:00:08 +00:00
return ip;
}
py: Rework bytecode and .mpy file format to be mostly static data. Background: .mpy files are precompiled .py files, built using mpy-cross, that contain compiled bytecode functions (and can also contain machine code). The benefit of using an .mpy file over a .py file is that they are faster to import and take less memory when importing. They are also smaller on disk. But the real benefit of .mpy files comes when they are frozen into the firmware. This is done by loading the .mpy file during compilation of the firmware and turning it into a set of big C data structures (the job of mpy-tool.py), which are then compiled and downloaded into the ROM of a device. These C data structures can be executed in-place, ie directly from ROM. This makes importing even faster because there is very little to do, and also means such frozen modules take up much less RAM (because their bytecode stays in ROM). The downside of frozen code is that it requires recompiling and reflashing the entire firmware. This can be a big barrier to entry, slows down development time, and makes it harder to do OTA updates of frozen code (because the whole firmware must be updated). This commit attempts to solve this problem by providing a solution that sits between loading .mpy files into RAM and freezing them into the firmware. The .mpy file format has been reworked so that it consists of data and bytecode which is mostly static and ready to run in-place. If these new .mpy files are located in flash/ROM which is memory addressable, the .mpy file can be executed (mostly) in-place. With this approach there is still a small amount of unpacking and linking of the .mpy file that needs to be done when it's imported, but it's still much better than loading an .mpy from disk into RAM (although not as good as freezing .mpy files into the firmware). The main trick to make static .mpy files is to adjust the bytecode so any qstrs that it references now go through a lookup table to convert from local qstr number in the module to global qstr number in the firmware. That means the bytecode does not need linking/rewriting of qstrs when it's loaded. Instead only a small qstr table needs to be built (and put in RAM) at import time. This means the bytecode itself is static/constant and can be used directly if it's in addressable memory. Also the qstr string data in the .mpy file, and some constant object data, can be used directly. Note that the qstr table is global to the module (ie not per function). In more detail, in the VM what used to be (schematically): qst = DECODE_QSTR_VALUE; is now (schematically): idx = DECODE_QSTR_INDEX; qst = qstr_table[idx]; That allows the bytecode to be fixed at compile time and not need relinking/rewriting of the qstr values. Only qstr_table needs to be linked when the .mpy is loaded. Incidentally, this helps to reduce the size of bytecode because what used to be 2-byte qstr values in the bytecode are now (mostly) 1-byte indices. If the module uses the same qstr more than two times then the bytecode is smaller than before. The following changes are measured for this commit compared to the previous (the baseline): - average 7%-9% reduction in size of .mpy files - frozen code size is reduced by about 5%-7% - importing .py files uses about 5% less RAM in total - importing .mpy files uses about 4% less RAM in total - importing .py and .mpy files takes about the same time as before The qstr indirection in the bytecode has only a small impact on VM performance. For stm32 on PYBv1.0 the performance change of this commit is: diff of scores (higher is better) N=100 M=100 baseline -> this-commit diff diff% (error%) bm_chaos.py 371.07 -> 357.39 : -13.68 = -3.687% (+/-0.02%) bm_fannkuch.py 78.72 -> 77.49 : -1.23 = -1.563% (+/-0.01%) bm_fft.py 2591.73 -> 2539.28 : -52.45 = -2.024% (+/-0.00%) bm_float.py 6034.93 -> 5908.30 : -126.63 = -2.098% (+/-0.01%) bm_hexiom.py 48.96 -> 47.93 : -1.03 = -2.104% (+/-0.00%) bm_nqueens.py 4510.63 -> 4459.94 : -50.69 = -1.124% (+/-0.00%) bm_pidigits.py 650.28 -> 644.96 : -5.32 = -0.818% (+/-0.23%) core_import_mpy_multi.py 564.77 -> 581.49 : +16.72 = +2.960% (+/-0.01%) core_import_mpy_single.py 68.67 -> 67.16 : -1.51 = -2.199% (+/-0.01%) core_qstr.py 64.16 -> 64.12 : -0.04 = -0.062% (+/-0.00%) core_yield_from.py 362.58 -> 354.50 : -8.08 = -2.228% (+/-0.00%) misc_aes.py 429.69 -> 405.59 : -24.10 = -5.609% (+/-0.01%) misc_mandel.py 3485.13 -> 3416.51 : -68.62 = -1.969% (+/-0.00%) misc_pystone.py 2496.53 -> 2405.56 : -90.97 = -3.644% (+/-0.01%) misc_raytrace.py 381.47 -> 374.01 : -7.46 = -1.956% (+/-0.01%) viper_call0.py 576.73 -> 572.49 : -4.24 = -0.735% (+/-0.04%) viper_call1a.py 550.37 -> 546.21 : -4.16 = -0.756% (+/-0.09%) viper_call1b.py 438.23 -> 435.68 : -2.55 = -0.582% (+/-0.06%) viper_call1c.py 442.84 -> 440.04 : -2.80 = -0.632% (+/-0.08%) viper_call2a.py 536.31 -> 532.35 : -3.96 = -0.738% (+/-0.06%) viper_call2b.py 382.34 -> 377.07 : -5.27 = -1.378% (+/-0.03%) And for unix on x64: diff of scores (higher is better) N=2000 M=2000 baseline -> this-commit diff diff% (error%) bm_chaos.py 13594.20 -> 13073.84 : -520.36 = -3.828% (+/-5.44%) bm_fannkuch.py 60.63 -> 59.58 : -1.05 = -1.732% (+/-3.01%) bm_fft.py 112009.15 -> 111603.32 : -405.83 = -0.362% (+/-4.03%) bm_float.py 246202.55 -> 247923.81 : +1721.26 = +0.699% (+/-2.79%) bm_hexiom.py 615.65 -> 617.21 : +1.56 = +0.253% (+/-1.64%) bm_nqueens.py 215807.95 -> 215600.96 : -206.99 = -0.096% (+/-3.52%) bm_pidigits.py 8246.74 -> 8422.82 : +176.08 = +2.135% (+/-3.64%) misc_aes.py 16133.00 -> 16452.74 : +319.74 = +1.982% (+/-1.50%) misc_mandel.py 128146.69 -> 130796.43 : +2649.74 = +2.068% (+/-3.18%) misc_pystone.py 83811.49 -> 83124.85 : -686.64 = -0.819% (+/-1.03%) misc_raytrace.py 21688.02 -> 21385.10 : -302.92 = -1.397% (+/-3.20%) The code size change is (firmware with a lot of frozen code benefits the most): bare-arm: +396 +0.697% minimal x86: +1595 +0.979% [incl +32(data)] unix x64: +2408 +0.470% [incl +800(data)] unix nanbox: +1396 +0.309% [incl -96(data)] stm32: -1256 -0.318% PYBV10 cc3200: +288 +0.157% esp8266: -260 -0.037% GENERIC esp32: -216 -0.014% GENERIC[incl -1072(data)] nrf: +116 +0.067% pca10040 rp2: -664 -0.135% PICO samd: +844 +0.607% ADAFRUIT_ITSYBITSY_M4_EXPRESS As part of this change the .mpy file format version is bumped to version 6. And mpy-tool.py has been improved to provide a good visualisation of the contents of .mpy files. In summary: this commit changes the bytecode to use qstr indirection, and reworks the .mpy file format to be simpler and allow .mpy files to be executed in-place. Performance is not impacted too much. Eventually it will be possible to store such .mpy files in a linear, read-only, memory- mappable filesystem so they can be executed from flash/ROM. This will essentially be able to replace frozen code for most applications. Signed-off-by: Damien George <damien@micropython.org>
2021-10-22 12:22:47 +01:00
void mp_bytecode_print2(const mp_print_t *print, const byte *ip, size_t len, const mp_module_constants_t *cm) {
showbc: Make code object start pointer semi-public. This allows to pring either absolute addresses or relative offsets in jumps and code references.
2014-12-28 19:34:45 +00:00
mp_showbc_code_start = ip;
py: Rework bytecode and .mpy file format to be mostly static data. Background: .mpy files are precompiled .py files, built using mpy-cross, that contain compiled bytecode functions (and can also contain machine code). The benefit of using an .mpy file over a .py file is that they are faster to import and take less memory when importing. They are also smaller on disk. But the real benefit of .mpy files comes when they are frozen into the firmware. This is done by loading the .mpy file during compilation of the firmware and turning it into a set of big C data structures (the job of mpy-tool.py), which are then compiled and downloaded into the ROM of a device. These C data structures can be executed in-place, ie directly from ROM. This makes importing even faster because there is very little to do, and also means such frozen modules take up much less RAM (because their bytecode stays in ROM). The downside of frozen code is that it requires recompiling and reflashing the entire firmware. This can be a big barrier to entry, slows down development time, and makes it harder to do OTA updates of frozen code (because the whole firmware must be updated). This commit attempts to solve this problem by providing a solution that sits between loading .mpy files into RAM and freezing them into the firmware. The .mpy file format has been reworked so that it consists of data and bytecode which is mostly static and ready to run in-place. If these new .mpy files are located in flash/ROM which is memory addressable, the .mpy file can be executed (mostly) in-place. With this approach there is still a small amount of unpacking and linking of the .mpy file that needs to be done when it's imported, but it's still much better than loading an .mpy from disk into RAM (although not as good as freezing .mpy files into the firmware). The main trick to make static .mpy files is to adjust the bytecode so any qstrs that it references now go through a lookup table to convert from local qstr number in the module to global qstr number in the firmware. That means the bytecode does not need linking/rewriting of qstrs when it's loaded. Instead only a small qstr table needs to be built (and put in RAM) at import time. This means the bytecode itself is static/constant and can be used directly if it's in addressable memory. Also the qstr string data in the .mpy file, and some constant object data, can be used directly. Note that the qstr table is global to the module (ie not per function). In more detail, in the VM what used to be (schematically): qst = DECODE_QSTR_VALUE; is now (schematically): idx = DECODE_QSTR_INDEX; qst = qstr_table[idx]; That allows the bytecode to be fixed at compile time and not need relinking/rewriting of the qstr values. Only qstr_table needs to be linked when the .mpy is loaded. Incidentally, this helps to reduce the size of bytecode because what used to be 2-byte qstr values in the bytecode are now (mostly) 1-byte indices. If the module uses the same qstr more than two times then the bytecode is smaller than before. The following changes are measured for this commit compared to the previous (the baseline): - average 7%-9% reduction in size of .mpy files - frozen code size is reduced by about 5%-7% - importing .py files uses about 5% less RAM in total - importing .mpy files uses about 4% less RAM in total - importing .py and .mpy files takes about the same time as before The qstr indirection in the bytecode has only a small impact on VM performance. For stm32 on PYBv1.0 the performance change of this commit is: diff of scores (higher is better) N=100 M=100 baseline -> this-commit diff diff% (error%) bm_chaos.py 371.07 -> 357.39 : -13.68 = -3.687% (+/-0.02%) bm_fannkuch.py 78.72 -> 77.49 : -1.23 = -1.563% (+/-0.01%) bm_fft.py 2591.73 -> 2539.28 : -52.45 = -2.024% (+/-0.00%) bm_float.py 6034.93 -> 5908.30 : -126.63 = -2.098% (+/-0.01%) bm_hexiom.py 48.96 -> 47.93 : -1.03 = -2.104% (+/-0.00%) bm_nqueens.py 4510.63 -> 4459.94 : -50.69 = -1.124% (+/-0.00%) bm_pidigits.py 650.28 -> 644.96 : -5.32 = -0.818% (+/-0.23%) core_import_mpy_multi.py 564.77 -> 581.49 : +16.72 = +2.960% (+/-0.01%) core_import_mpy_single.py 68.67 -> 67.16 : -1.51 = -2.199% (+/-0.01%) core_qstr.py 64.16 -> 64.12 : -0.04 = -0.062% (+/-0.00%) core_yield_from.py 362.58 -> 354.50 : -8.08 = -2.228% (+/-0.00%) misc_aes.py 429.69 -> 405.59 : -24.10 = -5.609% (+/-0.01%) misc_mandel.py 3485.13 -> 3416.51 : -68.62 = -1.969% (+/-0.00%) misc_pystone.py 2496.53 -> 2405.56 : -90.97 = -3.644% (+/-0.01%) misc_raytrace.py 381.47 -> 374.01 : -7.46 = -1.956% (+/-0.01%) viper_call0.py 576.73 -> 572.49 : -4.24 = -0.735% (+/-0.04%) viper_call1a.py 550.37 -> 546.21 : -4.16 = -0.756% (+/-0.09%) viper_call1b.py 438.23 -> 435.68 : -2.55 = -0.582% (+/-0.06%) viper_call1c.py 442.84 -> 440.04 : -2.80 = -0.632% (+/-0.08%) viper_call2a.py 536.31 -> 532.35 : -3.96 = -0.738% (+/-0.06%) viper_call2b.py 382.34 -> 377.07 : -5.27 = -1.378% (+/-0.03%) And for unix on x64: diff of scores (higher is better) N=2000 M=2000 baseline -> this-commit diff diff% (error%) bm_chaos.py 13594.20 -> 13073.84 : -520.36 = -3.828% (+/-5.44%) bm_fannkuch.py 60.63 -> 59.58 : -1.05 = -1.732% (+/-3.01%) bm_fft.py 112009.15 -> 111603.32 : -405.83 = -0.362% (+/-4.03%) bm_float.py 246202.55 -> 247923.81 : +1721.26 = +0.699% (+/-2.79%) bm_hexiom.py 615.65 -> 617.21 : +1.56 = +0.253% (+/-1.64%) bm_nqueens.py 215807.95 -> 215600.96 : -206.99 = -0.096% (+/-3.52%) bm_pidigits.py 8246.74 -> 8422.82 : +176.08 = +2.135% (+/-3.64%) misc_aes.py 16133.00 -> 16452.74 : +319.74 = +1.982% (+/-1.50%) misc_mandel.py 128146.69 -> 130796.43 : +2649.74 = +2.068% (+/-3.18%) misc_pystone.py 83811.49 -> 83124.85 : -686.64 = -0.819% (+/-1.03%) misc_raytrace.py 21688.02 -> 21385.10 : -302.92 = -1.397% (+/-3.20%) The code size change is (firmware with a lot of frozen code benefits the most): bare-arm: +396 +0.697% minimal x86: +1595 +0.979% [incl +32(data)] unix x64: +2408 +0.470% [incl +800(data)] unix nanbox: +1396 +0.309% [incl -96(data)] stm32: -1256 -0.318% PYBV10 cc3200: +288 +0.157% esp8266: -260 -0.037% GENERIC esp32: -216 -0.014% GENERIC[incl -1072(data)] nrf: +116 +0.067% pca10040 rp2: -664 -0.135% PICO samd: +844 +0.607% ADAFRUIT_ITSYBITSY_M4_EXPRESS As part of this change the .mpy file format version is bumped to version 6. And mpy-tool.py has been improved to provide a good visualisation of the contents of .mpy files. In summary: this commit changes the bytecode to use qstr indirection, and reworks the .mpy file format to be simpler and allow .mpy files to be executed in-place. Performance is not impacted too much. Eventually it will be possible to store such .mpy files in a linear, read-only, memory- mappable filesystem so they can be executed from flash/ROM. This will essentially be able to replace frozen code for most applications. Signed-off-by: Damien George <damien@micropython.org>
2021-10-22 12:22:47 +01:00
mp_showbc_constants = cm;
py, unix: Allow to compile with -Wsign-compare. See issue #699.
2015-01-16 17:47:07 +00:00
while (ip < len + mp_showbc_code_start) {
py/showbc: Pass in an mp_print_t struct to all bytecode-print functions. So the output can be redirected if needed. Signed-off-by: Damien George <damien@micropython.org>
2020-09-11 08:22:28 +01:00
mp_printf(print, "%02u ", (uint)(ip - mp_showbc_code_start));
ip = mp_bytecode_print_str(print, ip);
mp_printf(print, "\n");
Add function to decode and show byte code.
2013-11-17 13:19:33 +00:00
}
}
Factor and simplify Makefile's and mpconfig, part 2.
2014-01-07 15:20:33 +00:00
py: Add module/function/class name to exceptions. Exceptions know source file, line and block name. Also tidy up some debug printing functions and provide a global flag to enable/disable them.
2014-01-19 11:48:48 +00:00
#endif // MICROPY_DEBUG_PRINTERS