mirrors
/
micropython
mirror of https://github.com/micropython/micropython.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
micropython/py/objmodule.c

Ignoring revisions in .git-blame-ignore-revs. Click here to bypass and see the normal blame view.

322 lines
12 KiB
C
Raw Normal View History

Add license header to (almost) all files. Blanket wide to all .c and .h files. Some files originating from ST are difficult to deal with (license wise) so it was left out of those. Also merged modpyb.h, modos.h, modstm.h and modtime.h in stmhal/.
2014-05-03 23:27:38 +01:00
/*
all: Use the name MicroPython consistently in comments There were several different spellings of MicroPython present in comments, when there should be only one.
2017-06-30 08:22:17 +01:00
* This file is part of the MicroPython project, http://micropython.org/
Add license header to (almost) all files. Blanket wide to all .c and .h files. Some files originating from ST are difficult to deal with (license wise) so it was left out of those. Also merged modpyb.h, modos.h, modstm.h and modtime.h in stmhal/.
2014-05-03 23:27:38 +01:00
*
* The MIT License (MIT)
*
py: Automatically provide weak links from "foo" to "ufoo" module name. This commit implements automatic module weak links for all built-in modules, by searching for "ufoo" in the built-in module list if "foo" cannot be found. This means that all modules named "ufoo" are always available as "foo". Also, a port can no longer add any other weak links, which makes strict the definition of a weak link. It saves some code size (about 100-200 bytes) on ports that previously had lots of weak links. Some changes from the previous behaviour: - It doesn't intern the non-u module names (eg "foo" is not interned), which saves code size, but will mean that "import foo" creates a new qstr (namely "foo") in RAM (unless the importing module is frozen). - help('modules') no longer lists non-u module names, only the u-variants; this reduces duplication in the help listing. Weak links are effectively the same as having a set of symbolic links on the filesystem that is searched last. So an "import foo" will search built-in modules first, then all paths in sys.path, then weak links last, importing "ufoo" if it exists. Thus a file called "foo.py" somewhere in sys.path will still have precedence over the weak link of "foo" to "ufoo". See issues: #1740, #4449, #5229, #5241.
2019-10-21 15:06:34 +01:00
* Copyright (c) 2013-2019 Damien P. George
various: Add and update my copyright line based on git history. For modules I initially created or made substantial contributions to.
2019-05-14 13:51:57 +01:00
* Copyright (c) 2014-2015 Paul Sokolovsky
Add license header to (almost) all files. Blanket wide to all .c and .h files. Some files originating from ST are difficult to deal with (license wise) so it was left out of those. Also merged modpyb.h, modos.h, modstm.h and modtime.h in stmhal/.
2014-05-03 23:27:38 +01:00
*
* Permission is hereby granted, free of charge, to any person obtaining a copy
* of this software and associated documentation files (the "Software"), to deal
* in the Software without restriction, including without limitation the rights
* to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
* copies of the Software, and to permit persons to whom the Software is
* furnished to do so, subject to the following conditions:
*
* The above copyright notice and this permission notice shall be included in
* all copies or substantial portions of the Software.
*
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
* FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
* AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
* LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
* OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
* THE SOFTWARE.
*/
Add module object, to be used eventually for import.
2014-01-02 21:30:26 +00:00
#include <stdlib.h>
py: Automatically provide weak links from "foo" to "ufoo" module name. This commit implements automatic module weak links for all built-in modules, by searching for "ufoo" in the built-in module list if "foo" cannot be found. This means that all modules named "ufoo" are always available as "foo". Also, a port can no longer add any other weak links, which makes strict the definition of a weak link. It saves some code size (about 100-200 bytes) on ports that previously had lots of weak links. Some changes from the previous behaviour: - It doesn't intern the non-u module names (eg "foo" is not interned), which saves code size, but will mean that "import foo" creates a new qstr (namely "foo") in RAM (unless the importing module is frozen). - help('modules') no longer lists non-u module names, only the u-variants; this reduces duplication in the help listing. Weak links are effectively the same as having a set of symbolic links on the filesystem that is searched last. So an "import foo" will search built-in modules first, then all paths in sys.path, then weak links last, importing "ufoo" if it exists. Thus a file called "foo.py" somewhere in sys.path will still have precedence over the weak link of "foo" to "ufoo". See issues: #1740, #4449, #5229, #5241.
2019-10-21 15:06:34 +01:00
#include <string.h>
Add module object, to be used eventually for import.
2014-01-02 21:30:26 +00:00
#include <assert.h>
py: Move to guarded includes, everywhere in py/ core. Addresses issue #1022.
2015-01-01 20:27:54 +00:00
#include "py/objmodule.h"
#include "py/runtime.h"
#include "py/builtin.h"
Proper support for registering builtin modules in ROM. Comes with some refactoring of code and renaming of files. All modules are now named mod*.[ch].
2014-03-25 14:18:18 +00:00
py: Update and rework build system for including external C modules. How to use this feature is documented in docs/develop/cmodules.rst.
2018-12-12 05:50:55 +00:00
#include "genhdr/moduledefs.h"
py/builtinimport: Refactor module importing. Simplify and document/comment the handling of builtin import for: - already-loaded modules - built-in modules - built-in umodules (formerly weak links) - filesystem modules Retains existing functionality with smaller code size but should also facilitate potential new features (built-in packages, controlling the frozen path). Also makes the (unix-only) -m behavior a bit more obvious and configurable. Code size change with this commit: bare-arm: +0 +0.000% minimal x86: -64 -0.039% unix x64: -32 -0.006% unix nanbox: -4 -0.001% stm32: -184 -0.047% PYBV10 cc3200: -120 -0.065% esp8266: -228 -0.033% GENERIC esp32: -268 -0.018% GENERIC[incl +16(data)] nrf: -152 -0.087% pca10040 rp2: -256 -0.052% PICO samd: -80 -0.057% ADAFRUIT_ITSYBITSY_M4_EXPRESS
2020-02-26 04:24:09 +00:00
#if MICROPY_MODULE_BUILTIN_INIT
STATIC void mp_module_call_init(mp_obj_t module_name, mp_obj_t module_obj);
#endif
py: Overhaul and simplify printf/pfenv mechanism. Previous to this patch the printing mechanism was a bit of a tangled mess. This patch attempts to consolidate printing into one interface. All (non-debug) printing now uses the mp_print* family of functions, mainly mp_printf. All these functions take an mp_print_t structure as their first argument, and this structure defines the printing backend through the "print_strn" function of said structure. Printing from the uPy core can reach the platform-defined print code via two paths: either through mp_sys_stdout_obj (defined pert port) in conjunction with mp_stream_write; or through the mp_plat_print structure which uses the MP_PLAT_PRINT_STRN macro to define how string are printed on the platform. The former is only used when MICROPY_PY_IO is defined. With this new scheme printing is generally more efficient (less layers to go through, less arguments to pass), and, given an mp_print_t* structure, one can call mp_print_str for efficiency instead of mp_printf("%s", ...). Code size is also reduced by around 200 bytes on Thumb2 archs.
2015-04-09 23:56:15 +01:00
STATIC void module_print(const mp_print_t *print, mp_obj_t self_in, mp_print_kind_t kind) {
py, unix: Allow to compile with -Wunused-parameter. See issue #699.
2015-01-20 12:47:20 +00:00
(void)kind;
py: Wrap all obj-ptr conversions in MP_OBJ_TO_PTR/MP_OBJ_FROM_PTR. This allows the mp_obj_t type to be configured to something other than a pointer-sized primitive type. This patch also includes additional changes to allow the code to compile when sizeof(mp_uint_t) != sizeof(void*), such as using size_t instead of mp_uint_t, and various casts.
2015-11-27 17:01:44 +00:00
mp_obj_module_t *self = MP_OBJ_TO_PTR(self_in);
py: Print imported module's location (__file__) if available.
2014-09-08 10:45:23 +01:00
all: Remove 'name' member from mp_obj_module_t struct. One can instead lookup __name__ in the modules dict to get the value.
2016-09-21 01:52:53 +01:00
const char *module_name = "";
mp_map_elem_t *elem = mp_map_lookup(&self->globals->map, MP_OBJ_NEW_QSTR(MP_QSTR___name__), MP_MAP_LOOKUP);
if (elem != NULL) {
module_name = mp_obj_str_get_str(elem->value);
}
py: Print imported module's location (__file__) if available.
2014-09-08 10:45:23 +01:00
#if MICROPY_PY___FILE__
// If we store __file__ to imported modules then try to lookup this
// symbol to give more information about the module.
all: Remove 'name' member from mp_obj_module_t struct. One can instead lookup __name__ in the modules dict to get the value.
2016-09-21 01:52:53 +01:00
elem = mp_map_lookup(&self->globals->map, MP_OBJ_NEW_QSTR(MP_QSTR___file__), MP_MAP_LOOKUP);
py: Print imported module's location (__file__) if available.
2014-09-08 10:45:23 +01:00
if (elem != NULL) {
all: Remove 'name' member from mp_obj_module_t struct. One can instead lookup __name__ in the modules dict to get the value.
2016-09-21 01:52:53 +01:00
mp_printf(print, "<module '%s' from '%s'>", module_name, mp_obj_str_get_str(elem->value));
py: Print imported module's location (__file__) if available.
2014-09-08 10:45:23 +01:00
return;
}
#endif
all: Remove 'name' member from mp_obj_module_t struct. One can instead lookup __name__ in the modules dict to get the value.
2016-09-21 01:52:53 +01:00
mp_printf(print, "<module '%s'>", module_name);
Add module object, to be used eventually for import.
2014-01-02 21:30:26 +00:00
}
py: Combine load_attr and store_attr type methods into one (attr). This simplifies the API for objects and reduces code size (by around 400 bytes on Thumb2, and around 2k on x86). Performance impact was measured with Pystone score, but change was barely noticeable.
2015-04-01 15:10:50 +01:00
STATIC void module_attr(mp_obj_t self_in, qstr attr, mp_obj_t *dest) {
py: Wrap all obj-ptr conversions in MP_OBJ_TO_PTR/MP_OBJ_FROM_PTR. This allows the mp_obj_t type to be configured to something other than a pointer-sized primitive type. This patch also includes additional changes to allow the code to compile when sizeof(mp_uint_t) != sizeof(void*), such as using size_t instead of mp_uint_t, and various casts.
2015-11-27 17:01:44 +00:00
mp_obj_module_t *self = MP_OBJ_TO_PTR(self_in);
py: Combine load_attr and store_attr type methods into one (attr). This simplifies the API for objects and reduces code size (by around 400 bytes on Thumb2, and around 2k on x86). Performance impact was measured with Pystone score, but change was barely noticeable.
2015-04-01 15:10:50 +01:00
if (dest[0] == MP_OBJ_NULL) {
// load attribute
mp_map_elem_t *elem = mp_map_lookup(&self->globals->map, MP_OBJ_NEW_QSTR(attr), MP_MAP_LOOKUP);
if (elem != NULL) {
dest[0] = elem->value;
py/objmodule: Implement PEP 562's __getattr__ for modules. Configurable via MICROPY_MODULE_GETATTR, disabled by default. Among other things __getattr__ for modules can help to build lazy loading / code unloading at runtime.
2018-10-22 17:34:29 +01:00
#if MICROPY_MODULE_GETATTR
} else if (attr != MP_QSTR___getattr__) {
elem = mp_map_lookup(&self->globals->map, MP_OBJ_NEW_QSTR(MP_QSTR___getattr__), MP_MAP_LOOKUP);
if (elem != NULL) {
dest[0] = mp_call_function_1(elem->value, MP_OBJ_NEW_QSTR(attr));
}
#endif
py: Allow builtins to be overridden. This patch adds a configuration option (MICROPY_CAN_OVERRIDE_BUILTINS) which, when enabled, allows to override all names within the builtins module. A builtins override dict is created the first time the user assigns to a name in the builtins model, and then that dict is searched first on subsequent lookups. Note that this implementation doesn't allow deleting of names. This patch also does some refactoring of builtins code, creating the modbuiltins.c file. Addresses issue #959.
2014-12-09 16:19:48 +00:00
}
py: Finish implementation of all del opcodes. At this point, all opcodes are now implemented! Some del opcodes have been combined with store opcodes, with the value to store being MP_OBJ_NULL.
2014-04-08 21:11:49 +01:00
} else {
py: Combine load_attr and store_attr type methods into one (attr). This simplifies the API for objects and reduces code size (by around 400 bytes on Thumb2, and around 2k on x86). Performance impact was measured with Pystone score, but change was barely noticeable.
2015-04-01 15:10:50 +01:00
// delete/store attribute
mp_obj_dict_t *dict = self->globals;
if (dict->map.is_fixed) {
#if MICROPY_CAN_OVERRIDE_BUILTINS
if (dict == &mp_module_builtins_globals) {
if (MP_STATE_VM(mp_module_builtins_override_dict) == NULL) {
py: Wrap all obj-ptr conversions in MP_OBJ_TO_PTR/MP_OBJ_FROM_PTR. This allows the mp_obj_t type to be configured to something other than a pointer-sized primitive type. This patch also includes additional changes to allow the code to compile when sizeof(mp_uint_t) != sizeof(void*), such as using size_t instead of mp_uint_t, and various casts.
2015-11-27 17:01:44 +00:00
MP_STATE_VM(mp_module_builtins_override_dict) = MP_OBJ_TO_PTR(mp_obj_new_dict(1));
py: Combine load_attr and store_attr type methods into one (attr). This simplifies the API for objects and reduces code size (by around 400 bytes on Thumb2, and around 2k on x86). Performance impact was measured with Pystone score, but change was barely noticeable.
2015-04-01 15:10:50 +01:00
}
dict = MP_STATE_VM(mp_module_builtins_override_dict);
} else
#endif
{
// can't delete or store to fixed map
return;
}
}
if (dest[1] == MP_OBJ_NULL) {
// delete attribute
py: Wrap all obj-ptr conversions in MP_OBJ_TO_PTR/MP_OBJ_FROM_PTR. This allows the mp_obj_t type to be configured to something other than a pointer-sized primitive type. This patch also includes additional changes to allow the code to compile when sizeof(mp_uint_t) != sizeof(void*), such as using size_t instead of mp_uint_t, and various casts.
2015-11-27 17:01:44 +00:00
mp_obj_dict_delete(MP_OBJ_FROM_PTR(dict), MP_OBJ_NEW_QSTR(attr));
py: Combine load_attr and store_attr type methods into one (attr). This simplifies the API for objects and reduces code size (by around 400 bytes on Thumb2, and around 2k on x86). Performance impact was measured with Pystone score, but change was barely noticeable.
2015-04-01 15:10:50 +01:00
} else {
// store attribute
py: Wrap all obj-ptr conversions in MP_OBJ_TO_PTR/MP_OBJ_FROM_PTR. This allows the mp_obj_t type to be configured to something other than a pointer-sized primitive type. This patch also includes additional changes to allow the code to compile when sizeof(mp_uint_t) != sizeof(void*), such as using size_t instead of mp_uint_t, and various casts.
2015-11-27 17:01:44 +00:00
mp_obj_dict_store(MP_OBJ_FROM_PTR(dict), MP_OBJ_NEW_QSTR(attr), dest[1]);
py: Combine load_attr and store_attr type methods into one (attr). This simplifies the API for objects and reduces code size (by around 400 bytes on Thumb2, and around 2k on x86). Performance impact was measured with Pystone score, but change was barely noticeable.
2015-04-01 15:10:50 +01:00
}
dest[0] = MP_OBJ_NULL; // indicate success
py: Finish implementation of all del opcodes. At this point, all opcodes are now implemented! Some del opcodes have been combined with store opcodes, with the value to store being MP_OBJ_NULL.
2014-04-08 21:11:49 +01:00
}
Improved type/class/instance code; mp_obj_type_t now has load_attr, store_attr. Creating of classes (types) and instances is much more like CPython now. You can use "type('name', (), {...})" to create classes.
2014-01-09 20:57:50 +00:00
}
Implement ROMable modules. Add math module. mp_module_obj_t can now be put in ROM. Configuration of float type is now similar to longint: can now choose none, float or double as the implementation. math module has basic math functions. For STM port, these are not yet implemented (they are just stub functions).
2014-03-08 15:24:39 +00:00
const mp_obj_type_t mp_type_module = {
Implement proper exception type hierarchy. Each built-in exception is now a type, with base type BaseException. C exceptions are created by passing a pointer to the exception type to make an instance of. When raising an exception from the VM, an instance is created automatically if an exception type is raised (as opposed to an exception instance). Exception matching (RT_BINARY_OP_EXCEPTION_MATCH) is now proper. Handling of parse error changed to match new exceptions. mp_const_type renamed to mp_type_type for consistency.
2014-02-15 16:10:44 +00:00
{ &mp_type_type },
Change mp_obj_type_t.name from const char * to qstr. Ultimately all static strings should be qstr. This entry in the type structure is only used for printing error messages (to tell the type of the bad argument), and printing objects that don't supply a .print method.
2014-02-15 11:34:50 +00:00
.name = MP_QSTR_module,
Merge branch 'cplusplus' of https://github.com/ian-v/micropython into ian-v-cplusplus Conflicts: py/objcomplex.c
2014-01-07 15:58:30 +00:00
.print = module_print,
py: Combine load_attr and store_attr type methods into one (attr). This simplifies the API for objects and reduces code size (by around 400 bytes on Thumb2, and around 2k on x86). Performance impact was measured with Pystone score, but change was barely noticeable.
2015-04-01 15:10:50 +01:00
.attr = module_attr,
Add module object, to be used eventually for import.
2014-01-02 21:30:26 +00:00
};
py: Rework bytecode and .mpy file format to be mostly static data. Background: .mpy files are precompiled .py files, built using mpy-cross, that contain compiled bytecode functions (and can also contain machine code). The benefit of using an .mpy file over a .py file is that they are faster to import and take less memory when importing. They are also smaller on disk. But the real benefit of .mpy files comes when they are frozen into the firmware. This is done by loading the .mpy file during compilation of the firmware and turning it into a set of big C data structures (the job of mpy-tool.py), which are then compiled and downloaded into the ROM of a device. These C data structures can be executed in-place, ie directly from ROM. This makes importing even faster because there is very little to do, and also means such frozen modules take up much less RAM (because their bytecode stays in ROM). The downside of frozen code is that it requires recompiling and reflashing the entire firmware. This can be a big barrier to entry, slows down development time, and makes it harder to do OTA updates of frozen code (because the whole firmware must be updated). This commit attempts to solve this problem by providing a solution that sits between loading .mpy files into RAM and freezing them into the firmware. The .mpy file format has been reworked so that it consists of data and bytecode which is mostly static and ready to run in-place. If these new .mpy files are located in flash/ROM which is memory addressable, the .mpy file can be executed (mostly) in-place. With this approach there is still a small amount of unpacking and linking of the .mpy file that needs to be done when it's imported, but it's still much better than loading an .mpy from disk into RAM (although not as good as freezing .mpy files into the firmware). The main trick to make static .mpy files is to adjust the bytecode so any qstrs that it references now go through a lookup table to convert from local qstr number in the module to global qstr number in the firmware. That means the bytecode does not need linking/rewriting of qstrs when it's loaded. Instead only a small qstr table needs to be built (and put in RAM) at import time. This means the bytecode itself is static/constant and can be used directly if it's in addressable memory. Also the qstr string data in the .mpy file, and some constant object data, can be used directly. Note that the qstr table is global to the module (ie not per function). In more detail, in the VM what used to be (schematically): qst = DECODE_QSTR_VALUE; is now (schematically): idx = DECODE_QSTR_INDEX; qst = qstr_table[idx]; That allows the bytecode to be fixed at compile time and not need relinking/rewriting of the qstr values. Only qstr_table needs to be linked when the .mpy is loaded. Incidentally, this helps to reduce the size of bytecode because what used to be 2-byte qstr values in the bytecode are now (mostly) 1-byte indices. If the module uses the same qstr more than two times then the bytecode is smaller than before. The following changes are measured for this commit compared to the previous (the baseline): - average 7%-9% reduction in size of .mpy files - frozen code size is reduced by about 5%-7% - importing .py files uses about 5% less RAM in total - importing .mpy files uses about 4% less RAM in total - importing .py and .mpy files takes about the same time as before The qstr indirection in the bytecode has only a small impact on VM performance. For stm32 on PYBv1.0 the performance change of this commit is: diff of scores (higher is better) N=100 M=100 baseline -> this-commit diff diff% (error%) bm_chaos.py 371.07 -> 357.39 : -13.68 = -3.687% (+/-0.02%) bm_fannkuch.py 78.72 -> 77.49 : -1.23 = -1.563% (+/-0.01%) bm_fft.py 2591.73 -> 2539.28 : -52.45 = -2.024% (+/-0.00%) bm_float.py 6034.93 -> 5908.30 : -126.63 = -2.098% (+/-0.01%) bm_hexiom.py 48.96 -> 47.93 : -1.03 = -2.104% (+/-0.00%) bm_nqueens.py 4510.63 -> 4459.94 : -50.69 = -1.124% (+/-0.00%) bm_pidigits.py 650.28 -> 644.96 : -5.32 = -0.818% (+/-0.23%) core_import_mpy_multi.py 564.77 -> 581.49 : +16.72 = +2.960% (+/-0.01%) core_import_mpy_single.py 68.67 -> 67.16 : -1.51 = -2.199% (+/-0.01%) core_qstr.py 64.16 -> 64.12 : -0.04 = -0.062% (+/-0.00%) core_yield_from.py 362.58 -> 354.50 : -8.08 = -2.228% (+/-0.00%) misc_aes.py 429.69 -> 405.59 : -24.10 = -5.609% (+/-0.01%) misc_mandel.py 3485.13 -> 3416.51 : -68.62 = -1.969% (+/-0.00%) misc_pystone.py 2496.53 -> 2405.56 : -90.97 = -3.644% (+/-0.01%) misc_raytrace.py 381.47 -> 374.01 : -7.46 = -1.956% (+/-0.01%) viper_call0.py 576.73 -> 572.49 : -4.24 = -0.735% (+/-0.04%) viper_call1a.py 550.37 -> 546.21 : -4.16 = -0.756% (+/-0.09%) viper_call1b.py 438.23 -> 435.68 : -2.55 = -0.582% (+/-0.06%) viper_call1c.py 442.84 -> 440.04 : -2.80 = -0.632% (+/-0.08%) viper_call2a.py 536.31 -> 532.35 : -3.96 = -0.738% (+/-0.06%) viper_call2b.py 382.34 -> 377.07 : -5.27 = -1.378% (+/-0.03%) And for unix on x64: diff of scores (higher is better) N=2000 M=2000 baseline -> this-commit diff diff% (error%) bm_chaos.py 13594.20 -> 13073.84 : -520.36 = -3.828% (+/-5.44%) bm_fannkuch.py 60.63 -> 59.58 : -1.05 = -1.732% (+/-3.01%) bm_fft.py 112009.15 -> 111603.32 : -405.83 = -0.362% (+/-4.03%) bm_float.py 246202.55 -> 247923.81 : +1721.26 = +0.699% (+/-2.79%) bm_hexiom.py 615.65 -> 617.21 : +1.56 = +0.253% (+/-1.64%) bm_nqueens.py 215807.95 -> 215600.96 : -206.99 = -0.096% (+/-3.52%) bm_pidigits.py 8246.74 -> 8422.82 : +176.08 = +2.135% (+/-3.64%) misc_aes.py 16133.00 -> 16452.74 : +319.74 = +1.982% (+/-1.50%) misc_mandel.py 128146.69 -> 130796.43 : +2649.74 = +2.068% (+/-3.18%) misc_pystone.py 83811.49 -> 83124.85 : -686.64 = -0.819% (+/-1.03%) misc_raytrace.py 21688.02 -> 21385.10 : -302.92 = -1.397% (+/-3.20%) The code size change is (firmware with a lot of frozen code benefits the most): bare-arm: +396 +0.697% minimal x86: +1595 +0.979% [incl +32(data)] unix x64: +2408 +0.470% [incl +800(data)] unix nanbox: +1396 +0.309% [incl -96(data)] stm32: -1256 -0.318% PYBV10 cc3200: +288 +0.157% esp8266: -260 -0.037% GENERIC esp32: -216 -0.014% GENERIC[incl -1072(data)] nrf: +116 +0.067% pca10040 rp2: -664 -0.135% PICO samd: +844 +0.607% ADAFRUIT_ITSYBITSY_M4_EXPRESS As part of this change the .mpy file format version is bumped to version 6. And mpy-tool.py has been improved to provide a good visualisation of the contents of .mpy files. In summary: this commit changes the bytecode to use qstr indirection, and reworks the .mpy file format to be simpler and allow .mpy files to be executed in-place. Performance is not impacted too much. Eventually it will be possible to store such .mpy files in a linear, read-only, memory- mappable filesystem so they can be executed from flash/ROM. This will essentially be able to replace frozen code for most applications. Signed-off-by: Damien George <damien@micropython.org>
2021-10-22 12:22:47 +01:00
#include "py/bc.h"
Add module object, to be used eventually for import.
2014-01-02 21:30:26 +00:00
mp_obj_t mp_obj_new_module(qstr module_name) {
py/modsys: Implement sys.modules. This for example will allow people to reload modules which didn't load successfully (e.g. due to syntax error).
2015-12-04 22:09:10 +00:00
mp_map_t *mp_loaded_modules_map = &MP_STATE_VM(mp_loaded_modules_dict).map;
mp_map_elem_t *el = mp_map_lookup(mp_loaded_modules_map, MP_OBJ_NEW_QSTR(module_name), MP_MAP_LOOKUP_ADD_IF_NOT_FOUND);
Implement modules as singletons Python semantics. In Python, importing module several times returns same underlying module object. This also fixes import statement handling for builtin modules. There're still issues: 1. CPython exposes set of loaded modules as sys.modules, we may want to do that either. 2. Builtin modules are implicitly imported, which is not really correct. We should separate registering a (builtin) module and importing a module. CPython keeps builtin module names in sys.builtin_module_names .
2014-01-19 22:03:34 +00:00
// We could error out if module already exists, but let C extensions
// add new members to existing modules.
if (el->value != MP_OBJ_NULL) {
return el->value;
}
Implement ROMable modules. Add math module. mp_module_obj_t can now be put in ROM. Configuration of float type is now similar to longint: can now choose none, float or double as the implementation. math module has basic math functions. For STM port, these are not yet implemented (they are just stub functions).
2014-03-08 15:24:39 +00:00
// create new module object
py: Rework bytecode and .mpy file format to be mostly static data. Background: .mpy files are precompiled .py files, built using mpy-cross, that contain compiled bytecode functions (and can also contain machine code). The benefit of using an .mpy file over a .py file is that they are faster to import and take less memory when importing. They are also smaller on disk. But the real benefit of .mpy files comes when they are frozen into the firmware. This is done by loading the .mpy file during compilation of the firmware and turning it into a set of big C data structures (the job of mpy-tool.py), which are then compiled and downloaded into the ROM of a device. These C data structures can be executed in-place, ie directly from ROM. This makes importing even faster because there is very little to do, and also means such frozen modules take up much less RAM (because their bytecode stays in ROM). The downside of frozen code is that it requires recompiling and reflashing the entire firmware. This can be a big barrier to entry, slows down development time, and makes it harder to do OTA updates of frozen code (because the whole firmware must be updated). This commit attempts to solve this problem by providing a solution that sits between loading .mpy files into RAM and freezing them into the firmware. The .mpy file format has been reworked so that it consists of data and bytecode which is mostly static and ready to run in-place. If these new .mpy files are located in flash/ROM which is memory addressable, the .mpy file can be executed (mostly) in-place. With this approach there is still a small amount of unpacking and linking of the .mpy file that needs to be done when it's imported, but it's still much better than loading an .mpy from disk into RAM (although not as good as freezing .mpy files into the firmware). The main trick to make static .mpy files is to adjust the bytecode so any qstrs that it references now go through a lookup table to convert from local qstr number in the module to global qstr number in the firmware. That means the bytecode does not need linking/rewriting of qstrs when it's loaded. Instead only a small qstr table needs to be built (and put in RAM) at import time. This means the bytecode itself is static/constant and can be used directly if it's in addressable memory. Also the qstr string data in the .mpy file, and some constant object data, can be used directly. Note that the qstr table is global to the module (ie not per function). In more detail, in the VM what used to be (schematically): qst = DECODE_QSTR_VALUE; is now (schematically): idx = DECODE_QSTR_INDEX; qst = qstr_table[idx]; That allows the bytecode to be fixed at compile time and not need relinking/rewriting of the qstr values. Only qstr_table needs to be linked when the .mpy is loaded. Incidentally, this helps to reduce the size of bytecode because what used to be 2-byte qstr values in the bytecode are now (mostly) 1-byte indices. If the module uses the same qstr more than two times then the bytecode is smaller than before. The following changes are measured for this commit compared to the previous (the baseline): - average 7%-9% reduction in size of .mpy files - frozen code size is reduced by about 5%-7% - importing .py files uses about 5% less RAM in total - importing .mpy files uses about 4% less RAM in total - importing .py and .mpy files takes about the same time as before The qstr indirection in the bytecode has only a small impact on VM performance. For stm32 on PYBv1.0 the performance change of this commit is: diff of scores (higher is better) N=100 M=100 baseline -> this-commit diff diff% (error%) bm_chaos.py 371.07 -> 357.39 : -13.68 = -3.687% (+/-0.02%) bm_fannkuch.py 78.72 -> 77.49 : -1.23 = -1.563% (+/-0.01%) bm_fft.py 2591.73 -> 2539.28 : -52.45 = -2.024% (+/-0.00%) bm_float.py 6034.93 -> 5908.30 : -126.63 = -2.098% (+/-0.01%) bm_hexiom.py 48.96 -> 47.93 : -1.03 = -2.104% (+/-0.00%) bm_nqueens.py 4510.63 -> 4459.94 : -50.69 = -1.124% (+/-0.00%) bm_pidigits.py 650.28 -> 644.96 : -5.32 = -0.818% (+/-0.23%) core_import_mpy_multi.py 564.77 -> 581.49 : +16.72 = +2.960% (+/-0.01%) core_import_mpy_single.py 68.67 -> 67.16 : -1.51 = -2.199% (+/-0.01%) core_qstr.py 64.16 -> 64.12 : -0.04 = -0.062% (+/-0.00%) core_yield_from.py 362.58 -> 354.50 : -8.08 = -2.228% (+/-0.00%) misc_aes.py 429.69 -> 405.59 : -24.10 = -5.609% (+/-0.01%) misc_mandel.py 3485.13 -> 3416.51 : -68.62 = -1.969% (+/-0.00%) misc_pystone.py 2496.53 -> 2405.56 : -90.97 = -3.644% (+/-0.01%) misc_raytrace.py 381.47 -> 374.01 : -7.46 = -1.956% (+/-0.01%) viper_call0.py 576.73 -> 572.49 : -4.24 = -0.735% (+/-0.04%) viper_call1a.py 550.37 -> 546.21 : -4.16 = -0.756% (+/-0.09%) viper_call1b.py 438.23 -> 435.68 : -2.55 = -0.582% (+/-0.06%) viper_call1c.py 442.84 -> 440.04 : -2.80 = -0.632% (+/-0.08%) viper_call2a.py 536.31 -> 532.35 : -3.96 = -0.738% (+/-0.06%) viper_call2b.py 382.34 -> 377.07 : -5.27 = -1.378% (+/-0.03%) And for unix on x64: diff of scores (higher is better) N=2000 M=2000 baseline -> this-commit diff diff% (error%) bm_chaos.py 13594.20 -> 13073.84 : -520.36 = -3.828% (+/-5.44%) bm_fannkuch.py 60.63 -> 59.58 : -1.05 = -1.732% (+/-3.01%) bm_fft.py 112009.15 -> 111603.32 : -405.83 = -0.362% (+/-4.03%) bm_float.py 246202.55 -> 247923.81 : +1721.26 = +0.699% (+/-2.79%) bm_hexiom.py 615.65 -> 617.21 : +1.56 = +0.253% (+/-1.64%) bm_nqueens.py 215807.95 -> 215600.96 : -206.99 = -0.096% (+/-3.52%) bm_pidigits.py 8246.74 -> 8422.82 : +176.08 = +2.135% (+/-3.64%) misc_aes.py 16133.00 -> 16452.74 : +319.74 = +1.982% (+/-1.50%) misc_mandel.py 128146.69 -> 130796.43 : +2649.74 = +2.068% (+/-3.18%) misc_pystone.py 83811.49 -> 83124.85 : -686.64 = -0.819% (+/-1.03%) misc_raytrace.py 21688.02 -> 21385.10 : -302.92 = -1.397% (+/-3.20%) The code size change is (firmware with a lot of frozen code benefits the most): bare-arm: +396 +0.697% minimal x86: +1595 +0.979% [incl +32(data)] unix x64: +2408 +0.470% [incl +800(data)] unix nanbox: +1396 +0.309% [incl -96(data)] stm32: -1256 -0.318% PYBV10 cc3200: +288 +0.157% esp8266: -260 -0.037% GENERIC esp32: -216 -0.014% GENERIC[incl -1072(data)] nrf: +116 +0.067% pca10040 rp2: -664 -0.135% PICO samd: +844 +0.607% ADAFRUIT_ITSYBITSY_M4_EXPRESS As part of this change the .mpy file format version is bumped to version 6. And mpy-tool.py has been improved to provide a good visualisation of the contents of .mpy files. In summary: this commit changes the bytecode to use qstr indirection, and reworks the .mpy file format to be simpler and allow .mpy files to be executed in-place. Performance is not impacted too much. Eventually it will be possible to store such .mpy files in a linear, read-only, memory- mappable filesystem so they can be executed from flash/ROM. This will essentially be able to replace frozen code for most applications. Signed-off-by: Damien George <damien@micropython.org>
2021-10-22 12:22:47 +01:00
mp_module_context_t *o = m_new_obj(mp_module_context_t);
o->module.base.type = &mp_type_module;
o->module.globals = MP_OBJ_TO_PTR(mp_obj_new_dict(MICROPY_MODULE_DICT_SIZE));
Implement ROMable modules. Add math module. mp_module_obj_t can now be put in ROM. Configuration of float type is now similar to longint: can now choose none, float or double as the implementation. math module has basic math functions. For STM port, these are not yet implemented (they are just stub functions).
2014-03-08 15:24:39 +00:00
// store __name__ entry in the module
py: Rework bytecode and .mpy file format to be mostly static data. Background: .mpy files are precompiled .py files, built using mpy-cross, that contain compiled bytecode functions (and can also contain machine code). The benefit of using an .mpy file over a .py file is that they are faster to import and take less memory when importing. They are also smaller on disk. But the real benefit of .mpy files comes when they are frozen into the firmware. This is done by loading the .mpy file during compilation of the firmware and turning it into a set of big C data structures (the job of mpy-tool.py), which are then compiled and downloaded into the ROM of a device. These C data structures can be executed in-place, ie directly from ROM. This makes importing even faster because there is very little to do, and also means such frozen modules take up much less RAM (because their bytecode stays in ROM). The downside of frozen code is that it requires recompiling and reflashing the entire firmware. This can be a big barrier to entry, slows down development time, and makes it harder to do OTA updates of frozen code (because the whole firmware must be updated). This commit attempts to solve this problem by providing a solution that sits between loading .mpy files into RAM and freezing them into the firmware. The .mpy file format has been reworked so that it consists of data and bytecode which is mostly static and ready to run in-place. If these new .mpy files are located in flash/ROM which is memory addressable, the .mpy file can be executed (mostly) in-place. With this approach there is still a small amount of unpacking and linking of the .mpy file that needs to be done when it's imported, but it's still much better than loading an .mpy from disk into RAM (although not as good as freezing .mpy files into the firmware). The main trick to make static .mpy files is to adjust the bytecode so any qstrs that it references now go through a lookup table to convert from local qstr number in the module to global qstr number in the firmware. That means the bytecode does not need linking/rewriting of qstrs when it's loaded. Instead only a small qstr table needs to be built (and put in RAM) at import time. This means the bytecode itself is static/constant and can be used directly if it's in addressable memory. Also the qstr string data in the .mpy file, and some constant object data, can be used directly. Note that the qstr table is global to the module (ie not per function). In more detail, in the VM what used to be (schematically): qst = DECODE_QSTR_VALUE; is now (schematically): idx = DECODE_QSTR_INDEX; qst = qstr_table[idx]; That allows the bytecode to be fixed at compile time and not need relinking/rewriting of the qstr values. Only qstr_table needs to be linked when the .mpy is loaded. Incidentally, this helps to reduce the size of bytecode because what used to be 2-byte qstr values in the bytecode are now (mostly) 1-byte indices. If the module uses the same qstr more than two times then the bytecode is smaller than before. The following changes are measured for this commit compared to the previous (the baseline): - average 7%-9% reduction in size of .mpy files - frozen code size is reduced by about 5%-7% - importing .py files uses about 5% less RAM in total - importing .mpy files uses about 4% less RAM in total - importing .py and .mpy files takes about the same time as before The qstr indirection in the bytecode has only a small impact on VM performance. For stm32 on PYBv1.0 the performance change of this commit is: diff of scores (higher is better) N=100 M=100 baseline -> this-commit diff diff% (error%) bm_chaos.py 371.07 -> 357.39 : -13.68 = -3.687% (+/-0.02%) bm_fannkuch.py 78.72 -> 77.49 : -1.23 = -1.563% (+/-0.01%) bm_fft.py 2591.73 -> 2539.28 : -52.45 = -2.024% (+/-0.00%) bm_float.py 6034.93 -> 5908.30 : -126.63 = -2.098% (+/-0.01%) bm_hexiom.py 48.96 -> 47.93 : -1.03 = -2.104% (+/-0.00%) bm_nqueens.py 4510.63 -> 4459.94 : -50.69 = -1.124% (+/-0.00%) bm_pidigits.py 650.28 -> 644.96 : -5.32 = -0.818% (+/-0.23%) core_import_mpy_multi.py 564.77 -> 581.49 : +16.72 = +2.960% (+/-0.01%) core_import_mpy_single.py 68.67 -> 67.16 : -1.51 = -2.199% (+/-0.01%) core_qstr.py 64.16 -> 64.12 : -0.04 = -0.062% (+/-0.00%) core_yield_from.py 362.58 -> 354.50 : -8.08 = -2.228% (+/-0.00%) misc_aes.py 429.69 -> 405.59 : -24.10 = -5.609% (+/-0.01%) misc_mandel.py 3485.13 -> 3416.51 : -68.62 = -1.969% (+/-0.00%) misc_pystone.py 2496.53 -> 2405.56 : -90.97 = -3.644% (+/-0.01%) misc_raytrace.py 381.47 -> 374.01 : -7.46 = -1.956% (+/-0.01%) viper_call0.py 576.73 -> 572.49 : -4.24 = -0.735% (+/-0.04%) viper_call1a.py 550.37 -> 546.21 : -4.16 = -0.756% (+/-0.09%) viper_call1b.py 438.23 -> 435.68 : -2.55 = -0.582% (+/-0.06%) viper_call1c.py 442.84 -> 440.04 : -2.80 = -0.632% (+/-0.08%) viper_call2a.py 536.31 -> 532.35 : -3.96 = -0.738% (+/-0.06%) viper_call2b.py 382.34 -> 377.07 : -5.27 = -1.378% (+/-0.03%) And for unix on x64: diff of scores (higher is better) N=2000 M=2000 baseline -> this-commit diff diff% (error%) bm_chaos.py 13594.20 -> 13073.84 : -520.36 = -3.828% (+/-5.44%) bm_fannkuch.py 60.63 -> 59.58 : -1.05 = -1.732% (+/-3.01%) bm_fft.py 112009.15 -> 111603.32 : -405.83 = -0.362% (+/-4.03%) bm_float.py 246202.55 -> 247923.81 : +1721.26 = +0.699% (+/-2.79%) bm_hexiom.py 615.65 -> 617.21 : +1.56 = +0.253% (+/-1.64%) bm_nqueens.py 215807.95 -> 215600.96 : -206.99 = -0.096% (+/-3.52%) bm_pidigits.py 8246.74 -> 8422.82 : +176.08 = +2.135% (+/-3.64%) misc_aes.py 16133.00 -> 16452.74 : +319.74 = +1.982% (+/-1.50%) misc_mandel.py 128146.69 -> 130796.43 : +2649.74 = +2.068% (+/-3.18%) misc_pystone.py 83811.49 -> 83124.85 : -686.64 = -0.819% (+/-1.03%) misc_raytrace.py 21688.02 -> 21385.10 : -302.92 = -1.397% (+/-3.20%) The code size change is (firmware with a lot of frozen code benefits the most): bare-arm: +396 +0.697% minimal x86: +1595 +0.979% [incl +32(data)] unix x64: +2408 +0.470% [incl +800(data)] unix nanbox: +1396 +0.309% [incl -96(data)] stm32: -1256 -0.318% PYBV10 cc3200: +288 +0.157% esp8266: -260 -0.037% GENERIC esp32: -216 -0.014% GENERIC[incl -1072(data)] nrf: +116 +0.067% pca10040 rp2: -664 -0.135% PICO samd: +844 +0.607% ADAFRUIT_ITSYBITSY_M4_EXPRESS As part of this change the .mpy file format version is bumped to version 6. And mpy-tool.py has been improved to provide a good visualisation of the contents of .mpy files. In summary: this commit changes the bytecode to use qstr indirection, and reworks the .mpy file format to be simpler and allow .mpy files to be executed in-place. Performance is not impacted too much. Eventually it will be possible to store such .mpy files in a linear, read-only, memory- mappable filesystem so they can be executed from flash/ROM. This will essentially be able to replace frozen code for most applications. Signed-off-by: Damien George <damien@micropython.org>
2021-10-22 12:22:47 +01:00
mp_obj_dict_store(MP_OBJ_FROM_PTR(o->module.globals), MP_OBJ_NEW_QSTR(MP_QSTR___name__), MP_OBJ_NEW_QSTR(module_name));
Implement ROMable modules. Add math module. mp_module_obj_t can now be put in ROM. Configuration of float type is now similar to longint: can now choose none, float or double as the implementation. math module has basic math functions. For STM port, these are not yet implemented (they are just stub functions).
2014-03-08 15:24:39 +00:00
// store the new module into the slot in the global dict holding all modules
py: Wrap all obj-ptr conversions in MP_OBJ_TO_PTR/MP_OBJ_FROM_PTR. This allows the mp_obj_t type to be configured to something other than a pointer-sized primitive type. This patch also includes additional changes to allow the code to compile when sizeof(mp_uint_t) != sizeof(void*), such as using size_t instead of mp_uint_t, and various casts.
2015-11-27 17:01:44 +00:00
el->value = MP_OBJ_FROM_PTR(o);
Implement ROMable modules. Add math module. mp_module_obj_t can now be put in ROM. Configuration of float type is now similar to longint: can now choose none, float or double as the implementation. math module has basic math functions. For STM port, these are not yet implemented (they are just stub functions).
2014-03-08 15:24:39 +00:00
// return the new module
py: Wrap all obj-ptr conversions in MP_OBJ_TO_PTR/MP_OBJ_FROM_PTR. This allows the mp_obj_t type to be configured to something other than a pointer-sized primitive type. This patch also includes additional changes to allow the code to compile when sizeof(mp_uint_t) != sizeof(void*), such as using size_t instead of mp_uint_t, and various casts.
2015-11-27 17:01:44 +00:00
return MP_OBJ_FROM_PTR(o);
Add module object, to be used eventually for import.
2014-01-02 21:30:26 +00:00
}
Proper support for registering builtin modules in ROM. Comes with some refactoring of code and renaming of files. All modules are now named mod*.[ch].
2014-03-25 14:18:18 +00:00
/******************************************************************************/
// Global module table and related functions
py: Add MP_ROM_* macros and mp_rom_* types and use them.
2015-11-27 13:38:15 +00:00
STATIC const mp_rom_map_elem_t mp_builtin_module_table[] = {
{ MP_ROM_QSTR(MP_QSTR___main__), MP_ROM_PTR(&mp_module___main__) },
{ MP_ROM_QSTR(MP_QSTR_builtins), MP_ROM_PTR(&mp_module_builtins) },
{ MP_ROM_QSTR(MP_QSTR_micropython), MP_ROM_PTR(&mp_module_micropython) },
py: Allow builtins to be overridden. This patch adds a configuration option (MICROPY_CAN_OVERRIDE_BUILTINS) which, when enabled, allows to override all names within the builtins module. A builtins override dict is created the first time the user assigns to a name in the builtins model, and then that dict is searched first on subsequent lookups. Note that this implementation doesn't allow deleting of names. This patch also does some refactoring of builtins code, creating the modbuiltins.c file. Addresses issue #959.
2014-12-09 16:19:48 +00:00
#if MICROPY_PY_IO
py/modio: Rename module name to "uio" for consistency with other modules.
2016-05-02 11:56:33 +01:00
{ MP_ROM_QSTR(MP_QSTR_uio), MP_ROM_PTR(&mp_module_io) },
py: Allow builtins to be overridden. This patch adds a configuration option (MICROPY_CAN_OVERRIDE_BUILTINS) which, when enabled, allows to override all names within the builtins module. A builtins override dict is created the first time the user assigns to a name in the builtins model, and then that dict is searched first on subsequent lookups. Note that this implementation doesn't allow deleting of names. This patch also does some refactoring of builtins code, creating the modbuiltins.c file. Addresses issue #959.
2014-12-09 16:19:48 +00:00
#endif
#if MICROPY_PY_COLLECTIONS
py/modcollections: Rename module name have "u" prefix for consistency.
2016-05-02 11:57:46 +01:00
{ MP_ROM_QSTR(MP_QSTR_ucollections), MP_ROM_PTR(&mp_module_collections) },
py: Allow builtins to be overridden. This patch adds a configuration option (MICROPY_CAN_OVERRIDE_BUILTINS) which, when enabled, allows to override all names within the builtins module. A builtins override dict is created the first time the user assigns to a name in the builtins model, and then that dict is searched first on subsequent lookups. Note that this implementation doesn't allow deleting of names. This patch also does some refactoring of builtins code, creating the modbuiltins.c file. Addresses issue #959.
2014-12-09 16:19:48 +00:00
#endif
#if MICROPY_PY_STRUCT
py: Add MP_ROM_* macros and mp_rom_* types and use them.
2015-11-27 13:38:15 +00:00
{ MP_ROM_QSTR(MP_QSTR_ustruct), MP_ROM_PTR(&mp_module_ustruct) },
py: Allow builtins to be overridden. This patch adds a configuration option (MICROPY_CAN_OVERRIDE_BUILTINS) which, when enabled, allows to override all names within the builtins module. A builtins override dict is created the first time the user assigns to a name in the builtins model, and then that dict is searched first on subsequent lookups. Note that this implementation doesn't allow deleting of names. This patch also does some refactoring of builtins code, creating the modbuiltins.c file. Addresses issue #959.
2014-12-09 16:19:48 +00:00
#endif
#if MICROPY_PY_BUILTINS_FLOAT
#if MICROPY_PY_MATH
py: Add MP_ROM_* macros and mp_rom_* types and use them.
2015-11-27 13:38:15 +00:00
{ MP_ROM_QSTR(MP_QSTR_math), MP_ROM_PTR(&mp_module_math) },
py: Allow builtins to be overridden. This patch adds a configuration option (MICROPY_CAN_OVERRIDE_BUILTINS) which, when enabled, allows to override all names within the builtins module. A builtins override dict is created the first time the user assigns to a name in the builtins model, and then that dict is searched first on subsequent lookups. Note that this implementation doesn't allow deleting of names. This patch also does some refactoring of builtins code, creating the modbuiltins.c file. Addresses issue #959.
2014-12-09 16:19:48 +00:00
#endif
py: Make it easy to build without MICROPY_PY_BUILTINS_COMPLEX. Automagically skip related modules.
2015-12-06 22:18:44 +00:00
#if MICROPY_PY_BUILTINS_COMPLEX && MICROPY_PY_CMATH
py: Add MP_ROM_* macros and mp_rom_* types and use them.
2015-11-27 13:38:15 +00:00
{ MP_ROM_QSTR(MP_QSTR_cmath), MP_ROM_PTR(&mp_module_cmath) },
py: Allow builtins to be overridden. This patch adds a configuration option (MICROPY_CAN_OVERRIDE_BUILTINS) which, when enabled, allows to override all names within the builtins module. A builtins override dict is created the first time the user assigns to a name in the builtins model, and then that dict is searched first on subsequent lookups. Note that this implementation doesn't allow deleting of names. This patch also does some refactoring of builtins code, creating the modbuiltins.c file. Addresses issue #959.
2014-12-09 16:19:48 +00:00
#endif
#endif
#if MICROPY_PY_SYS
all: Rename "sys" module to "usys". This is consistent with the other 'micro' modules and allows implementing additional features in Python via e.g. micropython-lib's sys. Note this is a breaking change (not backwards compatible) for ports which do not enable weak links, as "import sys" must now be replaced with "import usys".
2020-06-18 10:19:14 +01:00
{ MP_ROM_QSTR(MP_QSTR_usys), MP_ROM_PTR(&mp_module_sys) },
py: Allow builtins to be overridden. This patch adds a configuration option (MICROPY_CAN_OVERRIDE_BUILTINS) which, when enabled, allows to override all names within the builtins module. A builtins override dict is created the first time the user assigns to a name in the builtins model, and then that dict is searched first on subsequent lookups. Note that this implementation doesn't allow deleting of names. This patch also does some refactoring of builtins code, creating the modbuiltins.c file. Addresses issue #959.
2014-12-09 16:19:48 +00:00
#endif
#if MICROPY_PY_GC && MICROPY_ENABLE_GC
py: Add MP_ROM_* macros and mp_rom_* types and use them.
2015-11-27 13:38:15 +00:00
{ MP_ROM_QSTR(MP_QSTR_gc), MP_ROM_PTR(&mp_module_gc) },
py: Allow builtins to be overridden. This patch adds a configuration option (MICROPY_CAN_OVERRIDE_BUILTINS) which, when enabled, allows to override all names within the builtins module. A builtins override dict is created the first time the user assigns to a name in the builtins model, and then that dict is searched first on subsequent lookups. Note that this implementation doesn't allow deleting of names. This patch also does some refactoring of builtins code, creating the modbuiltins.c file. Addresses issue #959.
2014-12-09 16:19:48 +00:00
#endif
py: Add basic _thread module, with ability to start a new thread.
2016-04-22 23:52:33 +01:00
#if MICROPY_PY_THREAD
{ MP_ROM_QSTR(MP_QSTR__thread), MP_ROM_PTR(&mp_module_thread) },
#endif
py: Allow builtins to be overridden. This patch adds a configuration option (MICROPY_CAN_OVERRIDE_BUILTINS) which, when enabled, allows to override all names within the builtins module. A builtins override dict is created the first time the user assigns to a name in the builtins model, and then that dict is searched first on subsequent lookups. Note that this implementation doesn't allow deleting of names. This patch also does some refactoring of builtins code, creating the modbuiltins.c file. Addresses issue #959.
2014-12-09 16:19:48 +00:00
// extmod modules
extmod/uasyncio: Add optional implementation of core uasyncio in C. Implements Task and TaskQueue classes in C, using a pairing-heap data structure. Using this reduces RAM use of each Task, and improves overall performance of the uasyncio scheduler.
2020-03-12 05:46:20 +00:00
#if MICROPY_PY_UASYNCIO
{ MP_ROM_QSTR(MP_QSTR__uasyncio), MP_ROM_PTR(&mp_module_uasyncio) },
#endif
py: Add uerrno module, with errno constants and dict.
2016-05-10 10:54:25 +01:00
#if MICROPY_PY_UERRNO
{ MP_ROM_QSTR(MP_QSTR_uerrno), MP_ROM_PTR(&mp_module_uerrno) },
#endif
py: Allow builtins to be overridden. This patch adds a configuration option (MICROPY_CAN_OVERRIDE_BUILTINS) which, when enabled, allows to override all names within the builtins module. A builtins override dict is created the first time the user assigns to a name in the builtins model, and then that dict is searched first on subsequent lookups. Note that this implementation doesn't allow deleting of names. This patch also does some refactoring of builtins code, creating the modbuiltins.c file. Addresses issue #959.
2014-12-09 16:19:48 +00:00
#if MICROPY_PY_UCTYPES
py: Add MP_ROM_* macros and mp_rom_* types and use them.
2015-11-27 13:38:15 +00:00
{ MP_ROM_QSTR(MP_QSTR_uctypes), MP_ROM_PTR(&mp_module_uctypes) },
py: Allow builtins to be overridden. This patch adds a configuration option (MICROPY_CAN_OVERRIDE_BUILTINS) which, when enabled, allows to override all names within the builtins module. A builtins override dict is created the first time the user assigns to a name in the builtins model, and then that dict is searched first on subsequent lookups. Note that this implementation doesn't allow deleting of names. This patch also does some refactoring of builtins code, creating the modbuiltins.c file. Addresses issue #959.
2014-12-09 16:19:48 +00:00
#endif
#if MICROPY_PY_UZLIB
py: Add MP_ROM_* macros and mp_rom_* types and use them.
2015-11-27 13:38:15 +00:00
{ MP_ROM_QSTR(MP_QSTR_uzlib), MP_ROM_PTR(&mp_module_uzlib) },
py: Allow builtins to be overridden. This patch adds a configuration option (MICROPY_CAN_OVERRIDE_BUILTINS) which, when enabled, allows to override all names within the builtins module. A builtins override dict is created the first time the user assigns to a name in the builtins model, and then that dict is searched first on subsequent lookups. Note that this implementation doesn't allow deleting of names. This patch also does some refactoring of builtins code, creating the modbuiltins.c file. Addresses issue #959.
2014-12-09 16:19:48 +00:00
#endif
#if MICROPY_PY_UJSON
py: Add MP_ROM_* macros and mp_rom_* types and use them.
2015-11-27 13:38:15 +00:00
{ MP_ROM_QSTR(MP_QSTR_ujson), MP_ROM_PTR(&mp_module_ujson) },
py: Allow builtins to be overridden. This patch adds a configuration option (MICROPY_CAN_OVERRIDE_BUILTINS) which, when enabled, allows to override all names within the builtins module. A builtins override dict is created the first time the user assigns to a name in the builtins model, and then that dict is searched first on subsequent lookups. Note that this implementation doesn't allow deleting of names. This patch also does some refactoring of builtins code, creating the modbuiltins.c file. Addresses issue #959.
2014-12-09 16:19:48 +00:00
#endif
extmod/moduos: Create general uos module to be used by all ports. Based on the rp2 port version, with the rp2 port converted to use this module. Signed-off-by: Damien George <damien@micropython.org>
2022-03-03 06:59:30 +00:00
#if MICROPY_PY_UOS
{ MP_ROM_QSTR(MP_QSTR_uos), MP_ROM_PTR(&mp_module_uos) },
#endif
py: Allow builtins to be overridden. This patch adds a configuration option (MICROPY_CAN_OVERRIDE_BUILTINS) which, when enabled, allows to override all names within the builtins module. A builtins override dict is created the first time the user assigns to a name in the builtins model, and then that dict is searched first on subsequent lookups. Note that this implementation doesn't allow deleting of names. This patch also does some refactoring of builtins code, creating the modbuiltins.c file. Addresses issue #959.
2014-12-09 16:19:48 +00:00
#if MICROPY_PY_URE
py: Add MP_ROM_* macros and mp_rom_* types and use them.
2015-11-27 13:38:15 +00:00
{ MP_ROM_QSTR(MP_QSTR_ure), MP_ROM_PTR(&mp_module_ure) },
py: Allow builtins to be overridden. This patch adds a configuration option (MICROPY_CAN_OVERRIDE_BUILTINS) which, when enabled, allows to override all names within the builtins module. A builtins override dict is created the first time the user assigns to a name in the builtins model, and then that dict is searched first on subsequent lookups. Note that this implementation doesn't allow deleting of names. This patch also does some refactoring of builtins code, creating the modbuiltins.c file. Addresses issue #959.
2014-12-09 16:19:48 +00:00
#endif
#if MICROPY_PY_UHEAPQ
py: Add MP_ROM_* macros and mp_rom_* types and use them.
2015-11-27 13:38:15 +00:00
{ MP_ROM_QSTR(MP_QSTR_uheapq), MP_ROM_PTR(&mp_module_uheapq) },
py: Allow builtins to be overridden. This patch adds a configuration option (MICROPY_CAN_OVERRIDE_BUILTINS) which, when enabled, allows to override all names within the builtins module. A builtins override dict is created the first time the user assigns to a name in the builtins model, and then that dict is searched first on subsequent lookups. Note that this implementation doesn't allow deleting of names. This patch also does some refactoring of builtins code, creating the modbuiltins.c file. Addresses issue #959.
2014-12-09 16:19:48 +00:00
#endif
extmod/modutimeq: Refactor into optimized class. import utimeq, utime # Max queue size, the queue allocated statically on creation q = utimeq.utimeq(10) q.push(utime.ticks_ms(), data1, data2) res = [0, 0, 0] # Items in res are filled up with results q.pop(res)
2016-12-21 21:29:32 +00:00
#if MICROPY_PY_UTIMEQ
{ MP_ROM_QSTR(MP_QSTR_utimeq), MP_ROM_PTR(&mp_module_utimeq) },
#endif
py: Allow builtins to be overridden. This patch adds a configuration option (MICROPY_CAN_OVERRIDE_BUILTINS) which, when enabled, allows to override all names within the builtins module. A builtins override dict is created the first time the user assigns to a name in the builtins model, and then that dict is searched first on subsequent lookups. Note that this implementation doesn't allow deleting of names. This patch also does some refactoring of builtins code, creating the modbuiltins.c file. Addresses issue #959.
2014-12-09 16:19:48 +00:00
#if MICROPY_PY_UHASHLIB
py: Add MP_ROM_* macros and mp_rom_* types and use them.
2015-11-27 13:38:15 +00:00
{ MP_ROM_QSTR(MP_QSTR_uhashlib), MP_ROM_PTR(&mp_module_uhashlib) },
py: Allow builtins to be overridden. This patch adds a configuration option (MICROPY_CAN_OVERRIDE_BUILTINS) which, when enabled, allows to override all names within the builtins module. A builtins override dict is created the first time the user assigns to a name in the builtins model, and then that dict is searched first on subsequent lookups. Note that this implementation doesn't allow deleting of names. This patch also does some refactoring of builtins code, creating the modbuiltins.c file. Addresses issue #959.
2014-12-09 16:19:48 +00:00
#endif
extmod/moducryptolib: Add ucryptolib module with crypto functions. The API follows guidelines of https://www.python.org/dev/peps/pep-0272/, but is optimized for code size, with the idea that full PEP 0272 compatibility can be added with a simple Python wrapper mode. The naming of the module follows (u)hashlib pattern. At the bare minimum, this module is expected to provide: * AES128, ECB (i.e. "null") mode, encrypt only Implementation in this commit is based on axTLS routines, and implements following: * AES 128 and 256 * ECB and CBC modes * encrypt and decrypt
2018-01-07 13:13:56 +00:00
#if MICROPY_PY_UCRYPTOLIB
{ MP_ROM_QSTR(MP_QSTR_ucryptolib), MP_ROM_PTR(&mp_module_ucryptolib) },
#endif
py: Allow builtins to be overridden. This patch adds a configuration option (MICROPY_CAN_OVERRIDE_BUILTINS) which, when enabled, allows to override all names within the builtins module. A builtins override dict is created the first time the user assigns to a name in the builtins model, and then that dict is searched first on subsequent lookups. Note that this implementation doesn't allow deleting of names. This patch also does some refactoring of builtins code, creating the modbuiltins.c file. Addresses issue #959.
2014-12-09 16:19:48 +00:00
#if MICROPY_PY_UBINASCII
py: Add MP_ROM_* macros and mp_rom_* types and use them.
2015-11-27 13:38:15 +00:00
{ MP_ROM_QSTR(MP_QSTR_ubinascii), MP_ROM_PTR(&mp_module_ubinascii) },
py: Allow builtins to be overridden. This patch adds a configuration option (MICROPY_CAN_OVERRIDE_BUILTINS) which, when enabled, allows to override all names within the builtins module. A builtins override dict is created the first time the user assigns to a name in the builtins model, and then that dict is searched first on subsequent lookups. Note that this implementation doesn't allow deleting of names. This patch also does some refactoring of builtins code, creating the modbuiltins.c file. Addresses issue #959.
2014-12-09 16:19:48 +00:00
#endif
extmod/modurandom: Add "urandom" module. Seedable and reproducible pseudo-random number generator. Implemented functions are getrandbits(n) (n <= 32) and seed(). The algorithm used is Yasmarang by Ilya Levin: http://www.literatecode.com/yasmarang
2016-01-17 10:10:28 +00:00
#if MICROPY_PY_URANDOM
{ MP_ROM_QSTR(MP_QSTR_urandom), MP_ROM_PTR(&mp_module_urandom) },
#endif
stmhal/moduselect: Move to extmod/ for reuse by other ports.
2016-11-20 20:49:45 +00:00
#if MICROPY_PY_USELECT
{ MP_ROM_QSTR(MP_QSTR_uselect), MP_ROM_PTR(&mp_module_uselect) },
#endif
modussl: SSL socket wrapper module based on axTLS.
2015-10-06 16:10:00 +01:00
#if MICROPY_PY_USSL
py: Add MP_ROM_* macros and mp_rom_* types and use them.
2015-11-27 13:38:15 +00:00
{ MP_ROM_QSTR(MP_QSTR_ussl), MP_ROM_PTR(&mp_module_ussl) },
modussl: SSL socket wrapper module based on axTLS.
2015-10-06 16:10:00 +01:00
#endif
extmod/modlwip: slip: Use stream protocol and be port-independent. Based on the original patch by Galen Hazelwood: https://github.com/micropython/micropython/pull/1517 .
2015-10-26 21:04:33 +00:00
#if MICROPY_PY_LWIP
py: Add MP_ROM_* macros and mp_rom_* types and use them.
2015-11-27 13:38:15 +00:00
{ MP_ROM_QSTR(MP_QSTR_lwip), MP_ROM_PTR(&mp_module_lwip) },
extmod/modlwip: slip: Use stream protocol and be port-independent. Based on the original patch by Galen Hazelwood: https://github.com/micropython/micropython/pull/1517 .
2015-10-26 21:04:33 +00:00
#endif
ports: Consolidate inclusion of umachine module in built-ins. The inclusion of `umachine` in the list of built-in modules is now done centrally in py/objmodule.c. Enabling MICROPY_PY_MACHINE will include this module. As part of this, all ports now have `umachine` as the core module name (previously some had only `machine` as the name). Signed-off-by: Damien George <damien@micropython.org>
2022-02-02 05:04:00 +00:00
#if MICROPY_PY_MACHINE
{ MP_ROM_QSTR(MP_QSTR_umachine), MP_ROM_PTR(&mp_module_machine) },
#endif
extmod/moduwebsocket: Refactor `websocket` to `uwebsocket`. As mentioned in #4450, `websocket` was experimental with a single intended user, `webrepl`. Therefore, we'll make this change without a weak link `websocket` -> `uwebsocket`.
2019-02-10 20:35:18 +00:00
#if MICROPY_PY_UWEBSOCKET
{ MP_ROM_QSTR(MP_QSTR_uwebsocket), MP_ROM_PTR(&mp_module_uwebsocket) },
extmod/modwebsocket: Start module for WebSocket helper functions. Currently, only write support is implemented (of limited buffer size).
2016-03-24 17:14:12 +00:00
#endif
extmod/modwebrepl: Module to handle WebREPL protocol. While just a websocket is enough for handling terminal part of WebREPL, handling file transfer operations requires demultiplexing and acting upon, which is encapsulated in _webrepl class provided by this module, which wraps a websocket object.
2016-04-28 22:52:52 +01:00
#if MICROPY_PY_WEBREPL
{ MP_ROM_QSTR(MP_QSTR__webrepl), MP_ROM_PTR(&mp_module_webrepl) },
#endif
extmod: Add initial framebuf module.
2016-04-08 11:08:37 +01:00
#if MICROPY_PY_FRAMEBUF
{ MP_ROM_QSTR(MP_QSTR_framebuf), MP_ROM_PTR(&mp_module_framebuf) },
#endif
py: Support to build berkeley db 1.85 and "btree" module.
2016-06-14 22:52:45 +01:00
#if MICROPY_PY_BTREE
{ MP_ROM_QSTR(MP_QSTR_btree), MP_ROM_PTR(&mp_module_btree) },
#endif
extmod/modbluetooth: Rename module to "ubluetooth". For consistency with "umachine". Now that weak links are enabled by default for built-in modules, this should be a no-op, but allows extension of the bluetooth module by user code. Also move registration of ubluetooth to objmodule rather than port-specific.
2019-10-22 07:03:59 +01:00
#if MICROPY_PY_BLUETOOTH
{ MP_ROM_QSTR(MP_QSTR_ubluetooth), MP_ROM_PTR(&mp_module_ubluetooth) },
#endif
extmod: Add platform module. It contains the compiler version, and underlying system HAL/SDK version.
2021-08-29 17:58:31 +01:00
#if MICROPY_PY_UPLATFORM
{ MP_ROM_QSTR(MP_QSTR_uplatform), MP_ROM_PTR(&mp_module_uplatform) },
#endif
py: Allow builtins to be overridden. This patch adds a configuration option (MICROPY_CAN_OVERRIDE_BUILTINS) which, when enabled, allows to override all names within the builtins module. A builtins override dict is created the first time the user assigns to a name in the builtins model, and then that dict is searched first on subsequent lookups. Note that this implementation doesn't allow deleting of names. This patch also does some refactoring of builtins code, creating the modbuiltins.c file. Addresses issue #959.
2014-12-09 16:19:48 +00:00
// extra builtin modules as defined by a port
MICROPY_PORT_BUILTIN_MODULES
py: Allow registration of modules at their definition. During make, makemoduledefs.py parses the current builds c files for MP_REGISTER_MODULE(module_name, obj_module, enabled_define) These are used to generate a header with the required entries for "mp_rom_map_elem_t mp_builtin_module_table[]" in py/objmodule.c
2019-02-18 03:58:44 +00:00
#ifdef MICROPY_REGISTERED_MODULES
// builtin modules declared with MP_REGISTER_MODULE()
MICROPY_REGISTERED_MODULES
#endif
py: Allow builtins to be overridden. This patch adds a configuration option (MICROPY_CAN_OVERRIDE_BUILTINS) which, when enabled, allows to override all names within the builtins module. A builtins override dict is created the first time the user assigns to a name in the builtins model, and then that dict is searched first on subsequent lookups. Note that this implementation doesn't allow deleting of names. This patch also does some refactoring of builtins code, creating the modbuiltins.c file. Addresses issue #959.
2014-12-09 16:19:48 +00:00
};
py: Move weak-link map to objmodule.c, and expose module maps as public.
2017-01-19 23:21:30 +00:00
MP_DEFINE_CONST_MAP(mp_builtin_module_map, mp_builtin_module_table);
py/builtinimport: Refactor module importing. Simplify and document/comment the handling of builtin import for: - already-loaded modules - built-in modules - built-in umodules (formerly weak links) - filesystem modules Retains existing functionality with smaller code size but should also facilitate potential new features (built-in packages, controlling the frozen path). Also makes the (unix-only) -m behavior a bit more obvious and configurable. Code size change with this commit: bare-arm: +0 +0.000% minimal x86: -64 -0.039% unix x64: -32 -0.006% unix nanbox: -4 -0.001% stm32: -184 -0.047% PYBV10 cc3200: -120 -0.065% esp8266: -228 -0.033% GENERIC esp32: -268 -0.018% GENERIC[incl +16(data)] nrf: -152 -0.087% pca10040 rp2: -256 -0.052% PICO samd: -80 -0.057% ADAFRUIT_ITSYBITSY_M4_EXPRESS
2020-02-26 04:24:09 +00:00
// Tries to find a loaded module, otherwise attempts to load a builtin, otherwise MP_OBJ_NULL.
mp_obj_t mp_module_get_loaded_or_builtin(qstr module_name) {
// First try loaded modules.
mp_map_elem_t *elem = mp_map_lookup(&MP_STATE_VM(mp_loaded_modules_dict).map, MP_OBJ_NEW_QSTR(module_name), MP_MAP_LOOKUP);
Implement ROMable modules. Add math module. mp_module_obj_t can now be put in ROM. Configuration of float type is now similar to longint: can now choose none, float or double as the implementation. math module has basic math functions. For STM port, these are not yet implemented (they are just stub functions).
2014-03-08 15:24:39 +00:00
py/builtinimport: Refactor module importing. Simplify and document/comment the handling of builtin import for: - already-loaded modules - built-in modules - built-in umodules (formerly weak links) - filesystem modules Retains existing functionality with smaller code size but should also facilitate potential new features (built-in packages, controlling the frozen path). Also makes the (unix-only) -m behavior a bit more obvious and configurable. Code size change with this commit: bare-arm: +0 +0.000% minimal x86: -64 -0.039% unix x64: -32 -0.006% unix nanbox: -4 -0.001% stm32: -184 -0.047% PYBV10 cc3200: -120 -0.065% esp8266: -228 -0.033% GENERIC esp32: -268 -0.018% GENERIC[incl +16(data)] nrf: -152 -0.087% pca10040 rp2: -256 -0.052% PICO samd: -80 -0.057% ADAFRUIT_ITSYBITSY_M4_EXPRESS
2020-02-26 04:24:09 +00:00
if (!elem) {
#if MICROPY_MODULE_WEAK_LINKS
return mp_module_get_builtin(module_name);
#else
// Otherwise try builtin.
elem = mp_map_lookup((mp_map_t *)&mp_builtin_module_map, MP_OBJ_NEW_QSTR(module_name), MP_MAP_LOOKUP);
if (!elem) {
py: Make globals and locals proper dictionary objects. Finishes addressing issue #424. In the end this was a very neat refactor that now makes things a lot more consistent across the py code base. It allowed some simplifications in certain places, now that everything is a dict object. Also converted builtins tables to dictionaries. This will be useful when we need to turn builtins into a proper module.
2014-04-05 22:36:42 +01:00
return MP_OBJ_NULL;
}
Implement ROMable modules. Add math module. mp_module_obj_t can now be put in ROM. Configuration of float type is now similar to longint: can now choose none, float or double as the implementation. math module has basic math functions. For STM port, these are not yet implemented (they are just stub functions).
2014-03-08 15:24:39 +00:00
py/builtinimport: Refactor module importing. Simplify and document/comment the handling of builtin import for: - already-loaded modules - built-in modules - built-in umodules (formerly weak links) - filesystem modules Retains existing functionality with smaller code size but should also facilitate potential new features (built-in packages, controlling the frozen path). Also makes the (unix-only) -m behavior a bit more obvious and configurable. Code size change with this commit: bare-arm: +0 +0.000% minimal x86: -64 -0.039% unix x64: -32 -0.006% unix nanbox: -4 -0.001% stm32: -184 -0.047% PYBV10 cc3200: -120 -0.065% esp8266: -228 -0.033% GENERIC esp32: -268 -0.018% GENERIC[incl +16(data)] nrf: -152 -0.087% pca10040 rp2: -256 -0.052% PICO samd: -80 -0.057% ADAFRUIT_ITSYBITSY_M4_EXPRESS
2020-02-26 04:24:09 +00:00
#if MICROPY_MODULE_BUILTIN_INIT
// If found, it's a newly loaded built-in, so init it.
mp_module_call_init(MP_OBJ_NEW_QSTR(module_name), elem->value);
#endif
#endif
}
Add mp_obj_module_register * Add function to load static modules. * Use module_register to pyb module.
2014-03-24 22:25:27 +00:00
py/builtinimport: Refactor module importing. Simplify and document/comment the handling of builtin import for: - already-loaded modules - built-in modules - built-in umodules (formerly weak links) - filesystem modules Retains existing functionality with smaller code size but should also facilitate potential new features (built-in packages, controlling the frozen path). Also makes the (unix-only) -m behavior a bit more obvious and configurable. Code size change with this commit: bare-arm: +0 +0.000% minimal x86: -64 -0.039% unix x64: -32 -0.006% unix nanbox: -4 -0.001% stm32: -184 -0.047% PYBV10 cc3200: -120 -0.065% esp8266: -228 -0.033% GENERIC esp32: -268 -0.018% GENERIC[incl +16(data)] nrf: -152 -0.087% pca10040 rp2: -256 -0.052% PICO samd: -80 -0.057% ADAFRUIT_ITSYBITSY_M4_EXPRESS
2020-02-26 04:24:09 +00:00
return elem->value;
Add module object, to be used eventually for import.
2014-01-02 21:30:26 +00:00
}
py/objmodule: Factor common code for calling __init__ on builtin module.
2018-02-20 06:56:58 +00:00
py: Automatically provide weak links from "foo" to "ufoo" module name. This commit implements automatic module weak links for all built-in modules, by searching for "ufoo" in the built-in module list if "foo" cannot be found. This means that all modules named "ufoo" are always available as "foo". Also, a port can no longer add any other weak links, which makes strict the definition of a weak link. It saves some code size (about 100-200 bytes) on ports that previously had lots of weak links. Some changes from the previous behaviour: - It doesn't intern the non-u module names (eg "foo" is not interned), which saves code size, but will mean that "import foo" creates a new qstr (namely "foo") in RAM (unless the importing module is frozen). - help('modules') no longer lists non-u module names, only the u-variants; this reduces duplication in the help listing. Weak links are effectively the same as having a set of symbolic links on the filesystem that is searched last. So an "import foo" will search built-in modules first, then all paths in sys.path, then weak links last, importing "ufoo" if it exists. Thus a file called "foo.py" somewhere in sys.path will still have precedence over the weak link of "foo" to "ufoo". See issues: #1740, #4449, #5229, #5241.
2019-10-21 15:06:34 +01:00
#if MICROPY_MODULE_WEAK_LINKS
py/builtinimport: Refactor module importing. Simplify and document/comment the handling of builtin import for: - already-loaded modules - built-in modules - built-in umodules (formerly weak links) - filesystem modules Retains existing functionality with smaller code size but should also facilitate potential new features (built-in packages, controlling the frozen path). Also makes the (unix-only) -m behavior a bit more obvious and configurable. Code size change with this commit: bare-arm: +0 +0.000% minimal x86: -64 -0.039% unix x64: -32 -0.006% unix nanbox: -4 -0.001% stm32: -184 -0.047% PYBV10 cc3200: -120 -0.065% esp8266: -228 -0.033% GENERIC esp32: -268 -0.018% GENERIC[incl +16(data)] nrf: -152 -0.087% pca10040 rp2: -256 -0.052% PICO samd: -80 -0.057% ADAFRUIT_ITSYBITSY_M4_EXPRESS
2020-02-26 04:24:09 +00:00
// Tries to find a loaded module, otherwise attempts to load a builtin, otherwise MP_OBJ_NULL.
mp_obj_t mp_module_get_builtin(qstr module_name) {
// Try builtin.
mp_map_elem_t *elem = mp_map_lookup((mp_map_t *)&mp_builtin_module_map, MP_OBJ_NEW_QSTR(module_name), MP_MAP_LOOKUP);
if (!elem) {
return MP_OBJ_NULL;
py: Automatically provide weak links from "foo" to "ufoo" module name. This commit implements automatic module weak links for all built-in modules, by searching for "ufoo" in the built-in module list if "foo" cannot be found. This means that all modules named "ufoo" are always available as "foo". Also, a port can no longer add any other weak links, which makes strict the definition of a weak link. It saves some code size (about 100-200 bytes) on ports that previously had lots of weak links. Some changes from the previous behaviour: - It doesn't intern the non-u module names (eg "foo" is not interned), which saves code size, but will mean that "import foo" creates a new qstr (namely "foo") in RAM (unless the importing module is frozen). - help('modules') no longer lists non-u module names, only the u-variants; this reduces duplication in the help listing. Weak links are effectively the same as having a set of symbolic links on the filesystem that is searched last. So an "import foo" will search built-in modules first, then all paths in sys.path, then weak links last, importing "ufoo" if it exists. Thus a file called "foo.py" somewhere in sys.path will still have precedence over the weak link of "foo" to "ufoo". See issues: #1740, #4449, #5229, #5241.
2019-10-21 15:06:34 +01:00
}
py/builtinimport: Refactor module importing. Simplify and document/comment the handling of builtin import for: - already-loaded modules - built-in modules - built-in umodules (formerly weak links) - filesystem modules Retains existing functionality with smaller code size but should also facilitate potential new features (built-in packages, controlling the frozen path). Also makes the (unix-only) -m behavior a bit more obvious and configurable. Code size change with this commit: bare-arm: +0 +0.000% minimal x86: -64 -0.039% unix x64: -32 -0.006% unix nanbox: -4 -0.001% stm32: -184 -0.047% PYBV10 cc3200: -120 -0.065% esp8266: -228 -0.033% GENERIC esp32: -268 -0.018% GENERIC[incl +16(data)] nrf: -152 -0.087% pca10040 rp2: -256 -0.052% PICO samd: -80 -0.057% ADAFRUIT_ITSYBITSY_M4_EXPRESS
2020-02-26 04:24:09 +00:00
#if MICROPY_MODULE_BUILTIN_INIT
// If found, it's a newly loaded built-in, so init it.
mp_module_call_init(MP_OBJ_NEW_QSTR(module_name), elem->value);
#endif
return elem->value;
py: Automatically provide weak links from "foo" to "ufoo" module name. This commit implements automatic module weak links for all built-in modules, by searching for "ufoo" in the built-in module list if "foo" cannot be found. This means that all modules named "ufoo" are always available as "foo". Also, a port can no longer add any other weak links, which makes strict the definition of a weak link. It saves some code size (about 100-200 bytes) on ports that previously had lots of weak links. Some changes from the previous behaviour: - It doesn't intern the non-u module names (eg "foo" is not interned), which saves code size, but will mean that "import foo" creates a new qstr (namely "foo") in RAM (unless the importing module is frozen). - help('modules') no longer lists non-u module names, only the u-variants; this reduces duplication in the help listing. Weak links are effectively the same as having a set of symbolic links on the filesystem that is searched last. So an "import foo" will search built-in modules first, then all paths in sys.path, then weak links last, importing "ufoo" if it exists. Thus a file called "foo.py" somewhere in sys.path will still have precedence over the weak link of "foo" to "ufoo". See issues: #1740, #4449, #5229, #5241.
2019-10-21 15:06:34 +01:00
}
#endif
py/objmodule: Factor common code for calling __init__ on builtin module.
2018-02-20 06:56:58 +00:00
#if MICROPY_MODULE_BUILTIN_INIT
py/builtinimport: Refactor module importing. Simplify and document/comment the handling of builtin import for: - already-loaded modules - built-in modules - built-in umodules (formerly weak links) - filesystem modules Retains existing functionality with smaller code size but should also facilitate potential new features (built-in packages, controlling the frozen path). Also makes the (unix-only) -m behavior a bit more obvious and configurable. Code size change with this commit: bare-arm: +0 +0.000% minimal x86: -64 -0.039% unix x64: -32 -0.006% unix nanbox: -4 -0.001% stm32: -184 -0.047% PYBV10 cc3200: -120 -0.065% esp8266: -228 -0.033% GENERIC esp32: -268 -0.018% GENERIC[incl +16(data)] nrf: -152 -0.087% pca10040 rp2: -256 -0.052% PICO samd: -80 -0.057% ADAFRUIT_ITSYBITSY_M4_EXPRESS
2020-02-26 04:24:09 +00:00
STATIC void mp_module_register(mp_obj_t module_name, mp_obj_t module) {
mp_map_t *mp_loaded_modules_map = &MP_STATE_VM(mp_loaded_modules_dict).map;
mp_map_lookup(mp_loaded_modules_map, module_name, MP_MAP_LOOKUP_ADD_IF_NOT_FOUND)->value = module;
}
STATIC void mp_module_call_init(mp_obj_t module_name, mp_obj_t module_obj) {
py/objmodule: Factor common code for calling __init__ on builtin module.
2018-02-20 06:56:58 +00:00
// Look for __init__ and call it if it exists
mp_obj_t dest[2];
mp_load_method_maybe(module_obj, MP_QSTR___init__, dest);
if (dest[0] != MP_OBJ_NULL) {
mp_call_method_n_kw(0, 0, dest);
// Register module so __init__ is not called again.
// If a module can be referenced by more than one name (eg due to weak links)
// then __init__ will still be called for each distinct import, and it's then
// up to the particular module to make sure it's __init__ code only runs once.
mp_module_register(module_name, module_obj);
}
}
#endif