225 lines
8.8 KiB
ReStructuredText
225 lines
8.8 KiB
ReStructuredText
.. _natmod:
|
|
|
|
Native machine code in .mpy files
|
|
=================================
|
|
|
|
This section describes how to build and work with .mpy files that contain native
|
|
machine code from a language other than Python. This allows you to
|
|
write code in a language like C, compile and link it into a .mpy file, and then
|
|
import this file like a normal Python module. This can be used for implementing
|
|
functionality which is performance critical, or for including an existing
|
|
library written in another language.
|
|
|
|
One of the main advantages of using native .mpy files is that native machine code
|
|
can be imported by a script dynamically, without the need to rebuild the main
|
|
MicroPython firmware. This is in contrast to :ref:`cmodules` which also allows
|
|
defining custom modules in C but they must be compiled into the main firmware image.
|
|
|
|
The focus here is on using C to build native modules, but in principle any
|
|
language which can be compiled to stand-alone machine code can be put into a
|
|
.mpy file.
|
|
|
|
A native .mpy module is built using the ``mpy_ld.py`` tool, which is found in the
|
|
``tools/`` directory of the project. This tool takes a set of object files
|
|
(.o files) and links them together to create a native .mpy files. It requires
|
|
CPython 3 and the library pyelftools v0.25 or greater.
|
|
|
|
Supported features and limitations
|
|
----------------------------------
|
|
|
|
A .mpy file can contain MicroPython bytecode and/or native machine code. If it
|
|
contains native machine code then the .mpy file has a specific architecture
|
|
associated with it. Current supported architectures are (these are the valid
|
|
options for the ``ARCH`` variable, see below):
|
|
|
|
* ``x86`` (32 bit)
|
|
* ``x64`` (64 bit x86)
|
|
* ``armv7m`` (ARM Thumb 2, eg Cortex-M3)
|
|
* ``armv7emsp`` (ARM Thumb 2, single precision float, eg Cortex-M4F, Cortex-M7)
|
|
* ``armv7emdp`` (ARM Thumb 2, double precision float, eg Cortex-M7)
|
|
* ``xtensa`` (non-windowed, eg ESP8266)
|
|
* ``xtensawin`` (windowed with window size 8, eg ESP32)
|
|
|
|
When compiling and linking the native .mpy file the architecture must be chosen
|
|
and the corresponding file can only be imported on that architecture. For more
|
|
details about .mpy files see :ref:`mpy_files`.
|
|
|
|
Native code must be compiled as position independent code (PIC) and use a global
|
|
offset table (GOT), although the details of this varies from architecture to
|
|
architecture. When importing .mpy files with native code the import machinery
|
|
is able to do some basic relocation of the native code. This includes
|
|
relocating text, rodata and BSS sections.
|
|
|
|
Supported features of the linker and dynamic loader are:
|
|
|
|
* executable code (text)
|
|
* read-only data (rodata), including strings and constant data (arrays, structs, etc)
|
|
* zeroed data (BSS)
|
|
* pointers in text to text, rodata and BSS
|
|
* pointers in rodata to text, rodata and BSS
|
|
|
|
The known limitations are:
|
|
|
|
* data sections are not supported; workaround: use BSS data and initialise the
|
|
data values explicitly
|
|
|
|
* static BSS variables are not supported; workaround: use global BSS variables
|
|
|
|
So, if your C code has writable data, make sure the data is defined globally,
|
|
without an initialiser, and only written to within functions.
|
|
|
|
Linker limitation: the native module is not linked against the symbol table of the
|
|
full MicroPython firmware. Rather, it is linked against an explicit table of exported
|
|
symbols found in ``mp_fun_table`` (in ``py/nativeglue.h``), that is fixed at firmware
|
|
build time. It is thus not possible to simply call some arbitrary HAL/OS/RTOS/system
|
|
function, for example.
|
|
|
|
New symbols can be added to the end of the table and the firmware rebuilt.
|
|
The symbols also need to be added to ``tools/mpy_ld.py``'s ``fun_table`` dict in the
|
|
same location. This allows ``mpy_ld.py`` to be able to pick the new symbols up and
|
|
provide relocations for them when the mpy is imported. Finally, if the symbol is a
|
|
function, a macro or stub should be added to ``py/dynruntime.h`` to make it easy to
|
|
call the function.
|
|
|
|
Defining a native module
|
|
------------------------
|
|
|
|
A native .mpy module is defined by a set of files that are used to build the .mpy.
|
|
The filesystem layout consists of two main parts, the source files and the Makefile:
|
|
|
|
* In the simplest case only a single C source file is required, which contains all
|
|
the code that will be compiled into the .mpy module. This C source code must
|
|
include the ``py/dynruntime.h`` file to access the MicroPython dynamic API, and
|
|
must at least define a function called ``mpy_init``. This function will be the
|
|
entry point of the module, called when the module is imported.
|
|
|
|
The module can be split into multiple C source files if desired. Parts of the
|
|
module can also be implemented in Python. All source files should be listed in
|
|
the Makefile, by adding them to the ``SRC`` variable (see below). This includes
|
|
both C source files as well as any Python files which will be included in the
|
|
resulting .mpy file.
|
|
|
|
* The ``Makefile`` contains the build configuration for the module and list the
|
|
source files used to build the .mpy module. It should define ``MPY_DIR`` as the
|
|
location of the MicroPython repository (to find header files, the relevant Makefile
|
|
fragment, and the ``mpy_ld.py`` tool), ``MOD`` as the name of the module, ``SRC``
|
|
as the list of source files, optionally specify the machine architecture via ``ARCH``,
|
|
and then include ``py/dynruntime.mk``.
|
|
|
|
Minimal example
|
|
---------------
|
|
|
|
This section provides a fully working example of a simple module named ``factorial``.
|
|
This module provides a single function ``factorial.factorial(x)`` which computes the
|
|
factorial of the input and returns the result.
|
|
|
|
Directory layout::
|
|
|
|
factorial/
|
|
├── factorial.c
|
|
└── Makefile
|
|
|
|
The file ``factorial.c`` contains:
|
|
|
|
.. code-block:: c
|
|
|
|
// Include the header file to get access to the MicroPython API
|
|
#include "py/dynruntime.h"
|
|
|
|
// Helper function to compute factorial
|
|
STATIC mp_int_t factorial_helper(mp_int_t x) {
|
|
if (x == 0) {
|
|
return 1;
|
|
}
|
|
return x * factorial_helper(x - 1);
|
|
}
|
|
|
|
// This is the function which will be called from Python, as factorial(x)
|
|
STATIC mp_obj_t factorial(mp_obj_t x_obj) {
|
|
// Extract the integer from the MicroPython input object
|
|
mp_int_t x = mp_obj_get_int(x_obj);
|
|
// Calculate the factorial
|
|
mp_int_t result = factorial_helper(x);
|
|
// Convert the result to a MicroPython integer object and return it
|
|
return mp_obj_new_int(result);
|
|
}
|
|
// Define a Python reference to the function above
|
|
STATIC MP_DEFINE_CONST_FUN_OBJ_1(factorial_obj, factorial);
|
|
|
|
// This is the entry point and is called when the module is imported
|
|
mp_obj_t mpy_init(mp_obj_fun_bc_t *self, size_t n_args, size_t n_kw, mp_obj_t *args) {
|
|
// This must be first, it sets up the globals dict and other things
|
|
MP_DYNRUNTIME_INIT_ENTRY
|
|
|
|
// Make the function available in the module's namespace
|
|
mp_store_global(MP_QSTR_factorial, MP_OBJ_FROM_PTR(&factorial_obj));
|
|
|
|
// This must be last, it restores the globals dict
|
|
MP_DYNRUNTIME_INIT_EXIT
|
|
}
|
|
|
|
The file ``Makefile`` contains:
|
|
|
|
.. code-block:: make
|
|
|
|
# Location of top-level MicroPython directory
|
|
MPY_DIR = ../../..
|
|
|
|
# Name of module
|
|
MOD = factorial
|
|
|
|
# Source files (.c or .py)
|
|
SRC = factorial.c
|
|
|
|
# Architecture to build for (x86, x64, armv7m, xtensa, xtensawin)
|
|
ARCH = x64
|
|
|
|
# Include to get the rules for compiling and linking the module
|
|
include $(MPY_DIR)/py/dynruntime.mk
|
|
|
|
Compiling the module
|
|
--------------------
|
|
|
|
The prerequisite tools needed to build a native .mpy file are:
|
|
|
|
* The MicroPython repository (at least the ``py/`` and ``tools/`` directories).
|
|
* CPython 3, and the library pyelftools (eg ``pip install 'pyelftools>=0.25'``).
|
|
* GNU make.
|
|
* A C compiler for the target architecture (if C source is used).
|
|
* Optionally ``mpy-cross``, built from the MicroPython repository (if .py source is used).
|
|
|
|
Be sure to select the correct ``ARCH`` for the target you are going to run on.
|
|
Then build with::
|
|
|
|
$ make
|
|
|
|
Without modifying the Makefile you can specify the target architecture via::
|
|
|
|
$ make ARCH=armv7m
|
|
|
|
Module usage in MicroPython
|
|
---------------------------
|
|
|
|
Once the module is built there should be a file called ``factorial.mpy``. Copy
|
|
this so it is accessible on the filesystem of your MicroPython system and can be
|
|
found in the import path. The module can now be accessed in Python just like any
|
|
other module, for example::
|
|
|
|
import factorial
|
|
print(factorial.factorial(10))
|
|
# should display 3628800
|
|
|
|
Further examples
|
|
----------------
|
|
|
|
See ``examples/natmod/`` for further examples which show many of the available
|
|
features of native .mpy modules. Such features include:
|
|
|
|
* using multiple C source files
|
|
* including Python code alongside C code
|
|
* rodata and BSS data
|
|
* memory allocation
|
|
* use of floating point
|
|
* exception handling
|
|
* including external C libraries
|