micropython/ports/esp8266
Maarten van der Schrieck 3bca93b2d0 ports: Fix sys.stdout.buffer.write() return value.
MicroPython code may rely on the return value of sys.stdout.buffer.write()
to reflect the number of bytes actually written. While in most scenarios a
write() operation is successful, there are cases where it fails, leading to
data loss. This problem arises because, currently, write() merely returns
the number of bytes it was supposed to write, without indication of
failure.

One scenario where write() might fail, is where USB is used and the
receiving end doesn't read quickly enough to empty the receive buffer. In
that case, write() on the MicroPython side can timeout, resulting in the
loss of data without any indication, a behavior observed notably in
communication between a Pi Pico as a client and a Linux host using the ACM
driver.

A complex issue arises with mp_hal_stdout_tx_strn() when it involves
multiple outputs, such as USB, dupterm and hardware UART. The challenge is
in handling cases where writing to one output is successful, but another
fails, either fully or partially. This patch implements the following
solution:

mp_hal_stdout_tx_strn() attempts to write len bytes to all of the possible
destinations for that data, and returns the minimum successful write
length.

The implementation of this is complicated by several factors:
- multiple outputs may be enabled or disabled at compiled time
- multiple outputs may be enabled or disabled at runtime
- mp_os_dupterm_tx_strn() is one such output, optionally containing
  multiple additional outputs
- each of these outputs may or may not be able to report success
- each of these outputs may or may not be able to report partial writes

As a result, there's no single strategy that fits all ports, necessitating
unique logic for each instance of mp_hal_stdout_tx_strn().

Note that addressing sys.stdout.write() is more complex due to its data
modification process ("cooked" output), and it remains unchanged in this
patch. Developers who are concerned about accurate return values from
write operations should use sys.stdout.buffer.write().

This patch might disrupt some existing code, but it's also expected to
resolve issues, considering that the peculiar return value behavior of
sys.stdout.buffer.write() is not well-documented and likely not widely
known. Therefore, it's improbable that much existing code relies on the
previous behavior.

Signed-off-by: Maarten van der Schrieck <maarten@thingsconnected.nl>
2023-12-22 10:32:46 +11:00
..
boards
modules
Makefile ports: Switch build to use common lib/libm list of source files. 2023-12-08 15:42:41 +11:00
README.md
esp_init_data.c esp8266: Avoid including ep_mphal.h directly. 2023-12-07 14:07:21 +11:00
esp_mphal.c ports: Fix sys.stdout.buffer.write() return value. 2023-12-22 10:32:46 +11:00
esp_mphal.h extmod/modos: Factor os.dupterm_notify() function to common extmod code. 2023-12-18 12:20:47 +11:00
espapa102.c
espapa102.h
esppwm.c
esppwm.h
ets_alt_task.c
ets_alt_task.h
etshal.h
fatfs_port.c
gccollect.c
gccollect.h
gchelper.s
help.c
hspi.c
hspi.h
hspi_register.h
lexerstr32.c
machine_adc.c
machine_bitstream.c
machine_pin.c
machine_pwm.c
machine_rtc.c
machine_spi.c
machine_uart.c esp8266: Update port to use new event functions. 2023-12-08 12:50:38 +11:00
machine_wdt.c
main.c extmod/modos: Factor os.dupterm_notify() function to common extmod code. 2023-12-18 12:20:47 +11:00
makeimg.py
modesp.c
modespnow.c esp8266: Update port to use new event functions. 2023-12-08 12:50:38 +11:00
modespnow.h
modmachine.c esp8266: Update port to use new event functions. 2023-12-08 12:50:38 +11:00
modmachine.h
modnetwork.h
modnetwork_globals.h
modos.c extmod/modos: Factor os.dupterm_notify() function to common extmod code. 2023-12-18 12:20:47 +11:00
modtime.c
mpconfigport.h esp8266: Update port to use new event functions. 2023-12-08 12:50:38 +11:00
network_wlan.c
posix_helpers.c
qstrdefsport.h
strtoll.c
uart.c esp8266: Update port to use new event functions. 2023-12-08 12:50:38 +11:00
uart.h
uart_register.h
user_config.h
xtirq.h

README.md

MicroPython port to ESP8266

This is a port of MicroPython to the Espressif ESP8266 WiFi microcontroller. MicroPython runs on this chip without any underlying operating system, using the ESP8266 NONOS SDK.

Supported features include:

  • REPL (Python prompt) over UART0.
  • Garbage collector, exceptions.
  • Unicode support.
  • Builtin modules: gc, array, collections, io, struct, sys, esp, network, many more.
  • Arbitrary-precision long integers and 30-bit precision floats.
  • WiFi support.
  • Sockets using modlwip.
  • GPIO and bit-banging I2C, SPI support.
  • 1-Wire and WS2812 (aka Neopixel) protocols support.
  • Internal filesystem using the flash.
  • WebREPL over WiFi from a browser (clients at https://github.com/micropython/webrepl).
  • Modules for HTTP, MQTT, many other formats and protocols via https://github.com/micropython/micropython-lib .

Documentation is available at http://docs.micropython.org/en/latest/esp8266/quickref.html.

The default build requires a 2MiB flash chip, but see below for support for 1MiB and 512kiB options.

Build instructions

You need the esp-open-sdk toolchain, which provides both the compiler and libraries.

There are two ways to do this:

  • By running the toolchain in Docker (recommended).
  • By installing a pre-built toolchain and adding it to your $PATH.

Regardless of which toolchain you use, the first step is to make sure required submodules are available:

$ make -C ports/esp8266 submodules

See the README in the repository root for more information about external dependencies.

Building with Docker

Once you have installed Docker, you can run all of the following build commands inside the Docker container by prefixing them with docker run --rm -v $HOME:$HOME -u $UID -w $PWD larsks/esp-open-sdk ...command.... This will automatically download the Docker image provided by @larsks which contains the full toolchain and SDK.

Then you need to compile the MicroPython cross-compiler (mpy-cross). From the root of this repository, run:

$ docker run --rm -v $HOME:$HOME -u $UID -w $PWD larsks/esp-open-sdk make -C mpy-cross

Note: The mpy-cross binary will likely only work inside the Docker container. This will not be a problem if you're only building ESP8266 firmware, but if you're also working on other ports then you will need to recompile for your host when switching between ports. To avoid this, use the local toolchain instead.

Then to compile the ESP8266 firmware:

$ cd ports/esp8266
$ docker run --rm -v $HOME:$HOME -u $UID -w $PWD larsks/esp-open-sdk make -j BOARD=ESP8266_GENERIC

This will produce binary images in the build-ESP8266_GENERIC/ subdirectory. Substitute the board for whichever board you're using.

Building with a local toolchain

First download the pre-built toolchain (thanks to @jepler from Adafruit). You will need to find somewhere to put it in your filesystem, e.g. ~/espressif. Create that directory first if necessary.

$ cd ~/espressif # Change as necessary
$ wget https://github.com/jepler/esp-open-sdk/releases/download/2018-06-10/xtensa-lx106-elf-standalone.tar.gz
$ tar zxvf xtensa-lx106-elf-standalone.tar.gz
$ rm xtensa-lx106-elf/bin/esptool.py  # Use system version of esptool.py instead.

Then append this to your $PATH variable so the compiler binaries can be found:

$ export "PATH=$HOME/espressif/xtensa-lx106-elf/bin/:$PATH"

(You will need to do this each time you start a new terminal)

Then you need to compile the MicroPython cross-compiler (mpy-cross). From the root of this repository, run:

$ make -C mpy-cross

Then to compile the ESP8266 firmware:

$ cd ports/esp8266
$ make -j BOARD=ESP8266_GENERIC

This will produce binary images in the build-ESP8266_GENERIC/ subdirectory. Substitute the board for whichever board you're using.

Installing MicroPython

To communicate with the board you will need to install esptool.py. This can be obtained from your system package manager or from PyPi via pip.

If you install MicroPython to your module for the first time, or after installing any other firmware, you should erase flash completely:

$ esptool.py --port /dev/ttyXXX erase_flash

Erasing the flash is also useful as a troubleshooting measure, if a module doesn't behave as expected.

To flash MicroPython image to your ESP8266, use:

$ make deploy

(If using the Docker instructions above, do not run this command via Docker as it will need access to the serial port. Run it directly instead.)

This will use the esptool.py script to download the images. You must have your ESP module in the bootloader mode, and connected to a serial port on your PC. The default serial port is /dev/ttyACM0, flash mode is qio and flash size is detect (auto-detect based on Flash ID).

To specify other values for esptool.py, use, e.g.:

$ make PORT=/dev/ttyUSB0 FLASH_MODE=qio FLASH_SIZE=32m deploy

(note that flash size is in megabits)

If you want to flash manually using esptool.py directly, the image produced is build-ESP8266_GENERIC/firmware.bin, to be flashed at 0x00000.

The default board definition is the directory boards/ESP8266_GENERIC. For a custom configuration you can define your own board in the directory boards/.

Reduced FlashROM variants

The normal build described above requires modules with at least 2MiB of FlashROM onboard. There's a special configuration for 512kiB modules, which can be built with the FLASH_512K variant. This configuration is highly limited, lacks filesystem support, WebREPL, and has many other features disabled. It's mostly suitable for advanced users who are interested to fine-tune options to achieve a required setup. If you are an end user, please consider using a module with at least 2MiB of FlashROM.

A variant is also provided for 1MiB modules which just lacks the included micropython-lib packages.

The variant can be set on the make command line, for example:

$ make BOARD=ESP8266_GENERIC BOARD_VARIANT=FLASH_512K
$ make BOARD=ESP8266_GENERIC BOARD_VARIANT=FLASH_1M

First start

Be sure to change ESP8266's WiFi access point password ASAP, see below.

Serial prompt

You can access the REPL (Python prompt) over UART (the same as used for programming).

  • Baudrate: 115200

Run help() for some basic information.

WiFi

Initially, the device configures itself as a WiFi access point (AP).

  • ESSID: MicroPython-xxxxxx (xs are replaced with part of the MAC address).
  • Password: micropythoN (note the upper-case N).
  • IP address of the board: 192.168.4.1.
  • DHCP-server is activated.
  • Please be sure to change the password to something non-guessable immediately. help() gives information how.

WebREPL

Python prompt over WiFi, connecting through a browser.

mip

The ESP8266 port comes with the built-in mip package manager, which can be used to install additional modules:

>>> import mip
>>> mip.install("hmac")
[...]
>>> import hmac
>>> hmac.new(b"1234567890", msg="hello world").hexdigest()

See Package management for more information about mip.

Downloading and installing packages may requite a lot of free memory, if you get an error, retry immediately after the hard reset.

Documentation

More detailed documentation and instructions can be found at http://docs.micropython.org/en/latest/esp8266/ , which includes Quick Reference, Tutorial, General Information related to ESP8266 port, and to MicroPython in general.

Troubleshooting

While the port is in beta, it's known to be generally stable. If you experience strange bootloops, crashes, lockups, here's a list to check against:

  • You didn't erase flash before programming MicroPython firmware.
  • Firmware can be occasionally flashed incorrectly. Just retry. Recent esptool.py versions have --verify option.
  • Power supply you use doesn't provide enough power for ESP8266 or isn't stable enough.
  • A module/flash may be defective (not unheard of for cheap modules).

Please consult dedicated ESP8266 forums/resources for hardware-related problems.

Additional information may be available by the documentation links above.