micropython/lib
Damien George c4feb806e0 lib/uzlib: Add memory-efficient, streaming LZ77 compression support.
The compression algorithm implemented in this commit uses much less memory
compared to the standard way of implementing it using a hash table and
large look-back window.  In particular the algorithm here doesn't allocate
hash table to store indices into the history of the previously seen text.
Instead it simply does a brute-force-search of the history text to find a
match for the compressor.  This is slower (linear search vs hash table
lookup) but with a small enough history (eg 512 bytes) it's not that slow.
And a small history does not impact the compression too much.

To give some more concrete numbers comparing memory use between the
approaches:

- Standard approach: inplace compression, all text to compress must be in
  RAM (or at least memory addressable), and then an additional 16k bytes
  RAM of hash table pointers, pointing into the text

- The approach in this commit: streaming compression, only a limited amount
  of previous text must be in RAM (user selectable, defaults to 512 bytes).

To compress, say, 1k of data, the standard approach requires all that data
to be in RAM, plus an additional 16k of RAM for the hash table pointers.
With this commit, you only need the 1k of data in RAM.  Or if it's
streaming from a file (or elsewhere), you could get away with only 256
bytes of RAM for the sliding history and still get very decent compression.

In summary: because compression takes such a large amount of RAM (in the
standard algorithm) and it's not really suitable for microcontrollers, the
approach taken in this commit is to minimise RAM usage as much as possible,
and still have acceptable performance (speed and compression ratio).

Signed-off-by: Damien George <damien@micropython.org>
2023-07-21 18:54:22 +10:00
..
asf4@84f56af132 lib/asf4: Point submodule to latest commit on circuitpython branch. 2021-11-18 12:44:30 +11:00
axtls@531cab9c27 lib/axtls: Update to latest axtls 2.1.5 wih additional commits. 2021-07-08 23:51:39 +10:00
berkeley-db-1.xx@35aaec4418 berkeley-db-1.xx: Update, allow to override MINCACHE, DEFPSIZE. 2017-09-10 13:51:51 +03:00
btstack@77e752abd6 lib/btstack: Update to v1.5.6.2. 2023-06-14 19:23:27 +10:00
cmsis/inc lib/cmsis: Upgrade to CMSIS 5.5.1. 2019-06-03 14:40:57 +10:00
crypto-algorithms lib/crypto-algorithms: Move crypto-algorithms code from extmod to lib. 2021-07-12 16:36:37 +10:00
cyw43-driver@8ef38a6d32 lib/cyw43-driver: Update driver to latest version v1.0.1. 2023-06-14 19:23:27 +10:00
fsp@e78939d32d lib/fsp: Update FSP for renesas-ra to the latest version v4.4.0. 2023-05-11 16:18:39 +10:00
libffi@e9de7e35f2 lib/libffi: Add libffi as a submodule. 2015-10-17 15:52:35 +03:00
libhydrogen@5c5d513093 lib/libhydrogen: Add new libhydrogen submodule. 2021-01-18 12:43:01 +11:00
libm lib/libm: Use __asm__ instead of asm. 2022-11-10 11:36:41 +11:00
libm_dbl lib/libm: Use __asm__ instead of asm. 2022-11-10 11:36:41 +11:00
littlefs lib/littlefs: Remove assignment of variables to themselves. 2022-05-24 12:51:49 +10:00
lwip@6ca936f6b5 lib/lwip: Update lwIP to v2.1.3, tag STABLE-2_1_3_RELEASE. 2022-08-29 14:41:21 +10:00
mbedtls@981743de6f lib/mbedtls: Update to mbedtls v2.28.3. 2023-05-03 17:33:38 +10:00
mbedtls_errors lib/mbedtls_errors: Update patch and error list for new mbedtls. 2023-05-01 10:54:02 +10:00
micropython-lib@c113611765 lib/micropython-lib: Update submodule to latest. 2023-04-26 12:59:26 +10:00
mynewt-nimble@42849560ba extmod/nimble: Update to NimBLE v1.4. 2021-10-26 16:43:18 +11:00
nrfx@7a4c9d946c lib/nrfx: Upgrade to nrfx v2.0.0. 2020-05-15 15:06:02 +10:00
nxp_driver@fa5a554c79 lib/nxp_driver: Update nxp_driver to v2.10. 2022-06-03 10:48:04 +10:00
oofatfs lib/oofatfs: Fix speculative read in create_name. 2023-06-21 09:47:44 +01:00
pico-sdk@6a7db34ff6 lib/pico-sdk: Update to version 1.5.1. 2023-06-14 22:20:20 +10:00
re1.5 lib/re1.5: Add support for named classes in class sets. 2023-01-18 09:15:32 +11:00
stm32lib@928df866e4 lib/stm32lib: Update library for H5 v1.0.0. 2023-06-15 11:02:56 +10:00
tinytest Revert "lib/tinytest: Clean up test reporting in the presence of std..." 2019-12-19 17:53:46 +11:00
tinyusb@868f2bcda0 lib/tinyusb: Update to the most recent master. 2022-07-19 11:26:54 +10:00
uzlib lib/uzlib: Add memory-efficient, streaming LZ77 compression support. 2023-07-21 18:54:22 +10:00
wiznet5k@0803fc519a lib/wiznet5k: Add submodule for Wiznet Ethernet drivers. 2022-06-03 14:29:11 +10:00
README.md lib,shared: Update README's based on contents of these dirs. 2021-07-13 00:17:02 +10:00

README.md

This directory contains third-party, low-level C libraries and SDKs. Libraries that do not target any specific platform are generally chosen based on them being independent and efficient.