msgpackc: MessagePack in C with umsgpack's API, 20-120x faster on ESP32

[Tangerino]

I've published msgpackc, a MessagePack serializer written in C as a user module for MicroPython v1.29: https://github.com/Tangerino/micropython-msgpackc

It is a drop-in replacement for Peter Hinch's excellent umsgpack. It has the same functions, options, exceptions and @ext_serializable extension types, and produces byte-identical output. Existing code only needs import msgpackc as umsgpack.

Packing and unpacking a list of 20 telemetry records:

	dumps	loads
ESP32, internal RAM	122x faster	25x faster
ESP32, PSRAM	82x faster	20x faster
ESP32-P4	76x faster	22x faster

It also allocates much less: 1 KiB instead of 60 KiB per dumps in that test.

Also included:

• Unpacker for incremental decoding (UART, sockets).
• ALoader for asyncio.
• Safe on hostile input: lengths are checked before allocating, and nesting is limited.
• About 12 KiB of flash.

Tests:

• Peter's test suite.
• A differential test against umsgpack.
• The cross-language msgpack-test-suite (318/318).
• Fuzzing under ASan/UBSan.
• Real ESP32 and ESP32-P4 boards.

Ready-made firmware for ESP32 (with and without PSRAM), S3, C3, C6 and P4 is attached to the release, so you can try it without building anything: https://github.com/Tangerino/micropython-msgpackc/releases/latest

Feedback, bug reports and results on other boards are very welcome.

msgpackc was developed with an AI coding agent (Claude Code) working at my request.

[peterhinch]

It's perhaps worth pointing out the purpose and some of the benefits of MessagePack.

It's an encoding/decoding system like JSON. It's as easy to use as JSON but provides data compression. Whether you're writing to a file or sending data to a remote receiver there is a substantial reduction in data volume. A wide range of Python data types are natively supported, but it's easy to add further types to the protocol. It's widely supported so can be used for communication with a variety of non-MP platforms.