What's the difference between this and the msgpack-python?

[lxyu]

Another repo here: https://github.com/msgpack/msgpack-python

While the official site recommend this u-msgpack-python, I'm a little confused.

Both implemented in pure python and both support the latest msgpack 2.0 and both support python2/3.

So what's the difference?

[vsergeev]

Hi Lx,

I released u-msgpack-python to support msgpack 2.0 back when msgpack-python only supported the old specification (see msgpack/msgpack#154). Recently, msgpack-python added support for msgpack 2.0 as well. To answer your question:

msgpack-python should be faster when used with its native library backend (it only falls back to pure Python when the library is unavailable). msgpack-python also supports a streaming unpacking mode and finer control over the unpacking process if you need them.

u-msgpack-python has the same top-level interface as msgpack-python (packb() and unpackb()), but less tuning options and modes, and u-msgpack-python is distributed as a single file to make it easy to drop into projects (e.g. embedded Linux systems, for which it is difficult to cross-compile native Python modules for). The u- prefix is meant in the "micro" connotation, as in lightweight and easy to integrate.

There may be other differences in exception handling and how Python types are mapped to and from msgpack types. In particular, u-msgpack-python is fairly strict about unicode strings and byte strings mapping between Python and msgpack. See the main README of u-msgpack-python for documentation on this.

To my best understanding, msgpack implementations are developed ad hoc. As for the website, the list is auto-generated from Github project descriptions (see https://github.com/msgpack/website). I'm not sure why msgpack-python is not listed on there -- it always used to be. I think the author took it down temporarily to implement msgpack 2.0, but I would expect it to have been back up by now.

Thanks,
~vsergeev
Ivan Sergeev

[lxyu]

Thank you very much!

[stinos]

Some extra information: there is a performance difference between u-msgpack-python and the pure Python implementation of msgpack-python (the one in it's fallback.py) so if you have to make a choice it might be worth checking if it matters for you.

(I didn't want to make a separate issue for this because it's not really an 'issue', I think, and this turns up in search engines so seems worth adding info here).

For the type of objects we use, something like {'a': 35, 'b': [1.0, 2.0, 3] * 20, (1): list(range(100)), 'longerKey': 'sab' * 100} msgpack-python seems to be slightly faster than u-msgpack-python when tested on Windows with CPython 3.7, and somehow twice as fast when tested with MicroPython. However with an object like list(range(1000)) the winner is u-msgpack-python on CPython (but still about twice as slow on MicroPython).

So if you need a Python implementation and know what objects you're going to use (and they work with both implementations) and performance matters then you probably want to measure first.