Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Pure Python (2.*) fallback #42

Merged
merged 13 commits into from Jan 29, 2013
Merged

Pure Python (2.*) fallback #42

merged 13 commits into from Jan 29, 2013

Conversation

bwesterb
Copy link
Contributor

There are several cases where the Cython module cannot be used.

  • When using PyPy.
  • On Google App Engine or another restricted environment.

For those cases it is useful to have a pure python implementation of msgpack.

I could release this as a separate project, but I prefer it merged upstream: it will be easier to keep the API the same.

There is another project which implements msgpack in pure Python, ( keisukefukuda/msgpack-python-pure ) but that is not a drop-in replacement.

Notes

  • Setup will automatically skip the module if it fails to build. Maybe this is not desired: it could be moved behind a commandline flag.
  • msgpack will first try to import the native module. If none is available, it will use the fallback.
  • The fallback module passes all current tests (on Python 2.*)
  • The fallback module does not yet work on Python 3. (That is why Travis complains.) I will look into that.
  • The fallback module has been written with compatibility as first priority. There is a lot opportunity to improve performance.

A benchmark

As a very rough benchmark, I timed the pack and unpack time of a 30MB msgpack file from one of my applications.

  1. Cython module 0.34; 0.24
  2. Python 15.4; 10.77
  3. PyPy 1.9 1.37; 0.43
  4. PyPy dev-version 1.26; 0.40

Signed-off-by: Bas Westerbaan <bas@westerbaan.name>
Signed-off-by: Bas Westerbaan <bas@westerbaan.name>
This increases performance *a lot* on PyPy.

Signed-off-by: Bas Westerbaan <bas@westerbaan.name>
@methane
Copy link
Member

methane commented Jan 28, 2013

Very thanks to this PR!
I'll check, merge and release this in this week.

@ghost ghost assigned methane Jan 29, 2013
@methane methane merged commit 4cde7f0 into msgpack:master Jan 29, 2013
@bwesterb
Copy link
Contributor Author

Thanks for the improvements!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants