New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Pure Python (2.*) fallback #42

Merged
merged 13 commits into from Jan 29, 2013

Conversation

Projects
None yet
2 participants
@bwesterb
Contributor

bwesterb commented Jan 28, 2013

There are several cases where the Cython module cannot be used.

  • When using PyPy.
  • On Google App Engine or another restricted environment.

For those cases it is useful to have a pure python implementation of msgpack.

I could release this as a separate project, but I prefer it merged upstream: it will be easier to keep the API the same.

There is another project which implements msgpack in pure Python, ( keisukefukuda/msgpack-python-pure ) but that is not a drop-in replacement.

Notes

  • Setup will automatically skip the module if it fails to build. Maybe this is not desired: it could be moved behind a commandline flag.
  • msgpack will first try to import the native module. If none is available, it will use the fallback.
  • The fallback module passes all current tests (on Python 2.*)
  • The fallback module does not yet work on Python 3. (That is why Travis complains.) I will look into that.
  • The fallback module has been written with compatibility as first priority. There is a lot opportunity to improve performance.

A benchmark

As a very rough benchmark, I timed the pack and unpack time of a 30MB msgpack file from one of my applications.

  1. Cython module 0.34; 0.24
  2. Python 15.4; 10.77
  3. PyPy 1.9 1.37; 0.43
  4. PyPy dev-version 1.26; 0.40

bwesterb added some commits Jan 28, 2013

Add pure Python fallback module
Signed-off-by: Bas Westerbaan <bas@westerbaan.name>
setup: automatically fallback to pure Python module
Signed-off-by: Bas Westerbaan <bas@westerbaan.name>
fallback: use __pypy__.builders.StringBuilder when available
This increases performance *a lot* on PyPy.

Signed-off-by: Bas Westerbaan <bas@westerbaan.name>
@methane

This comment has been minimized.

Show comment
Hide comment
@methane

methane Jan 28, 2013

Member

Very thanks to this PR!
I'll check, merge and release this in this week.

Member

methane commented Jan 28, 2013

Very thanks to this PR!
I'll check, merge and release this in this week.

@ghost ghost assigned methane Jan 29, 2013

@methane methane merged commit 4cde7f0 into msgpack:master Jan 29, 2013

1 check passed

default The Travis build passed
Details
@bwesterb

This comment has been minimized.

Show comment
Hide comment
@bwesterb

bwesterb Jan 29, 2013

Contributor

Thanks for the improvements!

Contributor

bwesterb commented Jan 29, 2013

Thanks for the improvements!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment