Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Pure Python (2.*) fallback #42

Merged
merged 13 commits into from Jan 29, 2013
Merged

Pure Python (2.*) fallback #42

merged 13 commits into from Jan 29, 2013

Conversation

@bwesterb
Copy link
Contributor

@bwesterb bwesterb commented Jan 28, 2013

There are several cases where the Cython module cannot be used.

  • When using PyPy.
  • On Google App Engine or another restricted environment.

For those cases it is useful to have a pure python implementation of msgpack.

I could release this as a separate project, but I prefer it merged upstream: it will be easier to keep the API the same.

There is another project which implements msgpack in pure Python, ( keisukefukuda/msgpack-python-pure ) but that is not a drop-in replacement.

Notes

  • Setup will automatically skip the module if it fails to build. Maybe this is not desired: it could be moved behind a commandline flag.
  • msgpack will first try to import the native module. If none is available, it will use the fallback.
  • The fallback module passes all current tests (on Python 2.*)
  • The fallback module does not yet work on Python 3. (That is why Travis complains.) I will look into that.
  • The fallback module has been written with compatibility as first priority. There is a lot opportunity to improve performance.

A benchmark

As a very rough benchmark, I timed the pack and unpack time of a 30MB msgpack file from one of my applications.

  1. Cython module 0.34; 0.24
  2. Python 15.4; 10.77
  3. PyPy 1.9 1.37; 0.43
  4. PyPy dev-version 1.26; 0.40
bwesterb added 4 commits Jan 28, 2013
Signed-off-by: Bas Westerbaan <bas@westerbaan.name>
Signed-off-by: Bas Westerbaan <bas@westerbaan.name>
This increases performance *a lot* on PyPy.

Signed-off-by: Bas Westerbaan <bas@westerbaan.name>
@methane
Copy link
Member

@methane methane commented Jan 28, 2013

Very thanks to this PR!
I'll check, merge and release this in this week.

bwesterb added 9 commits Jan 28, 2013
Increases performance on PyPy.
Signed-off-by: Bas Westerbaan <bas@westerbaan.name>
Signed-off-by: Bas Westerbaan <bas@westerbaan.name>
Signed-off-by: Bas Westerbaan <bas@westerbaan.name>
@ghost ghost assigned methane Jan 29, 2013
@methane methane merged commit 4cde7f0 into msgpack:master Jan 29, 2013
1 check passed
1 check passed
default The Travis build passed
Details
@bwesterb
Copy link
Contributor Author

@bwesterb bwesterb commented Jan 29, 2013

Thanks for the improvements!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked issues

Successfully merging this pull request may close these issues.

None yet

2 participants
You can’t perform that action at this time.