-
-
Notifications
You must be signed in to change notification settings - Fork 176
Add Base85 encoding support for Python 3 #251
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
Sweet, this looks perfect except for one little trivial tweak that I'll fixup here before pushing out the merge. I'm going to rename the variables from |
Related-to: jsonpickle#251 Signed-off-by: David Aguilar <davvid@gmail.com>
* base85: api: rename prefer_base85 to use_base85 for consistency with use_decimal Update changelog Add Base85 encoding support for Python 3 Signed-off-by: David Aguilar <davvid@gmail.com>
Hi, Has anyone done any perf mesurement after switching to base85? |
Oh, good point. All of my testing has been on relatively simple stuff. Updatebase85 appears to be slower on Python 3.6 and 3.7 by a factor of about 2.25. 😬 Maybe it should be an opt-in feature. (Base64 is implemented in C, base85 in Python.) |
We haven't tagged a release yet so changing this default is still fair game. |
I looked at the Python 3.6 source code and it appears that there's significant overhead on the first call to b85encode because it builds some tables, but subsequent calls are faster. Still, it's noticeably slower than base64. I'll open a PR to switch base64 to be the default later today. Thanks @cyberic99 for pointing out something I obviously should've done first. Sorry about this! |
Done: #252 |
Thx @dargueta ! For my use case, the thoughput is more important than the file size, so I thought about the performance impact right away ;-) |
TL;DR: Allow using Base85 text encoding on Python 3.
Details
This doesn't remove base64 encoding support for Python 3; rather, it makes base85 the default but still allows overriding the default to only use base64 for backwards compatibility with Python 2. (Python 2 doesn't support base85, nor does there appear to be a backport for it.)
Why
It's a more compact text encoding than Base64, and some manual experimentation gives about a 10% space savings for randomized binary data.
Closes #250.