New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Optimized alternative to urllib.quote #134

Closed
kgriffs opened this Issue May 13, 2013 · 6 comments

Comments

Projects
None yet
3 participants
@kgriffs
Contributor

kgriffs commented May 13, 2013

See if we can do better than ~2 us (and ~4 us on Py33)

@queertypes

This comment has been minimized.

Contributor

queertypes commented Sep 11, 2013

Here's some benchmarks: http://paste.openstack.org/show/46739/

@queertypes

This comment has been minimized.

Contributor

queertypes commented Sep 11, 2013

Things are especially slow in python 3, but are reasonably snappy under pypy. The first thing that comes to mind here is calling out to cffi using a C module for quoting and unquoting URLs.

I found python-percentcoding on PyPI, but it doesn't work on Python 3 and it might not work on pypy, either.

@kgriffs

This comment has been minimized.

Contributor

kgriffs commented Sep 13, 2013

Thanks for taking a look! Good data points.

@syllog1sm

This comment has been minimized.

syllog1sm commented Sep 22, 2013

I remember looking at the urllib.quote function a long time ago, because I hit the unicode bug in it.

Check out the implementation: https://gist.github.com/syllog1sm/6656171

It's easy to see that this could be made more efficient, just by reimplementing it in a more straight-forward way.

Looking at the falcon source, it seems this is only used in utils.percent_escape, right? So the set of reserved characters is fixed. That makes things much simpler.

If the simple Python way isn't good enough, a Cython array would be very efficient.

@syllog1sm

This comment has been minimized.

syllog1sm commented Sep 22, 2013

Oh, also, it seems you want urllib.parse.quote_from_bytes for Python 3?

http://docs.python.org/3.4/library/urllib.parse.html

@ghost ghost assigned kgriffs Jan 3, 2014

@kgriffs

This comment has been minimized.

Contributor

kgriffs commented Jan 3, 2014

Cool, thanks for the tips! I ended up referencing py2 and py3 implementations of urllib and then wrote a lean-and-mean url encoder for Falcon.

@kgriffs kgriffs closed this in b04a3a0 Jan 6, 2014

@kgriffs kgriffs removed their assignment Oct 12, 2015

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment