html.escape 10x slower than cgi.escape

[florentx]

BPO	18020
Nosy	@akuchling, @orsenthil, @jwilk, @ezio-melotti, @florentx
Files	htmlescape.patch: Speed up html.escape()

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/ezio-melotti'
closed_at = <Date 2013-07-07.09:12:39.800>
created_at = <Date 2013-05-20.08:21:38.319>
labels = ['library', 'performance']
title = 'html.escape 10x slower than cgi.escape'
updated_at = <Date 2013-07-07.09:12:39.797>
user = 'https://github.com/florentx'

bugs.python.org fields:

activity = <Date 2013-07-07.09:12:39.797>
actor = 'ezio.melotti'
assignee = 'ezio.melotti'
closed = True
closed_date = <Date 2013-07-07.09:12:39.800>
closer = 'ezio.melotti'
components = ['Library (Lib)']
creation = <Date 2013-05-20.08:21:38.319>
creator = 'flox'
dependencies = []
files = ['30325']
hgrepos = []
issue_num = 18020
keywords = ['patch']
message_count = 8.0
messages = ['189641', '189643', '189644', '189647', '189711', '190267', '192527', '192528']
nosy_count = 8.0
nosy_names = ['akuchling', 'orsenthil', 'jwilk', 'ezio.melotti', 'grahamd', 'flox', 'python-dev', 'Teh Matt']
pr_nums = []
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'performance'
url = 'https://bugs.python.org/issue18020'
versions = ['Python 3.4']

[florentx]

I noticed the convenient html.escape in Python 3.2 and cgi.escape is marked as deprecated.

However, the former is an order of magnitude slower than the latter.

$ python3 --version
Python 3.3.2

With html.escape:

$ python3 -m timeit -s "from html import escape as html; from cgi import escape; s = repr(copyright)" "h = html(s)"
10000 loops, best of 3: 48.7 usec per loop
$ python3 -m timeit -s "from html import escape as html; from cgi import escape; s = repr(copyright) * 19" "h = html(s)"
1000 loops, best of 3: 898 usec per loop

With cgi.escape:

$ python3 -m timeit -s "from html import escape as html; from cgi import escape; s = repr(copyright)" "h = escape(s)"
100000 loops, best of 3: 7.42 usec per loop
$ python3 -m timeit -s "from html import escape as html; from cgi import escape; s = repr(copyright) * 19" "h = escape(s)"
10000 loops, best of 3: 21.5 usec per loop

Since this kind of function is called frequently in template engines, it makes a difference.
Of course C replacements are available on PyPI: MarkupSafe or Webext

But it would be nice to restore the performance of cgi.escape with a pragmatic .replace( approach.

[grahamd]

Importing the cgi module the first time even in Python 2.X was always very expensive. I would suggest you redo the test using timing done inside of the script after modules have been imported so as to properly separate module import time in both cases from execution time of the specific function.

[florentx]

I would suggest you redo the test using timing done inside of the script after modules have been imported.

The -s switch takes care of this.

[grahamd]

Whoops. Missed the quoting.

[TehMatt]

I did a few more tests and am seeing the same speed differences Florent noticed.
It seems reasonable to use .replace() instead, as it does the same thing significantly faster.
I've attached a patch doing just this.

[akuchling]

Matt's patch looks good to me. It removes two module-level dicts, but they're marked as internal, so that's OK. There's already a test case that exercises html.escape(), so I don't think any additional tests are needed.

[python-dev]

New changeset db5f2b74e369 by Ezio Melotti in branch 'default':
bpo-18020: improve html.escape speed by an order of magnitude. Patch by Matt Bryant.
http://hg.python.org/cpython/rev/db5f2b74e369

[ezio-melotti]

Fixed, thanks for the report and the patch!