Optimize ASCII/latin1 encoder with surrogateescape error handlers

[vstinner]

BPO	25227
Nosy	@vstinner, @methane, @serhiy-storchaka
Files	encode_ucs1_surrogateescape.patch encode_ucs1_surrogateescape-2.patch bench.py

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2015-09-29.10:39:51.861>
created_at = <Date 2015-09-24.12:48:03.415>
labels = ['performance']
title = 'Optimize ASCII/latin1 encoder with surrogateescape error handlers'
updated_at = <Date 2015-09-29.10:39:51.805>
user = 'https://github.com/vstinner'

bugs.python.org fields:

activity = <Date 2015-09-29.10:39:51.805>
actor = 'vstinner'
assignee = 'none'
closed = True
closed_date = <Date 2015-09-29.10:39:51.861>
closer = 'vstinner'
components = []
creation = <Date 2015-09-24.12:48:03.415>
creator = 'vstinner'
dependencies = []
files = ['40562', '40563', '40564']
hgrepos = []
issue_num = 25227
keywords = ['patch']
message_count = 6.0
messages = ['251516', '251518', '251525', '251526', '251841', '251843']
nosy_count = 4.0
nosy_names = ['vstinner', 'methane', 'python-dev', 'serhiy.storchaka']
pr_nums = []
priority = 'normal'
resolution = 'fixed'
stage = None
status = 'closed'
superseder = None
type = 'performance'
url = 'https://bugs.python.org/issue25227'
versions = ['Python 3.6']

[vstinner]

Attached patch is based on faster_surrogates_hadling.patch written by Serhiy Storchaka for the issue bpo-24870. It optimizes str.encode('ascii', 'surrogateescape') and str.encode('ascii', 'latin1').

[python-dev]

New changeset fa65c32d7134 by Victor Stinner in branch 'default':
Issue bpo-25227: Cleanup unicode_encode_ucs1() error handler
https://hg.python.org/cpython/rev/fa65c32d7134

[vstinner]

Updated test now with more unit tests.

[vstinner]

Result of a micro-benchmark with encode_ucs1_surrogateescape-2.patch.

Common platform:
Timer info: namespace(adjustable=False, implementation='clock_gettime(CLOCK_MONOTONIC)', monotonic=True, resolution=1e-09)
Timer: time.perf_counter
CPU model: Intel(R) Core(TM) i7-2600 CPU @ 3.40GHz
Python unicode implementation: PEP-393
Platform: Linux-4.1.6-200.fc22.x86_64-x86_64-with-fedora-22-Twenty_Two
CFLAGS: -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes
Bits: int=32, long=64, long long=64, size_t=64, void*=64

Platform of campaign before:
Date: 2015-09-24 16:12:35
Timer precision: 54 ns
Python version: 3.6.0a0 (default:fa65c32d7134, Sep 24 2015, 16:11:44) [GCC 5.1.1 20150618 (Red Hat 5.1.1-4)]
SCM: hg revision=fa65c32d7134 tag=tip branch=default date="2015-09-24 14:45 +0200"

Platform of campaign after:
Python version: 3.6.0a0 (default:fa65c32d7134+, Sep 24 2015, 16:13:20) [GCC 5.1.1 20150618 (Red Hat 5.1.1-4)]
Timer precision: 55 ns
SCM: hg revision=fa65c32d7134+ tag=tip branch=default date="2015-09-24 14:45 +0200"
Date: 2015-09-24 16:13:21

-----------------------+-------------+---------------
ascii | before | after
-----------------------+-------------+---------------
100 x 10**1 characters | 6.65 us () | 1.93 us (-71%)
100 x 10**3 characters | 512 us () | 158 us (-69%)
100 x 10**2 characters | 52.2 us () | 16.2 us (-69%)
100 x 10**4 characters | 5.09 ms () | 1.59 ms (-69%)
-----------------------+-------------+---------------
Total | 5.66 ms (*) | 1.77 ms (-69%)
-----------------------+-------------+---------------

-----------------------+-------------+---------------
latin1 | before | after
-----------------------+-------------+---------------
100 x 10**1 characters | 6.24 us () | 1.89 us (-70%)
100 x 10**3 characters | 500 us () | 160 us (-68%)
100 x 10**2 characters | 51 us () | 16.3 us (-68%)
100 x 10**4 characters | 5 ms () | 1.59 ms (-68%)
-----------------------+-------------+---------------
Total | 5.56 ms (*) | 1.77 ms (-68%)
-----------------------+-------------+---------------

--------+-------------+---------------
Summary | before | after
--------+-------------+---------------
ascii | 5.66 ms () | 1.77 ms (-69%)
latin1 | 5.56 ms () | 1.77 ms (-68%)
--------+-------------+---------------
Total | 11.2 ms (*) | 3.53 ms (-69%)
--------+-------------+---------------

[python-dev]

New changeset 128a3f03ddeb by Victor Stinner in branch 'default':
Optimize ascii/latin1+surrogateescape encoders
https://hg.python.org/cpython/rev/128a3f03ddeb

[vstinner]

INADA Naoki: The ASCII and latin1 encoders are now up to 3 times as fast when the surrogateescape error handler is used in Python 3.6.