Optimize BytesIO to do less reallocations when written, similarly to StringIO

[elibendersky]

BPO	15381
Nosy	@ncoghlan, @pitrou, @scoder, @vstinner, @benjaminp, @meadori, @berkerpeksag, @hynek, @dw, @serhiy-storchaka
Files	bytesio_resized_bytes-2.patch bytesio_resized_bytes-3.patch bytesio_resized_bytes-4.patch bytesio_faster_readline.patch bytesio_resized_bytes-5.patch bytesio_resized_bytes-6.patch

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/serhiy-storchaka'
closed_at = None
created_at = <Date 2012-07-17.12:12:45.969>
labels = ['library', 'performance']
title = 'Optimize BytesIO to do  less reallocations when written, similarly to StringIO'
updated_at = <Date 2019-03-15.22:56:18.774>
user = 'https://bugs.python.org/elibendersky'

bugs.python.org fields:

activity = <Date 2019-03-15.22:56:18.774>
actor = 'BreamoreBoy'
assignee = 'serhiy.storchaka'
closed = False
closed_date = None
closer = None
components = ['Library (Lib)']
creation = <Date 2012-07-17.12:12:45.969>
creator = 'eli.bendersky'
dependencies = []
files = ['26461', '32536', '36180', '36369', '36371', '37939']
hgrepos = []
issue_num = 15381
keywords = ['patch']
message_count = 47.0
messages = ['165715', '165716', '165718', '165748', '165760', '165780', '165795', '165883', '165884', '165901', '165935', '165937', '165976', '165977', '165978', '165979', '165980', '165981', '165995', '166005', '166007', '166008', '166043', '171205', '202374', '203034', '203045', '224409', '225272', '225274', '225276', '225295', '225298', '225306', '225311', '225316', '225317', '225318', '234885', '235103', '235326', '235339', '235340', '235353', '236003', '236007', '236011']
nosy_count = 17.0
nosy_names = ['ncoghlan', 'pitrou', 'scoder', 'vstinner', 'benjamin.peterson', 'stutzbach', 'Arfrever', 'eli.bendersky', 'meador.inge', 'tshepang', 'python-dev', 'jcon', 'berker.peksag', 'hynek', 'dw', 'serhiy.storchaka', 'kmike']
pr_nums = []
priority = 'normal'
resolution = None
stage = 'patch review'
status = 'open'
superseder = None
type = 'performance'
url = 'https://bugs.python.org/issue15381'
versions = ['Python 3.5']

[elibendersky]

From this pydev thread: http://mail.python.org/pipermail/python-dev/2012-July/120981.html

"BytesIO is actually missing an optimisation that is already used in
StringIO: the StringIO C implementation uses a fragment accumulator
internally, and collapses that into a single string object when
getvalue() is called. BytesIO is still using the old
"resize-the-buffer-as-you-go" strategy, and thus ends up repeatedly
reallocating the buffer as the data sequence grows incrementally.

It should be optimised to work the same way StringIO does (which is
effectively the same way that the monkeypatched version works)"

[elibendersky]

This optimization for StringIO was done in issue bpo-13149

[serhiy-storchaka]

I am not see that BytesIO is slower than StringIO (both are about 30% slower than append/join).

$ ./python -m timeit -s "import io; d=[b'a'*10,b'bb'*5,b'ccc'*5]*1000"  "b=[]; w=b.append"  "for x in d: w(x)"  "b''.join(b)"
1000 loops, best of 3: 966 usec per loop

$ ./python -m timeit -s "import io; d=['a'*10,'bb'*5,'ccc'*5]*1000"  "b=[]; w=b.append"  "for x in d: w(x)"  "''.join(b)"
1000 loops, best of 3: 918 usec per loop

$ ./python -m timeit -s "import io; d=[b'a'*10,b'bb'*5,b'ccc'*5]*1000"  "b=io.BytesIO(); w=b.write"  "for x in d: w(x)"  "b.getvalue()"
1000 loops, best of 3: 1.22 msec per loop

$ ./python -m timeit -s "import io; d=['a'*10,'bb'*5,'ccc'*5]*1000"  "b=io.StringIO(); w=b.write"  "for x in d: w(x)"  "b.getvalue()"
1000 loops, best of 3: 1.24 msec per loop

[elibendersky]

I wonder if this is a fair comparison, Serhiy. Strings are unicode underneath, so they have a large overhead per string (more data to copy around). Increasing the length of the strings changes the game because due to PEP-393, the overhead for ASCII-only Unicode strings is constant:

>>> import sys
>>> sys.getsizeof('a')
50
>>> sys.getsizeof(b'a')
34
>>> sys.getsizeof('a' * 1000)
1049
>>> sys.getsizeof(b'a' * 1000)
1033
>>> 

When re-running your tests with larger chunks, the results are quite interesting:

$ ./python -m timeit -s "import io; d=[b'a'*100,b'bb'*50,b'ccc'*50]*1000"  "b=io.BytesIO(); w=b.write"  "for x in d: w(x)"  "b.getvalue()"
1000 loops, best of 3: 509 usec per loop
$ ./python -m timeit -s "import io; d=['a'*100,'bb'*50,'ccc'*50]*1000"  "s=io.StringIO(); w=s.write"  "for x in d: w(x)"  "s.getvalue()"
1000 loops, best of 3: 282 usec per loop

So, it seems to me that BytesIO could use some optimization!

[elibendersky]

I'd like to take a shot at this, if Antoine doesn't mind. I'll prepare a patch for bytesio.c

Question: what set of benchmarks would it be good to run to make sure this doesn't degrade the performance of BytesIO in various cases?

[serhiy-storchaka]

I wonder if this is a fair comparison, Serhiy.

I just used your examples.

When re-running your tests with larger chunks, the results are quite interesting:

Well, now I see, that BytesIO slower StringIO.

[serhiy-storchaka]

Here is a preliminary version of the patch. I am not sure that it is fully correct.

Microbenchmark results:

$ ./python -m timeit -s "import io; n=100; d=['a'*n,'bb'*n,'ccc'*n]*10000"  "s=io.StringIO(); w=s.write"  "for x in d: w(x)"  "s.getvalue()"
10 loops, best of 3: 25.5 msec per loop
$ ./python -m timeit -s "import io; n=100; d=[b'a'*n,b'bb'*n,b'ccc'*n]*10000"  "s=io.BytesIO(); w=s.write"  "for x in d: w(x)"  "s.getvalue()"
10 loops, best of 3: 39.9 msec per loop
$ ./python-patched -m timeit -s "import io; n=100; d=[b'a'*n,b'bb'*n,b'ccc'*n]*10000"  "s=io.BytesIO(); w=s.write"  "for x in d: w(x)"  "s.getvalue()"
10 loops, best of 3: 26.1 msec per loop

$ ./python -m timeit -s "import io; n=1000; d=['a'*n,'bb'*n,'ccc'*n]*1000"  "s=io.StringIO(); w=s.write"  "for x in d: w(x)"  "s.getvalue()"
100 loops, best of 3: 12.1 msec per loop
$ ./python -m timeit -s "import io; n=1000; d=[b'a'*n,b'bb'*n,b'ccc'*n]*1000"  "s=io.BytesIO(); w=s.write"  "for x in d: w(x)"  "s.getvalue()"
10 loops, best of 3: 26.5 msec per loop
$ ./python-patched -m timeit -s "import io; n=1000; d=[b'a'*n,b'bb'*n,b'ccc'*n]*1000"  "s=io.BytesIO(); w=s.write"  "for x in d: w(x)"  "s.getvalue()"
100 loops, best of 3: 13.6 msec per loop

[pitrou]

Here is a preliminary version of the patch.

I don't understand the purpose of your patch. It just replaces a direct realloc() call with an indirect one (since _PyBytes_Resize() will call realloc() internally).

[pitrou]

> Here is a preliminary version of the patch.

I don't understand the purpose of your patch. It just replaces a
direct realloc() call with an indirect one (since _PyBytes_Resize()
will call realloc() internally).

Ok, I understand. You're trying to make the getvalue() call cheaper,
right?

[serhiy-storchaka]

Ok, I understand. You're trying to make the getvalue() call cheaper,
right?

Yes, it saves up to half of time on large data (on Linux). It would be
interesting to see the results of these microbenchmarks on Windows.

[pitrou]

There seems to be a problem with the patch: when you store the getvalue() result somewhere (instead of discarding it), things get much slower:

$ ./python -m timeit -s "import io; n=2000; d=[b'a'*n,b'bb'*n,b'ccc'*n]*1000"  "s=io.BytesIO(); w=s.write"  "for x in d: w(x)"  "s.getvalue()"
1000 loops, best of 3: 913 usec per loop

$ ./python -m timeit -s "import io; n=2000; d=[b'a'*n,b'bb'*n,b'ccc'*n]*1000"  "s=io.BytesIO(); w=s.write"  "for x in d: w(x)"  "global y; y = s.getvalue()"
100 loops, best of 3: 4.67 msec per loop

This does not happen without the patch.