fix output string length for binascii.b2a_uu()

[vstinner]

BPO	7701
Nosy	@pitrou, @vstinner
Files	binascii_b2a_uu_length-3.patch

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2010-01-15.00:31:38.894>
created_at = <Date 2010-01-14.11:56:21.186>
labels = ['extension-modules', 'type-crash']
title = 'fix output string length for binascii.b2a_uu()'
updated_at = <Date 2010-01-15.00:31:38.892>
user = 'https://github.com/vstinner'

bugs.python.org fields:

activity = <Date 2010-01-15.00:31:38.892>
actor = 'pitrou'
assignee = 'none'
closed = True
closed_date = <Date 2010-01-15.00:31:38.894>
closer = 'pitrou'
components = ['Extension Modules']
creation = <Date 2010-01-14.11:56:21.186>
creator = 'vstinner'
dependencies = []
files = ['15879']
hgrepos = []
issue_num = 7701
keywords = ['patch']
message_count = 6.0
messages = ['97759', '97760', '97764', '97775', '97777', '97796']
nosy_count = 2.0
nosy_names = ['pitrou', 'vstinner']
pr_nums = []
priority = 'normal'
resolution = 'fixed'
stage = None
status = 'closed'
superseder = None
type = 'crash'
url = 'https://bugs.python.org/issue7701'
versions = ['Python 2.6', 'Python 3.1', 'Python 2.7', 'Python 3.2']

[vstinner]

binascii_b2a_uu() estimate the output string length using 2+bin_len*2.
It's almost correct... except for bin_len=1. The result is a memory
write into unallocated memory:

   $ ./python -c "import binascii; binascii.b2a_uu('x')"
   Debug memory block at address p=0x87da568: API 'o'
       33 bytes originally requested
       The 3 pad bytes at p-3 are FORBIDDENBYTE, as expected.
       The 4 pad bytes at tail=0x87da589 are not all FORBIDDENBYTE (0xfb):
           at tail+0: 0x0a *** OUCH
           at tail+1: 0xfb
           at tail+2: 0xfb
           at tail+3: 0xfb
       The block was made by call python/issues-test-cpython#25195 to debug malloc/realloc.
       Data at p: 00 00 00 00 00 00 00 00 ... 00 00 00 21 3e 20 20 20
   Fatal Python error: bad trailing pad byte
   Abandon

Current output string length estimation for input string 0..10:

    >>> [len(binascii.b2a_uu("x"*bin_len)) for bin_len in xrange(10)]
    [2, 6, 6, 6, 10, 10, 10, 14, 14, 14]
    >>> [(2+bin_len*2) for bin_len in xrange(10)]
    [2, 4, 6, 8, 10, 12, 14, 16, 18, 20]

The estimation is correct for all lengths... except for bin_len=1. And
it's oversized for bin_len >= 9. The exact length is:

2+ceil(bin_len*8/6) <=> 2+(bin_len+5)*8//6 <=> 2+(bin_len+2)*4//3

Example with length 0..10:

    >>> [len(binascii.b2a_uu("x"*bin_len)) for bin_len in xrange(10)]
    [2, 6, 6, 6, 10, 10, 10, 14, 14, 14]
    >>> [(2+(bin_len+2)*4//3) for bin_len in xrange(10)]
    [4, 6, 7, 8, 10, 11, 12, 14, 15, 16]

Attached patch uses the correct estimation.

[pitrou]

>>> [len(binascii.b2a_uu("x"*bin_len)) for bin_len in xrange(10)]
    [2, 6, 6, 6, 10, 10, 10, 14, 14, 14]
    >>> [(2+(bin_len+2)*4//3) for bin_len in xrange(10)]
    [4, 6, 7, 8, 10, 11, 12, 14, 15, 16]

How is this the correct estimation? The results are different.

Try the following:

>>> [(2+(bin_len+2)//3*4) for bin_len in xrange(10)]
[2, 6, 6, 6, 10, 10, 10, 14, 14, 14]

[vstinner]

How is this the correct estimation? The results are different.

The estimation have be bigger or equal, but not smaller.

Try the following:
>>> [(2+(bin_len+2)//3*4) for bin_len in xrange(10)]
[2, 6, 6, 6, 10, 10, 10, 14, 14, 14]

Cool, it's not an estimation but the exact result :-) I prefer to leave the resize unchanged. The new patch uses your "estimation" ;-)

[pitrou]

The patch doesn't apply cleanly against trunk (due to today's commits I fear, sorry).
Also, it would be nice to add a test.

[vstinner]

The patch doesn't apply cleanly against trunk

Because of r77497 (issue #770). No problem, here is the new patch. I'm now using a git-svn repository to keep all my patches. It's much easier to update them to trunk ;-)

Also, it would be nice to add a test.

done

[pitrou]

Patch committed in r77506, r77507, r77508 and r77509. Thank you!