builtin __format__ methods cannot fill with \x00 char

[GavinAndresen]

BPO	12546
Nosy	@vstinner, @ericvsmith, @ezio-melotti, @florentx, @sorcio
Files	format00.patch

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/ericvsmith'
closed_at = <Date 2014-04-14.16:09:44.041>
created_at = <Date 2011-07-13.06:58:06.161>
labels = ['interpreter-core', 'type-bug']
title = 'builtin __format__ methods cannot fill with \\x00 char'
updated_at = <Date 2014-05-19.15:15:04.784>
user = 'https://bugs.python.org/GavinAndresen'

bugs.python.org fields:

activity = <Date 2014-05-19.15:15:04.784>
actor = 'eric.smith'
assignee = 'eric.smith'
closed = True
closed_date = <Date 2014-04-14.16:09:44.041>
closer = 'eric.smith'
components = ['Interpreter Core']
creation = <Date 2011-07-13.06:58:06.161>
creator = 'Gavin.Andresen'
dependencies = []
files = ['22640']
hgrepos = []
issue_num = 12546
keywords = ['patch']
message_count = 14.0
messages = ['140225', '140231', '140233', '140235', '140238', '140242', '140654', '215457', '215793', '216093', '216105', '216106', '218782', '218797']
nosy_count = 7.0
nosy_names = ['vstinner', 'eric.smith', 'ezio.melotti', 'flox', 'davide.rizzo', 'python-dev', 'Gavin.Andresen']
pr_nums = []
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue12546'
versions = ['Python 2.7', 'Python 3.4', 'Python 3.5']

[GavinAndresen]

This gives me "foo " instead of expected "foo\x00\x00\x00" :

"{0:\x00<6}".format('foo')

[ericvsmith]

\x00 is used as a flag internally meaning "use the default fill character". That's clearly a bug. I'll look at fixing it.

I think there are other places in the built in __format__ functions where special values are used instead of flags. I'll review those as well.

Thanks for the report!

[sorcio]

This patch removes the special meaning for \x00 and defines the default padding character (' ') in parse_internal_render_format_spec. Test included. Maybe the default padding character should be defined elsewhere?

[sorcio]

Oops, sorry. Above patch was overly buggy. Please just ignore it.

[sorcio]

Here's the patch. Same rationale as above (removed the special meaning of '\x00', default specified in parse_internal_render_format_spec). Sorry about the mess again!

[ericvsmith]

Patch looks good at first glance. I'll review it some more today and commit it. Thanks!

[ericvsmith]

I finally got around to reviewing the patch. A couple of comments:

1. There should be some tests for str.__format__, not just str.format. This is really a bug with str.__format__, after all. I can add those.

2. The bigger issue is that the other built in formatters have this same problem.

>>> format(3, '\x00<6')
'3     '
>>> format(3., '\x00<6')
'3.0   '
>>> format('3', '\x00<6')
'3\x00\x00\x00\x00\x00'
>>> format(3+1j, '\x00<6')
'(3+1j)'
[38654 refs]
>>> format(3+1j, '\x00<10')
'(3+1j)    '

I think the fix is basically the same as str.__format__ (but in format_int_or_long_internal, format_float_internal, and format_complex_internal). I tried that and it worked, but I haven't had time to write tests for them. If you (Davide) can do that, great. Otherwise I'll try and grab some time this week.

Changing the subject to match the wider scope of the problem.

[vstinner]

bpo-17705 has been closed as a duplicate of this issue.

[vstinner]

The patch looks good to me.