[regression] str.format sublevel format parsing broken in Python 3.3.3

[hct]

BPO	19729
Nosy	@birkenfeld, @vstinner, @ericvsmith, @benjaminp, @bitdancer, @vadmium, @1st1

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2013-11-27.01:24:13.952>
created_at = <Date 2013-11-23.02:16:42.568>
labels = ['interpreter-core', 'type-bug', 'release-blocker']
title = '[regression] str.format sublevel format parsing broken in Python 3.3.3'
updated_at = <Date 2016-02-07.02:40:41.623>
user = 'https://bugs.python.org/hct'

bugs.python.org fields:

activity = <Date 2016-02-07.02:40:41.623>
actor = 'python-dev'
assignee = 'none'
closed = True
closed_date = <Date 2013-11-27.01:24:13.952>
closer = 'python-dev'
components = ['Interpreter Core']
creation = <Date 2013-11-23.02:16:42.568>
creator = 'hct'
dependencies = []
files = []
hgrepos = []
issue_num = 19729
keywords = []
message_count = 10.0
messages = ['203954', '203962', '203963', '204370', '204373', '204533', '204545', '204546', '207264', '259755']
nosy_count = 10.0
nosy_names = ['georg.brandl', 'vstinner', 'eric.smith', 'benjamin.peterson', 'Arfrever', 'r.david.murray', 'python-dev', 'martin.panter', 'yselivanov', 'hct']
pr_nums = []
priority = 'release blocker'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue19729'
versions = ['Python 3.3']

[hct]

can't find a way around it... maybe a new regression test on this.

Python 3.3.3 (v3.3.3:c3896275c0f6, Nov 18 2013, 21:18:40) [MSC v.1600 32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> "0x{:0{:d}X}".format(0x0,16)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ValueError: unmatched '{' in format
>>> "0x{:0{{:d}}X}".format(0x0,16)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ValueError: Invalid format specifier
>>> "0x{:0{:d}X}".format(0x0,16)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ValueError: unmatched '{' in format
>>> "0x{:{:d}X}".format(0x0,16)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ValueError: unmatched '{' in format
>>> "0x{:0{:d}X}".format(0x0,16)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ValueError: unmatched '{' in format
>>> "0x{:0{:d}x}".format(0x0,16)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ValueError: unmatched '{' in format
>>>
>>> "{:0{}x}".format(0,16)
'0000000000000000'

"0x{:0{:d}X}".format(0x0,16) and "{:0{}x}".format(0,16)
work with Python 3.1, 3.2 and up to 3.3.2

[benjaminp]

Amusingly, it also works in 3.4.

[vstinner]

According to hg bisect, the regression comes from:

changeset: 83817:6786e681ed58
branch: 3.3
parent: 83812:cd2457463eeb
user: Benjamin Peterson <benjamin@python.org>
date: Fri May 17 17:34:30 2013 -0500
files: Lib/test/test_unicode.py Misc/NEWS Objects/stringlib/unicode_format.h
description:
only recursively expand in the format spec (closes bpo-17644)

[1st1]

This broke a lot of our code, I think that priority needs to be raised to urgent.

[hct]

my projects are total broken by this, so my temporary solution is to down grade to 3.3.2

somehow we don't have any test to check this before releasing 3.3.3

[birkenfeld]

Hmm, sucks.

Benjamin, can you come up with a fix? I'll give it a few weeks to wait for more potential regressions and then do a 3.3.4.

[python-dev]

New changeset bab7dc2ffc16 by Benjamin Peterson in branch '3.3':
fix format spec recursive expansion (closes bpo-19729)
http://hg.python.org/cpython/rev/bab7dc2ffc16

New changeset e27684eed3b6 by Benjamin Peterson in branch 'default':
merge 3.3 (bpo-19729)
http://hg.python.org/cpython/rev/e27684eed3b6

[benjaminp]

Sorry about that.

[hct]

just want to know if there is any tentative schedule to release 3.3.4. the PEP for 3.3 schedule doesn't talk about 3.3.4, so I'm not sure if the decision was to leave latest 3.3 with this broken regression or fix it and release 3.3.4.

[python-dev]

New changeset f25d8cbd074a by Martin Panter in branch '3.5':
Issue bpo-25179: Preparatory cleanup of existing docs on string formatting
https://hg.python.org/cpython/rev/f25d8cbd074a

New changeset fe692ee6d19a by Martin Panter in branch '2.7':
Issue bpo-25179: Preparatory cleanup of existing docs on string formatting
https://hg.python.org/cpython/rev/fe692ee6d19a