string.Formatter doesn't understand the a conversion specifier

[ncoghlan]

BPO	13579
Nosy	@terryjreedy, @ncoghlan, @ericvsmith, @merwok, @bitdancer
Files	issue13579_80a50b7ad88f.patch: Changes as proposed by Terry issue13579_a99632426af5.patch issue13579_47288880cc9e.patch issue13579_0554183066b5.patch issue13579_910a4b12c796.patch

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/ericvsmith'
closed_at = <Date 2012-08-19.22:01:19.605>
created_at = <Date 2011-12-11.07:09:49.903>
labels = ['type-bug']
title = "string.Formatter doesn't understand the a conversion specifier"
updated_at = <Date 2012-08-19.22:01:19.604>
user = 'https://github.com/ncoghlan'

bugs.python.org fields:

activity = <Date 2012-08-19.22:01:19.604>
actor = 'r.david.murray'
assignee = 'eric.smith'
closed = True
closed_date = <Date 2012-08-19.22:01:19.605>
closer = 'r.david.murray'
components = []
creation = <Date 2011-12-11.07:09:49.903>
creator = 'ncoghlan'
dependencies = []
files = ['24512', '24526', '24555', '24909', '25425']
hgrepos = []
issue_num = 13579
keywords = ['patch']
message_count = 19.0
messages = ['149214', '149645', '153291', '153389', '153393', '153426', '153429', '153430', '153641', '153646', '153669', '153704', '156153', '156158', '156182', '158496', '159705', '168597', '168598']
nosy_count = 7.0
nosy_names = ['terry.reedy', 'ncoghlan', 'eric.smith', 'eric.araujo', 'r.david.murray', 'python-dev', 'francismb']
pr_nums = []
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue13579'
versions = ['Python 2.7', 'Python 3.2', 'Python 3.3']

[ncoghlan]

As the subject line says:

>>> fmt = "{0!a}"
>>> fmt.format(10)
'10'
>>> import string
>>> string.Formatter().format(fmt, 10)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python3.2/string.py", line 180, in format
    return self.vformat(format_string, args, kwargs)
  File "/usr/lib/python3.2/string.py", line 184, in vformat
    result = self._vformat(format_string, args, kwargs, used_args, 2)
  File "/usr/lib/python3.2/string.py", line 210, in _vformat
    obj = self.convert_field(obj, conversion)
  File "/usr/lib/python3.2/string.py", line 245, in convert_field
    raise ValueError("Unknown conversion specifier {0!s}".format(conversion))
ValueError: Unknown conversion specifier a

[terryjreedy]

At line 237 of string.py:
def convert_field(self, value, conversion):
# do any conversion on the resulting object
if conversion == 'r':
return repr(value)
elif conversion == 's':
return str(value)
elif conversion is None:
return value
add
elif conversion == 'a':
return ascii(value)

The alternatives should be ordered by decreasing frequency. I suspect that should be None, 's', 'r', 'a'.

[francismb]

Hi,
here's a patch with the changes proposed by Terry

Cheers,
francis

[terryjreedy]

Doc/library/string.rst string.Formatter().format(fmt, 10)needs a couple of changes also.

1. "format(format_string, *args, **kwargs)
   format() is the primary API method. It takes a format template string,"

I think 'template' should be removed as we elsewhere (as in the next section) just call them 'format strings'.

Note: I consider this and the preceding sentences

"The built-in string class provides the ability to do complex variable substitutions and value formatting via the format() method described in PEP-3101. The Formatter class in the string module allows you to create and customize your own string formatting behaviors using the same implementation as the built-in format() method."

to imply that this method should accept any format string that str.format does, and that the patch is a bug fix rather than enhancement. There is no mention of any limitation or subseting.

2. "convert_field(value, conversion)
   Converts the value (returned by get_field()) given a conversion type (as in the tuple returned by the parse() method). The default version understands ‘r’ (repr) and ‘s’ (str) conversion types."

Change "‘r’ (repr) and ‘s’ (str)" to "‘s’ (str), ‘r’ (repr), and ‘a’ (ascii)". I consider this a bug in the doc.

---
I discovered that string.Formatter.format has another problem:
it does not auto-number fields

>>> '{0}'.format(10) == '{}'.format(10) == '10'
True
>>> from string import Formatter as F
>>> F().format('{0}', 10)
'10'
>>> F().format('{}', 10)
Traceback (most recent call last):
  File "<pyshell#32>", line 1, in <module>
    F().format('{}', 10)
  File "C:\Programs\Python32\lib\string.py", line 180, in format
    return self.vformat(format_string, args, kwargs)
  File "C:\Programs\Python32\lib\string.py", line 184, in vformat
    result = self._vformat(format_string, args, kwargs, used_args, 2)
  File "C:\Programs\Python32\lib\string.py", line 206, in _vformat
    obj, arg_used = self.get_field(field_name, args, kwargs)
  File "C:\Programs\Python32\lib\string.py", line 267, in get_field
    obj = self.get_value(first, args, kwargs)
  File "C:\Programs\Python32\lib\string.py", line 226, in get_value
    return kwargs[key]
KeyError: ''

So it seems that str.format is either calling something other than _string.formatter_parser or else pre or post processing.

The underlying problem is trying to keep separate codes that do the same thing in sync without testing both with the same set of examples. If examples were in a table of (fmt, args, kwargs, output) tuples, it would be easy to call fmt.format(*args, **kwargs) and F().vformat(fmt, args, kwargs) with each and compare to expected. There are also the %format torture tests in test_format.py. Do % and {} formatting use any of the same code?

[ncoghlan]

Agreed that this is a bug in string.Formatter rather than a new feature.

There's already a separate bug for the autonumbering problem: http://bugs.python.org/issue13598

And I created a new issue about unifying some of the tests: http://bugs.python.org/issue14019

%-style string formatting is its own thing - it doesn't share semantics or code with str.format or string.Formatter.

[francismb]

I have updated the patch with the documentation changes proposed and also added a news entry (does a new entry has to be added in some order?).

Thank for the review in advance.

[terryjreedy]

I just noticed two more minor errors that were and remain in "It takes a format string, and an arbitrary set of positional and keyword argument." Remove the comma and make 'argument' plural.

As near as I can tell, the two change block beginning with
+In less formal terms,
+keyword. If it's a number,
are strictly re-wrapping and no text changes. Correct?

Francisco - News entries are usually entered at the top of the section, just under the heading. But they are a bit tricky for multi-version patches because the context following is often different for different version. I have not decided which is easier -- resolving a merge conflict with a 3-way merge tool or separately editing Misc/NEWS for each version.

Eric, I near as I can tell, and given Nick's last response, this will be ready to commit with the two minor text changes and movement of the NEWS entry. Francisco is already in Misc/ACKS.

[terryjreedy]

However, I have not actually applied and run test_string.py.

[francismb]

Updated. The NEWS line is now at the top of the section.

As near as I can tell, the two change block beginning with
+In less formal terms,
[...]
are strictly re-wrapping and no text changes. Correct?

Yes. Just a pydev question here: I now this re-wrapping
adds noise to the patch, but can/should be done?

Please correct me if I'm wrong: one should try to
limit the line width to 80. If one word starts before
and ends behind is not bad but new words should start
in a new line.

Now in the devguide states “To perform a quick sanity check on your path, you can run make patchcheck”

I did that (I'm using the mq approach here) but I get no useful information (AFAIN):
-----------------------------------------------
Modules/Setup.dist is newer than Modules/Setup;
check to make sure you have all the updates you
need in your Modules/Setup file.
Usually, copying Modules/Setup.dist to Modules/Setup will work.
-----------------------------------------------
./python ./Tools/scripts/patchcheck.py
Getting the list of files that have been added/changed ... 0 files
Fixing whitespace ... 0 files
Fixing C file whitespace ... 0 files
Fixing docs whitespace ... 0 files
Docs modified ... NO
Misc/ACKS updated ... NO
Misc/NEWS updated ... NO

Thanks in advance !

[ncoghlan]

On Sat, Feb 18, 2012 at 10:10 PM, Francisco Martín Brugué
<report@bugs.python.org> wrote:

./python ./Tools/scripts/patchcheck.py
Getting the list of files that have been added/changed ... 0 files
Fixing whitespace ... 0 files
Fixing C file whitespace ... 0 files
Fixing docs whitespace ... 0 files
Docs modified ... NO
Misc/ACKS updated ... NO
Misc/NEWS updated ... NO

I suspect mq will confuse make patchcheck (since it looks for
differences between the working copy and the hg branch tip, and there
are no such differences for an applied mq patch).

Cheers,
Nick.

[francismb]

I suspect mq will confuse make patchcheck (since it looks for
differences between the working copy and the hg branch tip, and there
are no such differences for an applied mq patch).

Does it makes sense to open a feature request for that?

[francismb]

Well, it's done:

http://bugs.python.org/issue14053

[francismb]

However, I have not actually applied and run test_string.py.
I've applied the issue13579_47288880cc9e.patch to the changeset 0554183066b5 and applies without errors for me.