str.format_map()

[rhettinger]

BPO	6081
Nosy	@rhettinger, @terryjreedy, @jaraco, @ericvsmith, @ezio-melotti, @merwok, @bitdancer, @florentx
Files	6081_1.patch issue6081.diff 6081_2.diff issue6081-1.diff

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/ericvsmith'
closed_at = <Date 2010-11-04.17:07:37.170>
created_at = <Date 2009-05-21.23:23:15.179>
labels = ['interpreter-core', 'easy', 'type-feature']
title = 'str.format_map()'
updated_at = <Date 2011-01-28.19:20:05.673>
user = 'https://github.com/rhettinger'

bugs.python.org fields:

activity = <Date 2011-01-28.19:20:05.673>
actor = 'jaraco'
assignee = 'eric.smith'
closed = True
closed_date = <Date 2010-11-04.17:07:37.170>
closer = 'eric.smith'
components = ['Interpreter Core']
creation = <Date 2009-05-21.23:23:15.179>
creator = 'rhettinger'
dependencies = []
files = ['16576', '16586', '16632', '19482']
hgrepos = []
issue_num = 6081
keywords = ['patch', 'easy', 'needs review']
message_count = 22.0
messages = ['88173', '88425', '101265', '101269', '101272', '101273', '101283', '101324', '101325', '101342', '101346', '101349', '101603', '101609', '101692', '113443', '113610', '113617', '113622', '120361', '120422', '127325']
nosy_count = 10.0
nosy_names = ['rhettinger', 'terry.reedy', 'jaraco', 'eric.smith', 'ezio.melotti', 'eric.araujo', 'r.david.murray', 'gruszczy', 'ebehar', 'flox']
pr_nums = []
priority = 'normal'
resolution = 'accepted'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue6081'
versions = ['Python 3.2']

[rhettinger]

The old % formatting allowed arbitrary mappings:

  >>> class Default(dict):
  ...     def __missing__(self, key):
  ...         return key
  ...
  >>> s = '%(name)s was born in %(country)s'
  >>> s % Default(name='Guido')
  'Guido was born in country'

But the new str.format() demands mappings be first converted to a
regular dict using **kwargs, so something like the above is not possible.

  >>> s = '{name} was born in {country}'
  >>> s.format(**Default(name='Guido'))
  Traceback (most recent call last):
    File "<pyshell#27>", line 1, in <module>
      s.format(**Default(name='Guido'))
  KeyError: 'country'

There is a work-around using string.vformat() but it is obscure and awkward:

  >>> import string
  >>> string.Formatter().vformat(s, (), Default(name='Guido'))
  'Guido was born in country'

Instead, it would be better to offer an alternate method:

  >>> s.format_from_mapping(Default(name='Guido'))
  'Guido was born in country'

[ericvsmith]

I think this would be useful.

I don't fee terribly strongly about it, but I think I'd like the name
str.format_using_mapping(). When I initially saw this, I thought from
the name it was creating a format object (whatever that would be) from a
mapping object.

[gruszczy]

I have created a small patch, that adds method that formats using a dict. It's the first time I have written anything in python implementation, so I would very appreciate any advice. Change allows the following:

>>> m = Mapping(a='b')
>>> '{a} {c}'.format_using_mapping(m)
'b c'
>>>

[ezio-melotti]

Thanks for the patch. It would be nice if you could include unit tests too.

[bitdancer]

I don't think this patch satisfies Raymond's request. It is explicitly checking for a __missing__ attribute, but Raymond was talking about a more general facility whereby you can pass in an arbitrary object that implements the mapping interface. Using the __missing__ facility was just an example of why this would be useful.

[ericvsmith]

I agree with David.

Although it's not clear to my why the code doesn't just work with the addition of do_string_format_using_mapping and without the other code. It's possible the existing code is too dict-specific and should be calling a more generic PyObject interface, like PyMapping_GetItemString instead of PyDict_GetItem.

[gruszczy]

My first intention was simply to push mapping from args to kwargs, just like Eric suggested, but that didn't help with __missing__, only with accepting a dict instead of pushing keyword arguments.

I didn't like explicitly asking for __missing__ either, but since I have little knowledge of what should be called, I didn't know what to use. I too believe something else the PyDict_GetItem should be called, something that would take care of __missing__ and other possibilities (I don't know what exactly and really would like to know what these are) internally.

I am going to check, whether PyMapping_GetItemString is going to help. But can this really be called on a dict (or a subclass of dict)? What about retrieving getitem method from the given object and simply calling it? Wouldn't that do the trick?

[ericvsmith]

I believe this patch fixes the issue. Tests and documentation are still needed, of course.

[ericvsmith]

Added a comment to explain the change.

[gruszczy]

Could you point me, where to add tests and documentation? I would happily add those.

[ericvsmith]

http://docs.python.org/library/stdtypes.html#str.format, for starters. This is in Doc/library/stdtypes.rst.

For tests, probably in Lib/test/test_unicode.py.

I'm not sure if we should add this to 2.7 (or even 3.2, for that matter), but if so, we should start by patching trunk and then porting to py3k.

[gruszczy]

Ok, unfortunately this code won't work for certain tests. Take those:

    self.assertEqual("My name is {0}".format('Fred'), "My name is Fred")

We pass only one argument, which is a dict and this won't satisfy such test. We need to think about a different way of passing those arguments there. We can do one of two thins:

• the last argument of args tuple would be an object that can be subscripted for format values
• keyword with above object

I believe the second version is more explicit and therefore better.

[gruszczy]

I have created a new patch, that should be satisfying now. There is help (though it is quite small, I tried to mimic those that were already in unicode.c) and tests. Right now format_using_mapping is called like this:

format_using_mapping(mapping, *args)

where mapping is a subscriptible object, that will be pushed to kwargs and the following args are used just like in normal format. This should be as similar to the normal format as possible.

There are also tests, including usage presented in the ticket.

[ericvsmith]

I'm not sure I'm wild about the *args parameter. Calling "Fred" the 0-th parameter here seems non-intuitive:

"My name is {0}".format_using_mapping({}, 'Fred')

If you're going to have *args, why not **kwargs and then merge/update the dicts? I'm being facetious, but I think even having *args is feature creep.

I think it's time to ask about this on python-dev. I'd vote for not using *args. It can always be added in the future if it's seen as a hole in the API.

[ericvsmith]

It occurs to me that Raymond's use case could be satisfied using existing Python, by slightly changing the format string. After all, str.format() supports mapping lookup already:

$ ./python.exe
Python 2.6.5+ (release26-maint:79421, Mar 25 2010, 08:51:39)
[GCC 4.2.1 (Apple Inc. build 5646) (dot 1)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> class Default(dict):
...  def __missing__(self, key):
...   return key
...
>>> s = '{m[name]} was born in {m[country]}'
>>> s.format(m=Default(name='Guido'))
'Guido was born in country'
>>>

Considering that, maybe the best thing is to do nothing. Or maybe update the documentation with this example.

Plus, you could mix and match this with *args as you'd like.

[terryjreedy]

I believe this is covered by the PEP-3003 3.2 change moratorium.

[rhettinger]

This can be done for Py3.2. It completes needed functionality for string formatting which is something we all want to take hold and is necessary for the 3.x series to succeed.

[ericvsmith]

I'll work on cleaning this up for 3.2.

Any comments on the name of the method?

[terryjreedy]

I understand now that new methods, as opposed to changed methods, are allowed.

I agree with Eric that this seems more like a convinience rather than absolute necessity, and that the doc should be augmented.

The doc for vformat (which I admit I had not noticed before) says it is exposed just for this case:

"vformat(format_string, args, kwargs)
This function does the actual work of formatting. It is exposed as a separate function for cases where you want to pass in a predefined dictionary of arguments, rather than unpacking and repacking the dictionary as individual arguments using the *args and **kwds syntax."

'Dictionary' should be replaced with 'mapping'.

string.Formatter.format is documented as "just a wrapper that calls vformat(). Is the same effectively true of str.format also?

If .format_map (I prefer shorted names) is added as a convenience str method, particularly for matching current %-formatting use, I think it should take just one parameter, mapping. I presume it could implemented as a wrapper for .vformat (or whatever internal function .vformat calls).

str.format_map(map) == string.Format.vformat(formstring, (), map)

More complicated, mixed cases can use the explict lookup with map arg.

[ericvsmith]

Updated patch which adds tests and minimal docs. Named changed to format_map. I'll commit this before 3.2b1 unless I hear a complaint.

[ericvsmith]

Committed to 3.2 in r86170.

[jaraco]

Good work Eric.

When I first heard of new string formatting, I was a little wary. The syntax to supply a dictionary of keyword replacements seemed awkward. It took me a while before I realized why it really bothered me. There's string formatting you can do with the old format operator (%) that you can't do with str.format.

Here's an example.

    import random
    class MyDynamicObject:
        def __getitem__(self, name):
            return name + ' ' + str(random.randint(1,10))

print("%(foo)s" % MyDynamicObject()) # works!
print("{foo}".format(**MyDynamicObject())) # can't do that because

MyDynamicObject can't enumerate every possible kwparam

As you can see, the % operator naturally accepts any object that responds to __getitem__ but .format requires that all keyword params be enumerated in advance. This limitation seems to me to be a serious problem to favoring .format over %.

I frequently use % to format the properties of an object... and while
it's true one can use myob.__dict__ or vars(myob) to get a dictionary of
some of the values, that doesn't work for properties and other dynamic
behavior.

format_map addresses this shortcoming nicely. Thanks.