String interpolation doesn't work with sys.version_info #52660

Arfrever · 2010-04-16T01:28:09Z

BPO	8413
Nosy	@rhettinger, @abalkin, @pitrou, @ericvsmith, @benjaminp, @ezio-melotti, @briancurtin
Files	structseq-inheritance.patch structseq-inheritance2.patch

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/benjaminp'
closed_at = <Date 2010-07-07.20:54:27.486>
created_at = <Date 2010-04-16.01:28:08.841>
labels = ['interpreter-core', 'type-bug']
title = "String interpolation doesn't work with sys.version_info"
updated_at = <Date 2010-07-08.19:33:03.669>
user = 'https://bugs.python.org/Arfrever'

bugs.python.org fields:

activity = <Date 2010-07-08.19:33:03.669>
actor = 'belopolsky'
assignee = 'benjamin.peterson'
closed = True
closed_date = <Date 2010-07-07.20:54:27.486>
closer = 'benjamin.peterson'
components = ['Interpreter Core']
creation = <Date 2010-04-16.01:28:08.841>
creator = 'Arfrever'
dependencies = []
files = ['17890', '17896']
hgrepos = []
issue_num = 8413
keywords = ['patch']
message_count = 23.0
messages = ['103282', '103305', '103797', '104519', '104521', '104523', '104524', '104525', '104529', '104530', '104531', '108497', '109445', '109495', '109496', '109497', '109498', '109499', '109500', '109501', '109502', '109556', '109577']
nosy_count = 8.0
nosy_names = ['rhettinger', 'belopolsky', 'pitrou', 'eric.smith', 'benjamin.peterson', 'ezio.melotti', 'Arfrever', 'brian.curtin']
pr_nums = []
priority = 'high'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue8413'
versions = ['Python 3.1', 'Python 2.7', 'Python 3.2']

Arfrever · 2010-04-16T01:28:08Z

String interpolation doesn't work with sys.version_info in Python versions in which sys.version_info is a named tuple. It's a regression in Python 2.7 and 3.1. This problem doesn't concern named tuples created using collections.namedtuple().

This problem might also concern sys.getwindowsversion(), but I don't have access to Windows system, so I can't check it.

$ python2.6 -c 'import sys; print("%s.%s.%s-%s-%s" % sys.version_info)'
2.6.5-final-0
$ python2.7 -c 'import sys; print("%s.%s.%s-%s-%s" % sys.version_info)'
Traceback (most recent call last):
  File "<string>", line 1, in <module>
TypeError: not enough arguments for format string
$ python3.0 -c 'import sys; print("%s.%s.%s-%s-%s" % sys.version_info)'
3.0.1-final-0
$ python3.1 -c 'import sys; print("%s.%s.%s-%s-%s" % sys.version_info)'
Traceback (most recent call last):
  File "<string>", line 1, in <module>
TypeError: not enough arguments for format string

ericvsmith · 2010-04-16T08:49:36Z

This affects any type implemented as PyStructSequence. For example, sys.flags has the same behavior.

ericvsmith · 2010-04-21T00:21:06Z

I think the right way to handle this is to modify the test:

if (Py_TYPE(args)->tp_as_mapping && !PyTuple_Check(args) &&
!PyObject_TypeCheck(args, &PyBaseString_Type))

to also exclude PyStructSequence's, but since they're all distinct types I don't see how to do that. I don't really think listing all of the PyStructSequence types would be acceptable.

This is the test that's trying to figure out if %-formatting should use mapping or tuple expansion.

AlexanderBelopolsky · 2010-04-29T15:18:55Z

I am not sure fixing this quirk of %-formatting is worth the trouble given that one simply use new style {}-formatting:

$ python-c 'import sys; print("{}.{}.{}-{}-{}".format(*sys.version_info))'
3.2.0-alpha-0

or a work-around

$ python -c 'import sys; print("%s.%s.%s-%s-%s" % tuple(sys.version_info))'
3.2.0-alpha-0

pitrou · 2010-04-29T15:22:25Z

Well, the problem is that it's a regression from 2.6.
It also shows why these "fancy" structs aren't necessarily a good idea.
I agree fixing in py3k is less important, though it would be nice too.

ericvsmith · 2010-04-29T15:36:51Z

The other thing to do would be to convince PyTuple_Check that PyStructSequences are tuples, but that would no doubt come with its own set of regressions.

Since this problem is 1) hard to fix and 2) probably of minimal impact, I think the best course is to do nothing.

AlexanderBelopolsky · 2010-04-29T15:37:02Z

The recommended way to check that an object is an instance of a namedtuple is to check for the _fields attribute. See bpo-7796, msg99869.

ericvsmith · 2010-04-29T15:43:36Z

Yes, but sys.version_info isn't a namedtuple (which are in fact tuples), it's the (sort-of) C equivalent, which isn't a real tuple.

>>> from collections import namedtuple
>>> x = namedtuple('x', 'a b c')
>>> '%s %s %s' % x(1, 2, 3)
'1 2 3'

Hmm, but sys.version_info is a tuple:
>>> isinstance(sys.version_info, tuple)
True

I'll have to check if PyTuple_Check returns true or not. Maybe the problem is in the mapping check.

I'll investigate more.

rhettinger · 2010-04-29T15:52:34Z

IIRC, there is an open ticket to make structseq inherit from tuple to avoid just this kind of problem.

ericvsmith · 2010-04-29T15:56:19Z

Looks like that's bpo-1820.

AlexanderBelopolsky · 2010-04-29T15:56:28Z

On Thu, Apr 29, 2010 at 11:52 AM, Raymond Hettinger
<report@bugs.python.org> wrote:
..

IIRC, there is an open ticket to make structseq inherit from tuple to avoid just this kind of problem.

Indeed, see bpo-1820.

Arfrever · 2010-06-24T01:13:17Z

Should this regression block final release of 2.7 or can it be fixed in e.g. 2.7.1?

benjaminp · 2010-07-07T00:29:57Z

Here's a patch. It makes structseq a subclass of tuple and along the way deletes tons of code. Please review.

abalkin · 2010-07-07T19:56:26Z

This is definitely the right way to do it. I expect that I will have only minor nit-picks as I go through the patch.

You can probably just do

#define PyStructSequence_SET_ITEM PyTuple_SET_ITEM
#define PyStructSequence_GET_ITEM PyTuple_GET_ITEM

there is no need to repeat the argument lists.

I am comparing PyStructSequence_New and PyTuple_New:

PyStructSequence_New does not fill ob_item array with NULLs.
PyStructSequence_New does not call _PyObject_GC_TRACK

I believe tp_free gets inherited, so structseq tp_new should follow what tuple's tp_new does. I am not 100% sure on the second point, though _PyObject_GC_TRACK may be redundant after PyObject_GC_NewVar.

In structseq_repr, do you need PyTuple_GetItem? I think you can get away with PyTuple_GET_ITEM, or better PyStructSequence_GET_ITEM.

benjaminp · 2010-07-07T20:19:00Z

2010/7/7 Alexander Belopolsky <report@bugs.python.org>:

Alexander Belopolsky <belopolsky@users.sourceforge.net> added the comment:

This is definitely the right way to do it. I expect that I will have only minor nit-picks as I go through the patch.

You can probably just do

#define PyStructSequence_SET_ITEM PyTuple_SET_ITEM
#define PyStructSequence_GET_ITEM PyTuple_GET_ITEM

there is no need to repeat the argument lists.

I think the preprocessor requires this. (Anyway it fails without.)

I am comparing PyStructSequence_New and PyTuple_New:
- PyStructSequence_New does not fill ob_item array with NULLs.

I'm not sure it really matters, since the fields should always be
filled in, but I suppose it can't hurt...

- PyStructSequence_New does not call _PyObject_GC_TRACK

I believe tp_free gets inherited, so structseq tp_new should follow what tuple's tp_new does. I am not 100% sure on the second point, though _PyObject_GC_TRACK may be redundant after PyObject_GC_NewVar.

This is only needed when doing freelist magic in tupleobject.c.

In structseq_repr, do you need PyTuple_GetItem? I think you can get away with PyTuple_GET_ITEM, or better PyStructSequence_GET_ITEM.

Yeah, that's fine.

abalkin · 2010-07-07T20:28:03Z

On Wed, Jul 7, 2010 at 4:19 PM, Benjamin Peterson
<report@bugs.python.org> wrote:

> - PyStructSequence_New does not fill ob_item array with NULLs.

I'm not sure it really matters, since the fields should always be
filled in, but I suppose it can't hurt...

The following pattern is quite common:

tuple = PyTuple_New(size):

for (i = 0; i < size; ++i) {
   item = create_item(...);
   if (item == NULL)
      goto fail;
}
...
fail:
Py_DECREF(tuple);

I don't think this should be outlawed.

benjaminp · 2010-07-07T20:30:37Z

(New patch addresses review.)

abalkin · 2010-07-07T20:43:29Z

Looks great. I love the patches where few '+'s are drowning in a sea of '-'s!

A gratuitous nitpick (feel free to ignore). In

+        val = PyStructSequence_GET_ITEM(obj, i);
         if (cname == NULL || val == NULL) {

The null check for val is now redundant, but won't hurt.

benjaminp · 2010-07-07T20:54:27Z

Applied in r82636.

Arfrever · 2010-07-07T21:02:29Z

I would suggest to document changed behavior of string interpolation with sys.version_info and sys.getwindowsversion() in "What’s New in Python 2.7" page and maybe "What’s New In Python 3.1" page.

rhettinger · 2010-07-07T21:20:08Z

This was a fantastic check-in. Thanks Benjamin.

briancurtin · 2010-07-08T17:22:43Z

A side effect of this change is that it kills the ability to have a PyStructSequence which has a smaller visible size than the total number of items. For example, sys.getwindowsversion used to have 5 items in the sequence and 4 items accessible by name only (for backwards compatibility) -- now it's a 9 item tuple.

abalkin · 2010-07-08T19:33:04Z

A side effect of this change is that it kills the ability to have a
PyStructSequence which has a smaller visible size than the total
number of items.

Hmm. I was looking for this precise issue when I was reviewing the code and it appeared right to me. I know, I should have tested instead. This should be easy to fix, though. I'll look into it now.

Arfrever mannequin added the stdlib Python modules in the Lib dir label Apr 16, 2010

benjaminp assigned ericvsmith Apr 16, 2010

ericvsmith added interpreter-core Interpreter core (Objects, Python, Grammar, and Parser dirs) type-bug An unexpected behavior, bug, or error and removed stdlib Python modules in the Lib dir labels Apr 16, 2010

benjaminp assigned benjaminp and unassigned ericvsmith Jul 7, 2010

benjaminp closed this as completed Jul 7, 2010

ezio-melotti transferred this issue from another repository Apr 10, 2022

String interpolation doesn't work with sys.version_info #52660

String interpolation doesn't work with sys.version_info #52660

Arfrever mannequin commented Apr 16, 2010

Arfrever mannequin commented Apr 16, 2010

ericvsmith commented Apr 16, 2010

ericvsmith commented Apr 21, 2010

AlexanderBelopolsky mannequin commented Apr 29, 2010

pitrou commented Apr 29, 2010

ericvsmith commented Apr 29, 2010

AlexanderBelopolsky mannequin commented Apr 29, 2010

ericvsmith commented Apr 29, 2010

rhettinger commented Apr 29, 2010

ericvsmith commented Apr 29, 2010

AlexanderBelopolsky mannequin commented Apr 29, 2010

Arfrever mannequin commented Jun 24, 2010

benjaminp commented Jul 7, 2010

abalkin commented Jul 7, 2010

benjaminp commented Jul 7, 2010

abalkin commented Jul 7, 2010

benjaminp commented Jul 7, 2010

abalkin commented Jul 7, 2010

benjaminp commented Jul 7, 2010

Arfrever mannequin commented Jul 7, 2010

rhettinger commented Jul 7, 2010

briancurtin commented Jul 8, 2010

abalkin commented Jul 8, 2010

String interpolation doesn't work with sys.version_info #52660

String interpolation doesn't work with sys.version_info #52660

Comments

Arfrever mannequin commented Apr 16, 2010

Arfrever mannequin commented Apr 16, 2010

ericvsmith commented Apr 16, 2010

ericvsmith commented Apr 21, 2010

AlexanderBelopolsky mannequin commented Apr 29, 2010

pitrou commented Apr 29, 2010

ericvsmith commented Apr 29, 2010

AlexanderBelopolsky mannequin commented Apr 29, 2010

ericvsmith commented Apr 29, 2010

rhettinger commented Apr 29, 2010

ericvsmith commented Apr 29, 2010

AlexanderBelopolsky mannequin commented Apr 29, 2010

Arfrever mannequin commented Jun 24, 2010

benjaminp commented Jul 7, 2010

abalkin commented Jul 7, 2010

benjaminp commented Jul 7, 2010

abalkin commented Jul 7, 2010

benjaminp commented Jul 7, 2010

abalkin commented Jul 7, 2010

benjaminp commented Jul 7, 2010

Arfrever mannequin commented Jul 7, 2010

rhettinger commented Jul 7, 2010

briancurtin commented Jul 8, 2010

abalkin commented Jul 8, 2010