sqlite3.Row doesn't support sequence protocol

[pfalcon]

BPO	10203
Nosy	@malemburg, @pfalcon, @pitrou, @merwok, @PCManticore, @serhiy-storchaka, @vajrasky
Files	sqlite1.patch sqlite2.patch issue10203.patch issue10203_1.patch issue10203_2.patch

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/serhiy-storchaka'
closed_at = <Date 2014-05-28.10:06:50.390>
created_at = <Date 2010-10-26.19:11:31.496>
labels = ['type-bug', 'library']
title = "sqlite3.Row doesn't support sequence protocol"
updated_at = <Date 2014-05-28.10:06:50.383>
user = 'https://github.com/pfalcon'

bugs.python.org fields:

activity = <Date 2014-05-28.10:06:50.383>
actor = 'serhiy.storchaka'
assignee = 'serhiy.storchaka'
closed = True
closed_date = <Date 2014-05-28.10:06:50.390>
closer = 'serhiy.storchaka'
components = ['Library (Lib)']
creation = <Date 2010-10-26.19:11:31.496>
creator = 'pfalcon'
dependencies = []
files = ['32902', '32968', '35354', '35373', '35376']
hgrepos = []
issue_num = 10203
keywords = ['patch', 'needs review']
message_count = 23.0
messages = ['119639', '119698', '119699', '119702', '204789', '205172', '205195', '205197', '205201', '213865', '213868', '213876', '213877', '213880', '217441', '219099', '219103', '219176', '219179', '219206', '219208', '219260', '219261']
nosy_count = 11.0
nosy_names = ['lemburg', 'pfalcon', 'ghaering', 'pitrou', 'eric.araujo', 'jesstess', 'Claudiu.Popa', 'BreamoreBoy', 'python-dev', 'serhiy.storchaka', 'vajrasky']
pr_nums = []
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue10203'
versions = ['Python 2.7', 'Python 3.4', 'Python 3.5']

[pfalcon]

sqlite.Row class doesn't implement sequence protocol, which is rather unfortunate, because it is described and expected to work like a tuple, with extra mapping-like functionality.

Specific issue I hit:

Adding rows to PyGTK ListStore,

        model = gtk.ListStore(*db.getSchema())
        for r in listGen():
            model.append(r)

I get:

TypeError: expecting a sequence

Looking at PyGTK sources, append() method uses PySequence Check() on the argument. Looking at Python 2.6.5 abstract.c:

int
PySequence_Check(PyObject *s)
{
        if (s && PyInstance_Check(s))
                return PyObject_HasAttrString(s, "__getitem__");
        if (PyObject_IsInstance(s, (PyObject *)&PyDict_Type))
                return 0;
        return s != NULL && s->ob_type->tp_as_sequence &&
                s->ob_type->tp_as_sequence->sq_item != NULL;
}

And sqlite3.Row doesn't set ob_type->tp_as_sequence as of Py 2.6.5 or 2.7.

[merwok]

Hello Paul, thanks for the report. The doc only describe Row as a tuple-like object, but tuple does implement the Sequence ABC, so I’m inclined to agree with you this is a bug and not a feature request (my first reaction).

Adding Georg, the maintainer of the module, to the nosy list (found his username in Misc/maintainers.rst).

[merwok]

(Gerhard, sorry, not well awake :)

[malemburg]

Just as data point: the DB-API 2.0 requires that the row objects returned by the various .fetch*() methods are sequences, i.e. they need to implement the sequence protocol.

[PCManticore]

Hello! Here's a simple patch which makes sqlite.Row to act like a real sequence.

[merwok]

Patch looks good! Are documentation changes needed?

[PCManticore]

I guess not, the documentation already states that Row tries to mimic a tuple in most of its features. Probably a MISC/News entry is required.

[vajrasky]

I got warning in compiling your patch:

gcc -pthread -fPIC -Wno-unused-result -Werror=declaration-after-statement -g -O0 -Wall -Wstrict-prototypes -DMODULE_NAME="sqlite3" -DSQLITE_OMIT_LOAD_EXTENSION=1 -IModules/_sqlite -I/usr/include -I./Include -I. -IInclude -I/usr/include/x86_64-linux-gnu -I/usr/local/include -I/home/ethan/Documents/code/python/cpython3.4/Include -I/home/ethan/Documents/code/python/cpython3.4 -c /home/ethan/Documents/code/python/cpython3.4/Modules/_sqlite/row.c -o build/temp.linux-x86_64-3.4-pydebug/home/ethan/Documents/code/python/cpython3.4/Modules/_sqlite/row.o
/home/ethan/Documents/code/python/cpython3.4/Modules/_sqlite/row.c:212:28: warning: initialization from incompatible pointer type [enabled by default]
/home/ethan/Documents/code/python/cpython3.4/Modules/_sqlite/row.c:212:28: warning: (near initialization for ‘pysqlite_row_as_sequence.sq_item’) [enabled by default]

$ gcc --version
gcc (Ubuntu/Linaro 4.7.2-2ubuntu1) 4.7.2

[PCManticore]

Thanks, Vajrasky! Here's an updated patch.

[PCManticore]

What can be done to move this forward?

[BreamoreBoy]

I've changed the versions, hope I've got them correct.

[merwok]

Thanks Mark. There will be a last 3.3 release with bugfixes before it switches to security fixes only.

[merwok]

I would like another core developer more intimate with C to review the patch.

[merwok]

Ah, 3.3 won’t follow that custom given that it had a bugfix release very recently.

[pitrou]

Issue bpo-13583 ("sqlite3.Row doesn't support slice indexes") is partly related.

[serhiy-storchaka]

LGTM. Perhaps it is worth to add a test for negative indices (valid (-1) and invalid (< -length)).

[PCManticore]

Thanks. Here's the updated patch. It supports negative indeces (my previous patch didn't do that).

[serhiy-storchaka]

Looks good, but there is one detail. Whith bpo-10203.patch when integer index overflows C long, sqlite3.Row.__getitem__() doesn't raise an exception. Instead overflow exception is raised later.

>>> import sqlite3
>>> con = sqlite3.connect(":memory:")
>>> con.row_factory = sqlite3.Row
>>> row = con.execute("select 1 as a, 2 as b").fetchone()
>>> row[2**1000]
2
>>> 
OverflowError: Python int too large to convert to C long

[PCManticore]

Thanks. Here's a fix.

[serhiy-storchaka]

Compare with tuple:

>>> (1, 2)[2**1000]
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
IndexError: cannot fit 'int' into an index-sized integer

[PCManticore]

Thanks. Patch modified.

[python-dev]

New changeset 6e2833ae1718 by Serhiy Storchaka in branch '2.7':
Issue bpo-10203: sqlite3.Row now truly supports sequence protocol. In particular
http://hg.python.org/cpython/rev/6e2833ae1718

New changeset 6af865f1a59d by Serhiy Storchaka in branch '3.4':
Issue bpo-10203: sqlite3.Row now truly supports sequence protocol. In particulr
http://hg.python.org/cpython/rev/6af865f1a59d

New changeset 474c97a5f0c8 by Serhiy Storchaka in branch 'default':
Issue bpo-10203: sqlite3.Row now truly supports sequence protocol. In particulr
http://hg.python.org/cpython/rev/474c97a5f0c8

[serhiy-storchaka]

Thank you Claudiu for your contribution.

But please be more careful, your patches contained trailing whitespaces.

Thank you Paul for your report.