Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

sqlite3.Row doesn't support sequence protocol #54412

Closed
pfalcon mannequin opened this issue Oct 26, 2010 · 23 comments
Closed

sqlite3.Row doesn't support sequence protocol #54412

pfalcon mannequin opened this issue Oct 26, 2010 · 23 comments
Assignees
Labels
stdlib Python modules in the Lib dir type-bug An unexpected behavior, bug, or error

Comments

@pfalcon
Copy link
Mannequin

pfalcon mannequin commented Oct 26, 2010

BPO 10203
Nosy @malemburg, @pfalcon, @pitrou, @merwok, @PCManticore, @serhiy-storchaka, @vajrasky
Files
  • sqlite1.patch
  • sqlite2.patch
  • issue10203.patch
  • issue10203_1.patch
  • issue10203_2.patch
  • Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = 'https://github.com/serhiy-storchaka'
    closed_at = <Date 2014-05-28.10:06:50.390>
    created_at = <Date 2010-10-26.19:11:31.496>
    labels = ['type-bug', 'library']
    title = "sqlite3.Row doesn't support sequence protocol"
    updated_at = <Date 2014-05-28.10:06:50.383>
    user = 'https://github.com/pfalcon'

    bugs.python.org fields:

    activity = <Date 2014-05-28.10:06:50.383>
    actor = 'serhiy.storchaka'
    assignee = 'serhiy.storchaka'
    closed = True
    closed_date = <Date 2014-05-28.10:06:50.390>
    closer = 'serhiy.storchaka'
    components = ['Library (Lib)']
    creation = <Date 2010-10-26.19:11:31.496>
    creator = 'pfalcon'
    dependencies = []
    files = ['32902', '32968', '35354', '35373', '35376']
    hgrepos = []
    issue_num = 10203
    keywords = ['patch', 'needs review']
    message_count = 23.0
    messages = ['119639', '119698', '119699', '119702', '204789', '205172', '205195', '205197', '205201', '213865', '213868', '213876', '213877', '213880', '217441', '219099', '219103', '219176', '219179', '219206', '219208', '219260', '219261']
    nosy_count = 11.0
    nosy_names = ['lemburg', 'pfalcon', 'ghaering', 'pitrou', 'eric.araujo', 'jesstess', 'Claudiu.Popa', 'BreamoreBoy', 'python-dev', 'serhiy.storchaka', 'vajrasky']
    pr_nums = []
    priority = 'normal'
    resolution = 'fixed'
    stage = 'resolved'
    status = 'closed'
    superseder = None
    type = 'behavior'
    url = 'https://bugs.python.org/issue10203'
    versions = ['Python 2.7', 'Python 3.4', 'Python 3.5']

    @pfalcon
    Copy link
    Mannequin Author

    pfalcon mannequin commented Oct 26, 2010

    sqlite.Row class doesn't implement sequence protocol, which is rather unfortunate, because it is described and expected to work like a tuple, with extra mapping-like functionality.

    Specific issue I hit:

    Adding rows to PyGTK ListStore,

            model = gtk.ListStore(*db.getSchema())
            for r in listGen():
                model.append(r)

    I get:

    TypeError: expecting a sequence

    Looking at PyGTK sources, append() method uses PySequence Check() on the argument. Looking at Python 2.6.5 abstract.c:

    int
    PySequence_Check(PyObject *s)
    {
            if (s && PyInstance_Check(s))
                    return PyObject_HasAttrString(s, "__getitem__");
            if (PyObject_IsInstance(s, (PyObject *)&PyDict_Type))
                    return 0;
            return s != NULL && s->ob_type->tp_as_sequence &&
                    s->ob_type->tp_as_sequence->sq_item != NULL;
    }

    And sqlite3.Row doesn't set ob_type->tp_as_sequence as of Py 2.6.5 or 2.7.

    @pfalcon pfalcon mannequin added stdlib Python modules in the Lib dir type-bug An unexpected behavior, bug, or error labels Oct 26, 2010
    @merwok
    Copy link
    Member

    merwok commented Oct 27, 2010

    Hello Paul, thanks for the report. The doc only describe Row as a tuple-like object, but tuple does implement the Sequence ABC, so I’m inclined to agree with you this is a bug and not a feature request (my first reaction).

    Adding Georg, the maintainer of the module, to the nosy list (found his username in Misc/maintainers.rst).

    @merwok
    Copy link
    Member

    merwok commented Oct 27, 2010

    (Gerhard, sorry, not well awake :)

    @malemburg
    Copy link
    Member

    malemburg commented Oct 27, 2010

    Just as data point: the DB-API 2.0 requires that the row objects returned by the various .fetch*() methods are sequences, i.e. they need to implement the sequence protocol.

    @PCManticore
    Copy link
    Mannequin

    PCManticore mannequin commented Nov 30, 2013

    Hello! Here's a simple patch which makes sqlite.Row to act like a real sequence.

    @merwok
    Copy link
    Member

    merwok commented Dec 3, 2013

    Patch looks good! Are documentation changes needed?

    @PCManticore
    Copy link
    Mannequin

    PCManticore mannequin commented Dec 4, 2013

    I guess not, the documentation already states that Row tries to mimic a tuple in most of its features. Probably a MISC/News entry is required.

    @vajrasky
    Copy link
    Mannequin

    vajrasky mannequin commented Dec 4, 2013

    I got warning in compiling your patch:

    gcc -pthread -fPIC -Wno-unused-result -Werror=declaration-after-statement -g -O0 -Wall -Wstrict-prototypes -DMODULE_NAME="sqlite3" -DSQLITE_OMIT_LOAD_EXTENSION=1 -IModules/_sqlite -I/usr/include -I./Include -I. -IInclude -I/usr/include/x86_64-linux-gnu -I/usr/local/include -I/home/ethan/Documents/code/python/cpython3.4/Include -I/home/ethan/Documents/code/python/cpython3.4 -c /home/ethan/Documents/code/python/cpython3.4/Modules/_sqlite/row.c -o build/temp.linux-x86_64-3.4-pydebug/home/ethan/Documents/code/python/cpython3.4/Modules/_sqlite/row.o
    /home/ethan/Documents/code/python/cpython3.4/Modules/_sqlite/row.c:212:28: warning: initialization from incompatible pointer type [enabled by default]
    /home/ethan/Documents/code/python/cpython3.4/Modules/_sqlite/row.c:212:28: warning: (near initialization for ‘pysqlite_row_as_sequence.sq_item’) [enabled by default]

    $ gcc --version
    gcc (Ubuntu/Linaro 4.7.2-2ubuntu1) 4.7.2

    @PCManticore
    Copy link
    Mannequin

    PCManticore mannequin commented Dec 4, 2013

    Thanks, Vajrasky! Here's an updated patch.

    @PCManticore
    Copy link
    Mannequin

    PCManticore mannequin commented Mar 17, 2014

    What can be done to move this forward?

    @BreamoreBoy
    Copy link
    Mannequin

    BreamoreBoy mannequin commented Mar 17, 2014

    I've changed the versions, hope I've got them correct.

    @merwok
    Copy link
    Member

    merwok commented Mar 17, 2014

    Thanks Mark. There will be a last 3.3 release with bugfixes before it switches to security fixes only.

    @merwok
    Copy link
    Member

    merwok commented Mar 17, 2014

    I would like another core developer more intimate with C to review the patch.

    @merwok
    Copy link
    Member

    merwok commented Mar 17, 2014

    Ah, 3.3 won’t follow that custom given that it had a bugfix release very recently.

    @pitrou
    Copy link
    Member

    pitrou commented Apr 28, 2014

    Issue bpo-13583 ("sqlite3.Row doesn't support slice indexes") is partly related.

    @serhiy-storchaka
    Copy link
    Member

    serhiy-storchaka commented May 25, 2014

    LGTM. Perhaps it is worth to add a test for negative indices (valid (-1) and invalid (< -length)).

    @PCManticore
    Copy link
    Mannequin

    PCManticore mannequin commented May 25, 2014

    Thanks. Here's the updated patch. It supports negative indeces (my previous patch didn't do that).

    @serhiy-storchaka
    Copy link
    Member

    serhiy-storchaka commented May 26, 2014

    Looks good, but there is one detail. Whith bpo-10203.patch when integer index overflows C long, sqlite3.Row.__getitem__() doesn't raise an exception. Instead overflow exception is raised later.

    >>> import sqlite3
    >>> con = sqlite3.connect(":memory:")
    >>> con.row_factory = sqlite3.Row
    >>> row = con.execute("select 1 as a, 2 as b").fetchone()
    >>> row[2**1000]
    2
    >>> 
    OverflowError: Python int too large to convert to C long

    @serhiy-storchaka serhiy-storchaka self-assigned this May 26, 2014
    @PCManticore
    Copy link
    Mannequin

    PCManticore mannequin commented May 26, 2014

    Thanks. Here's a fix.

    @serhiy-storchaka
    Copy link
    Member

    serhiy-storchaka commented May 27, 2014

    Compare with tuple:

    >>> (1, 2)[2**1000]
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    IndexError: cannot fit 'int' into an index-sized integer

    @PCManticore
    Copy link
    Mannequin

    PCManticore mannequin commented May 27, 2014

    Thanks. Patch modified.

    @python-dev
    Copy link
    Mannequin

    python-dev mannequin commented May 28, 2014

    New changeset 6e2833ae1718 by Serhiy Storchaka in branch '2.7':
    Issue bpo-10203: sqlite3.Row now truly supports sequence protocol. In particular
    http://hg.python.org/cpython/rev/6e2833ae1718

    New changeset 6af865f1a59d by Serhiy Storchaka in branch '3.4':
    Issue bpo-10203: sqlite3.Row now truly supports sequence protocol. In particulr
    http://hg.python.org/cpython/rev/6af865f1a59d

    New changeset 474c97a5f0c8 by Serhiy Storchaka in branch 'default':
    Issue bpo-10203: sqlite3.Row now truly supports sequence protocol. In particulr
    http://hg.python.org/cpython/rev/474c97a5f0c8

    @serhiy-storchaka
    Copy link
    Member

    serhiy-storchaka commented May 28, 2014

    Thank you Claudiu for your contribution.

    But please be more careful, your patches contained trailing whitespaces.

    Thank you Paul for your report.

    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Labels
    stdlib Python modules in the Lib dir type-bug An unexpected behavior, bug, or error
    Projects
    None yet
    Development

    No branches or pull requests

    4 participants