Python3.11: sqlite3.connect passes more arguments to factory()

[Steap]

Hello,

Bug report

Consider the following code snippet:

$ cat /tmp/test.py

import sqlite3


class SqliteConnection(sqlite3.Connection):
    def __init__(self, *args, **kwargs):
        print(args)
        print(kwargs)
        super(SqliteConnection, self).__init__(*args, **kwargs)


db_path = '/tmp/foo.db'
sqlite3.connect(db_path, factory=SqliteConnection)

And the execution:

$ python3.10 /tmp/test.py
('/tmp/foo.db',)
{'factory': <class '__main__.SqliteConnection'>}

$ python3.11 /tmp/test.py
('/tmp/foo.db', 5.0, 0, '', 1, <class '__main__.SqliteConnection'>, 128, 0)
{}

The default values of sqlite3.connect's arguments are passed to the factory in
Python3.11, which was not the case in Python3.10. This means that code like
this will fail:

$ cat /tmp/failure.py

import sqlite3


class SqliteConnection(sqlite3.Connection):
    def __init__(self, *args, **kwargs):
        kwargs['timeout'] = 42
        super(SqliteConnection, self).__init__(*args, **kwargs)


db_path = '/tmp/foo.db'
sqlite3.connect(db_path, factory=SqliteConnection)

$ python3.10 /tmp/failure.py
$

$ python3.11 /tmp/failure.py
Traceback (most recent call last):
  File "/tmp/failure.py", line 11, in <module>
    sqlite3.connect(db_path, factory=SqliteConnection)
  File "/tmp/failure.py", line 7, in __init__
    super(SqliteConnection, self).__init__(*args, **kwargs)
TypeError: Connection() takes at most 8 arguments (9 given)

Your environment
This behaviour appeared in 185ecdc and affects
Python 3.11 and later.

It was mentioned in #93044 (see the
second bullet point in the reporter's message) but has not been fixed.

[erlend-aasland]

Thanks for the report. This is unfortunate. Reverting the offending commit is going to be hairy; there has been a lot of changes since that point in time.

The only thing sqlite3.connect needs in order to relay the call to sqlite3.Connection.__init__ is: *args, **kwargs, and factory. Unfortunately, doing such a manoeuvre is impossible (AFAIK) with Argument Clinic. Trying to bend AC to our needs, could be an option. Another option is to just parse the factory argument manually (kinda like the old code), and pass the rest using vector call (or PyObject_Call). I've already tried to look for a way to do these things because of keyword-only argument needs in another PR (#93823); I might actually be able to adopt that approach.

@serhiy-storchaka, do you have an opinion here?

Thanks again for testing the 3.11 beta! Reports like this help us improve.

[erlend-aasland]

I've already tried to look for a way to do these things because of keyword-only argument needs in another PR (#93823); I might actually be able to adopt that approach.

FTR, this won't work. Arguments will still be passed incorrectly.

[erlend-aasland]

I've partially reverted 185ecdc in #95146.

[erlend-aasland]

I've updated #95146 to use METH_FASTCALL and PyObject_Vectorcall.

[erlend-aasland]

Fixed in:

• main: gh-95132: Correctly relay *args and **kwds from sqlite3.connect to factory #95146
• 3.11: [3.11] gh-95132: Correctly relay *args and **kwds from sqlite3.connect to factory (GH-95146) #95158

[vstinner]

Nice fix.