assertion failed in pqpath.c

[psycoteam]

Originally submitted by: Edouard Chamillard

rel-2.5.1 os-linux

celery tasks are using psycopg through sqlalchemy from two different machines. on one it works fine, on the other it segfaults. after a bit of digging a recompiling stuff with debug enabled i got

[5538] pq_execute: entering synchronous DBAPI compatibility mode
[5538] pq_fetch: pgstatus = PGRES_TUPLES_OK
[5538] pq_fetch: got tuples
[5538] _pq_fetch_tuples: looking for cast 26:
[5538] curs_get_cast:        per-connection dict: (nil)
[5538] curs_get_cast:        global dict: 0x7f3b18ad0628
python3.3: psycopg/pqpath.c:1139: _pq_fetch_tuples: Assertion `((((((PyObject*)(((typecastObject*)cast)->name))->ob_type))->tp_flags & ((1L<<27))) != 0)' failed.
Aborted (core dumped)

full trace attached.

[psycoteam]

Originally submitted by: Edouard Chamillard

for reference this is the relevant pqpath.c code block

@@@ c
/* else if we got binary tuples and if we got a field that
is binary use the default cast
FIXME: what the hell am I trying to do here? This just can't work..
*/
if (pgbintuples && cast == psyco_default_binary_cast) {
Dprintf("_pq_fetch_tuples: Binary cursor and "
"binary field: %i using default cast",
PQftype(curs->pgres,i));
cast = psyco_default_cast;
}

[dvarrazzo]

I've not been able to reproduce the issue, although I've run psycopg test suite againts Python 3.3.

I don't think the block you have reported is the correct one. The one to blame looks:

    Dprintf("_pq_fetch_tuples: using cast at %p (%s) for type %d",
            cast, Bytes_AS_STRING(((typecastObject*)cast)->name),
            PQftype(curs->pgres,i));
    Py_INCREF(cast);
    PyTuple_SET_ITEM(casts, i, cast);

there is indeed some sort of error as running cur.execute("select 1") in Python 3.3 64 bit I see

[12119] pq_execute: entering synchronous DBAPI compatibility mode
[12119] pq_fetch: pgstatus = PGRES_TUPLES_OK
[12119] pq_fetch: got tuples
[12119] _pq_fetch_tuples: looking for cast 23:
[12119] curs_get_cast:        per-connection dict: (nil)
[12119] curs_get_cast:        global dict: 0x7f4b035edb50
[12119] _pq_fetch_tuples: using cast at 0x7f4b035edb50 (�) for type 23
[12119] pq_fetch: fetching done; check for critical errors
[12119] psyco_curs_execute: res = 1, pgres = 0x1929ce0

the ? at "using cast" is probably the result of a bad dereferencing; probably you are running it in a debug version of python with asserts enabled.

I think the above idiom is broken for all Python 3. Mmm... too bad I've overlooked the issue and just released.

[psycoteam]

Originally submitted by: Edouard Chamillard

the trace was indeed produced by a debug version of python. originally gdb pointed to pqpath.c:1140 which is Py_INCREF(cast);

i can try to reproduce the issue and link the whole coredump if you need.

[dvarrazzo]

A script to reproduce the issue would be preferred. A patch to fix the issue would be even better accepted, above all if checking the issue implies recompiling python in debug mode. You'd save me quite some time.

I've not understood yet what is the problem though. The problem in the Dprintf is probably just that the name is an unicode object under Py3. But why the incref should fail? curs_get_cast() doesn't seem able to return NULL.

[dvarrazzo]

Fixed in master and maint_2_5. As I expected it was the bad use of Bytes_AS_STRING causing the assertion failure. Tested on a debug build Py3.4. Also found an unrelated bug exposed by debug build, so good stuff.

If you can test everything works in your environment it would be great, thank you.