Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Pickling OperationalError causes segfault with Python 2.7.3 #170

Closed
psycoteam opened this issue Jul 12, 2013 · 4 comments
Closed

Pickling OperationalError causes segfault with Python 2.7.3 #170

psycoteam opened this issue Jul 12, 2013 · 4 comments

Comments

@psycoteam
Copy link

Originally submitted by: Chris Schmich

When attempting to pickle an OperationalError, I get an immediate segfault. I understand that OperationalError might not be pickle-able, but I would expect a separate Python exception as a result, not a segfault.

  • Host: Ubuntu Linux 3.5.0-34-generic SMP i686
  • Python: 2.7.3
  • psycopg2: 2.5

Minimal repro script:

@@@ python
import pickle
from psycopg2 import connect, OperationalError

print 'Start.'
try:
conn = connect("host='doesnotexist' dbname='foo'")
print 'Connected.'
except OperationalError as e:
print 'Exception.'
pickled = pickle.dumps(e)
print 'Pickled.'

print 'Fin.'

Output:

Start.
Exception.
Segmentation fault (core dumped)

When debugging with gdb:

Program received signal SIGSEGV, Segmentation fault.
dict_set_item_by_hash_or_entry (value=0x0, ep=0x0, hash=-2121493956, key='pgerror', op={})
at ../Objects/dictobject.c:762
762 ../Objects/dictobject.c: No such file or directory.

(gdb) bt
#230 dict_set_item_by_hash_or_entry (value=0x0, ep=0x0, hash=-2121493956, key='pgerror', op={})

at ../Objects/dictobject.c:762

#231 PyDict_SetItem (op=op@entry={}, key='pgerror', value=value@entry=0x0) at ../Objects/dictobject.c:818
#232 0x080f482f in PyDict_SetItemString (v={}, key=key@entry=0xb7ac6b9b "pgerror", item=0x0)

at ../Objects/dictobject.c:2438

#233 0xb7ab9990 in psyco_error_reduce (self=0x838ef7c) at psycopg/error_type.c:166
#234 0x080e4975 in PyObject_Call (kw=0x0, arg=(), func=

<built-in method __reduce__ of OperationalError object at remote 0x838ef7c>) at ../Objects/abstract.c:2529

#235 PyEval_CallObjectWithKeywords (func=func@entry=

<built-in method __reduce__ of OperationalError object at remote 0x838ef7c>, arg=(), arg@entry=0x0, kw=kw@entry=
0x0) at ../Python/ceval.c:3890

#236 0x0814869f in PyObject_CallObject (o=o@entry=

<built-in method __reduce__ of OperationalError object at remote 0x838ef7c>, a=a@entry=0x0)
at ../Objects/abstract.c:2517

#237 0x08115568 in object_reduce_ex.25465 (self=<OperationalError at remote 0x838ef7c>, args=(0,))

at ../Objects/typeobject.c:3412

#238 0x080a517b in call_function (oparg=, pp_stack=0xbfffeb9c) at ../Python/ceval.c:4021
#239 PyEval_EvalFrameEx (f=f@entry=

Frame 0x83ec5a4, for file /usr/lib/python2.7/pickle.py, line 306, in save (self=<Pickler(write=<built-in method write of cStringIO.StringO object at remote 0xb7d02540>, bin=False, memo={}, fast=0, proto=0) at remote 0x839646c>, obj=<OperationalError at remote 0x838ef7c>, pid=None, x=None, t=<type at remote 0x83702bc>, f=None, reduce=<built-in method __reduce_ex__ of OperationalError object at remote 0x838ef7c>, issc=False), throwflag=throwflag@entry=0)
at ../Python/ceval.c:2666

#240 0x080a5728 in fast_function (nk=, na=, n=2, pp_stack=0xbfffec8c, func=

<function at remote 0xb7cfe924>) at ../Python/ceval.c:4107

#241 call_function (oparg=, pp_stack=0xbfffec8c) at ../Python/ceval.c:4042
#242 PyEval_EvalFrameEx (f=f@entry=

Frame 0x83c39a4, for file /usr/lib/python2.7/pickle.py, line 224, in dump (self=<Pickler(write=<built-in method write of cStringIO.StringO object at remote 0xb7d02540>, bin=False, memo={}, fast=0, proto=0) at remote 0x839646c>, obj=<OperationalError at remote 0x838ef7c>), throwflag=throwflag@entry=0) at ../Python/ceval.c:2666

#243 0x080a5728 in fast_function (nk=, na=, n=2, pp_stack=0xbfffed7c, func=

<function at remote 0xb7cfe844>) at ../Python/ceval.c:4107

#244 call_function (oparg=, pp_stack=0xbfffed7c) at ../Python/ceval.c:4042
#245 PyEval_EvalFrameEx (f=f@entry=

Frame 0x83fa114, for file /usr/lib/python2.7/pickle.py, line 1374, in dumps (obj=<OperationalError at remote 0x838ef7c>, protocol=None, file=<cStringIO.StringO at remote 0xb7d02540>), throwflag=throwflag@entry=0)
at ../Python/ceval.c:2666

#246 0x080abac9 in PyEval_EvalCodeEx (co=0xb7cfca88, globals=globals@entry=

{'EMPTY_DICT': '}', 'NEWTRUE': '\x88', 'TypeType': <type at remote 0x8281c80>, 'Unpickler': <classobj at remote 0xb7cedecc>, 'whichmodule': <function at remote 0xb7cfed4c>, 'BuiltinFunctionType': <type at remote 0x8281740>, 'CodeType': <type at remote 0x82814a0>, 'TRUE': 'I01\n', 'SETITEMS': 'u', 'struct': <module at remote 0xb7cf93ec>, 'LONG': 'L', 'DictProxyType': <type at remote 0x827be00>, '_EmptyClass': <classobj at remote 0xb7cedefc>, 'GET': 'g', 'ObjectType': <type at remote 0x827e3a0>, 'DictType': <type at remote 0x8281ac0>, 'EXT4': '\x84', 'EXT2': '\x83', 'EXT1': '\x82', '__file__': '/usr/lib/python2.7/pickle.pyc', 'BUILD': 'b', 'ListType': <type at remote 0x8281ba0>, 'MethodType': <type at remote 0x827b700>, 'mloads': <built-in function loads>, 'ModuleType': <type at remote 0x8281580>, 'TracebackType': <type at remote 0x8281380>, 'dumps': <function at remote 0xb7d01bc4>, 'loads': <function at remote 0xb7d01c34>, 'LambdaType': <type at remote 0x8281820>, 'XRangeType': <type at remote 0x827d220>, 'for...(truncated), locals=locals@entry=0x0, 
args=args@entry=0x82d9320, argcount=argcount@entry=1, kws=0x82d9324, kwcount=0, defs=0xb7d022b8, defcount=1, 
closure=0x0) at ../Python/ceval.c:3253

#247 0x080a57cd in fast_function (nk=, na=1, n=, pp_stack=0xbfffeeec, func=

<function at remote 0xb7d01bc4>) at ../Python/ceval.c:4117

#248 call_function (oparg=, pp_stack=0xbfffeeec) at ../Python/ceval.c:4042
#249 PyEval_EvalFrameEx (f=f@entry=Frame 0x82d91e4, for file p.py, line 10, in (), throwflag=throwflag@entry=0)

at ../Python/ceval.c:2666

#250 0x080abac9 in PyEval_EvalCodeEx (co=co@entry=0xb7d50cc8, globals=globals@entry=

{'e': <OperationalError at remote 0x838ef7c>, '__builtins__': <module at remote 0xb7d5611c>, '__file__': 'p.py', '__package__': None, 'connect': <function at remote 0x838ef0c>, '__name__': '__main__', 'pickle': <module at remote 0xb7cf93bc>, '__doc__': None, 'OperationalError': <type at remote 0x83702bc>}, locals=locals@entry=
{'e': <OperationalError at remote 0x838ef7c>, '__builtins__': <module at remote 0xb7d5611c>, '__file__': 'p.py', '__package__': None, 'connect': <function at remote 0x838ef0c>, '__name__': '__main__', 'pickle': <module at remote 0xb7cf93bc>, '__doc__': None, 'OperationalError': <type at remote 0x83702bc>}, args=args@entry=0x0, 
argcount=argcount@entry=0, kws=kws@entry=0x0, kwcount=kwcount@entry=0, defs=defs@entry=0x0, 
defcount=defcount@entry=0, closure=closure@entry=0x0) at ../Python/ceval.c:3253

#251 0x08113b57 in PyEval_EvalCode (co=co@entry=0xb7d50cc8, globals=globals@entry=

{'e': <OperationalError at remote 0x838ef7c>, '__builtins__': <module at remote 0xb7d5611c>, '__file__': 'p.py', '__package__': None, 'connect': <function at remote 0x838ef0c>, '__name__': '__main__', 'pickle': <module at remote 0xb7cf93bc>, '__doc__': None, 'OperationalError': <type at remote 0x83702bc>}, locals=locals@entry=
{'e': <OperationalError at remote 0x838ef7c>, '__builtins__': <module at remote 0xb7d5611c>, '__file__': 'p.py', '__package__': None, 'connect': <function at remote 0x838ef0c>, '__name__': '__main__', 'pickle': <module at remote 0xb7cf93bc>, '__doc__': None, 'OperationalError': <type at remote 0x83702bc>}) at ../Python/ceval.c:667

#252 0x0814ca7d in run_mod.42766 (mod=mod@entry=0x8354410, filename=filename@entry=0xbffff3c4 "p.py",

globals=globals@entry=
{'e': <OperationalError at remote 0x838ef7c>, '__builtins__': <module at remote 0xb7d5611c>, '__file__': 'p.py', '__package__': None, 'connect': <function at remote 0x838ef0c>, '__name__': '__main__', 'pickle': <module at remote 0xb7cf93bc>, '__doc__': None, 'OperationalError': <type at remote 0x83702bc>}, locals=locals@entry=
{'e': <OperationalError at remote 0x838ef7c>, '__builtins__': <module at remote 0xb7d5611c>, '__file__': 'p.py', '__package__': None, 'connect': <function at remote 0x838ef0c>, '__name__': '__main__', 'pickle': <module at remote 0xb7cf93bc>, '__doc__': None, 'OperationalError': <type at remote 0x83702bc>}, flags=flags@entry=0xbffff0ec, 
arena=arena@entry=0x82e2f88) at ../Python/pythonrun.c:1365

#253 0x0808e4d2 in PyRun_FileExFlags (fp=fp@entry=0x82d91d8, filename=filename@entry=0xbffff3c4 "p.py",

start=start@entry=257, globals=globals@entry=
{'e': <OperationalError at remote 0x838ef7c>, '__builtins__': <module at remote 0xb7d5611c>, '__file__': 'p.py', '__package__': None, 'connect': <function at remote 0x838ef0c>, '__name__': '__main__', 'pickle': <module at remote 0xb7cf93bc>, '__doc__': None, 'OperationalError': <type at remote 0x83702bc>}, locals=locals@entry=
{'e': <OperationalError at remote 0x838ef7c>, '__builtins__': <module at remote 0xb7d5611c>, '__file__': 'p.py', '__package__': None, 'connect': <function at remote 0x838ef0c>, '__name__': '__main__', 'pickle': <module at remote 0xb7cf93bc>, '__doc__': None, 'OperationalError': <type at remote 0x83702bc>}, closeit=closeit@entry=1, flags=flags@entry=
0xbffff0ec) at ../Python/pythonrun.c:1351

#254 0x0808ea80 in PyRun_SimpleFileExFlags (fp=0x82d91d8, filename=, closeit=1, flags=0xbffff0ec)

at ../Python/pythonrun.c:943

#255 0x0808eb42 in PyRun_AnyFileExFlags (fp=fp@entry=0x82d91d8, filename=filename@entry=0xbffff3c4 "p.py",

closeit=closeit@entry=1, flags=flags@entry=0xbffff0ec) at ../Python/pythonrun.c:747

#26 0x0808f917 in Py_Main (argc=argc@entry=2, argv=argv@entry=0xbffff224) at ../Modules/main.c:639
#27 0x0808f9e9 in main (argc=2, argv=0xbffff224) at ../Modules/python.c:23

(gdb) py-list
301 return
302
303 # Check for a reduce_ex method, fall back to reduce
304 reduce = getattr(obj, "reduce_ex", None)
305 if reduce:

306 rv = reduce(self.proto)
307 else:
308 reduce = getattr(obj, "reduce", None)
309 if reduce:
310 rv = reduce()
311 else:

For a little more context on this issue, I am using psycopg2 connections in some worker code running under Celery. When these errors occur, Celery attempts to pickle the exception, which causes a segfault that Python then handles and (for some reason) ignores, causing an infinite loop of segfault > trap > resume > segfault > ... and 100% CPU usage.

Thanks & regards,
Chris

P.S. While I completely respect your decision in how you manage your software, I must put in my vote for you to use GitHub's issue tracker. I say this because I am far more inclined to raise issues and support open source with a single, centralized account rather than having to create a new account for each piece of software I wish to give feedback on. Also, running into Lighthouse problems while even trying to post this issue was pretty discouraging. With that said, though, thanks for your work on this project!

@psycoteam
Copy link
Author

Originally submitted by: Chris Schmich

More details: this appears to be an issue with the base error class, psycopg2.Error. I hit a segfault when trying to pickle any of the exceptions inheriting from this ("OperationalError, InterfaceError, DatabaseError, ...":http://initd.org/psycopg/docs/module.html#exceptions ).

I was able to successfully pickle psycopg2.Warning instances.

@dvarrazzo
Copy link
Member

Chris,

thank you for your report. Yes, bug confirmed. The problem is probably with exceptions containing no result (e.g. generated on connect), because for the ones generated after db operations pickling is fine (and unit-tested):

print 'Start.'
try:
  conn = connect("")
  cur = conn.cursor()
  cur.execute("select * from nowhere")
  print 'Connected.'
except Error as e:
  print 'Exception.'
  pickled = pickle.dumps(e)
  print 'Pickled.'

I'll add your case to the test suite too.

About this issue tracker: yes, I was thinking about the problem yesterday after answering to #169. I would love to have Lighthouse history exported to github though.
Please check #171 for my thoughts (and feel free to give a hand).

@psycoteam
Copy link
Author

Originally submitted by: Chris Schmich

Hi Daniele,

Thanks for the quick response, and sorry for being so delayed in mine.

Your explanation makes sense, and it's consistent with what I'm seeing.

I'd be happy to help with the migration to GitHub (I started watching issue #171). I don't want to overlap with any work you guys are doing, so if there's a specific task you'd like me to do (e.g. writing an automation script to copy tickets from Lighthouse to GitHub), let me know.

Thanks again!
Chris

@dvarrazzo
Copy link
Member

Issue fixed in my branch: dvarrazzo/psycopg@0e08fbb

To be released in 2.5.2.

joulez pushed a commit to joulez/psycopg2 that referenced this issue Nov 13, 2014
gencer pushed a commit to gencer/psycopg2 that referenced this issue Aug 22, 2016
This issue was closed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants