seg fault when calling bsddb.hashopen()

[smontanaro]

BPO	994100
Nosy	@smontanaro, @gpshead

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/smontanaro'
closed_at = <Date 2005-10-03.14:46:00.000>
created_at = <Date 2004-07-19.21:21:36.000>
labels = ['library']
title = 'seg fault when calling bsddb.hashopen()'
updated_at = <Date 2005-10-03.14:46:00.000>
user = 'https://github.com/smontanaro'

bugs.python.org fields:

activity = <Date 2005-10-03.14:46:00.000>
actor = 'skip.montanaro'
assignee = 'skip.montanaro'
closed = True
closed_date = None
closer = None
components = ['Library (Lib)']
creation = <Date 2004-07-19.21:21:36.000>
creator = 'skip.montanaro'
dependencies = []
files = []
hgrepos = []
issue_num = 994100
keywords = []
message_count = 7.0
messages = ['21688', '21689', '21690', '21691', '21692', '21693', '21694']
nosy_count = 3.0
nosy_names = ['skip.montanaro', 'nnorwitz', 'gregory.p.smith']
pr_nums = []
priority = 'normal'
resolution = 'fixed'
stage = None
status = 'closed'
superseder = None
type = None
url = 'https://bugs.python.org/issue994100'
versions = ['Python 2.3']

[smontanaro]

Fresh 2.3.4 build (didn't even install!) on Mandrake
8.1-ish system
using Berkeley DB 3.3 yields this output when trying to
create a new hash db:

Python 2.3.4 (#1, Jul 19 2004, 16:02:09) 
[GCC 3.0.1] on linux2
Type "help", "copyright", "credits" or "license" for
more information.
>>> import bsddb
>>> bsddb.__file__
'/home/skip/src/Python-2.3.4/Lib/bsddb/__init__.pyc'
>>> bsddb.hashopen("foo.db", "c")

Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 1024 (LWP 22763)]
0x00000000 in ?? ()
(gdb) bt
#0 0x00000000 in ?? ()
#1 0x404e284b in newDBObject (arg=0x0, flags=0)
at /home/skip/src/Python-2.3.4/Modules/_bsddb.c:697
#2 0x404ee808 in DB_construct (self=0x0,
args=0x401a602c, kwargs=0x0)
at /home/skip/src/Python-2.3.4/Modules/_bsddb.c:4322
#3 0x080f718a in PyCFunction_Call (func=0x4047344c,
arg=0x401a602c, kw=0x0)
at Objects/methodobject.c:93
#4 0x080b122d in call_function (pp_stack=0xbffff31c,
oparg=0)
at Python/ceval.c:3439
#5 0x080af680 in eval_frame (f=0x8195a0c) at
Python/ceval.c:2116
#6 0x080b0382 in PyEval_EvalCodeEx (co=0x4047ff60,
globals=0x404859bc,
locals=0x0, args=0x818e13c, argcount=2,
kws=0x818e144, kwcount=0,
defs=0x404868f8, defcount=8, closure=0x0) at
Python/ceval.c:2663
#7 0x080b2ea1 in fast_function (func=0x40486454,
pp_stack=0xbffff4bc, n=2,
na=2, nk=0) at Python/ceval.c:3529
#8 0x080b1119 in call_function (pp_stack=0xbffff4bc,
oparg=2)
at Python/ceval.c:3458
#9 0x080af680 in eval_frame (f=0x818dfec) at
Python/ceval.c:2116
#10 0x080b0382 in PyEval_EvalCodeEx (co=0x4046a920,
globals=0x401bd79c,
locals=0x401bd79c, args=0x0, argcount=0, kws=0x0,
kwcount=0, defs=0x0,
defcount=0, closure=0x0) at Python/ceval.c:2663
#11 0x080b2dd5 in PyEval_EvalCode (co=0x4046a920,
globals=0x401bd79c,
locals=0x401bd79c) at Python/ceval.c:537
#12 0x080d34ab in run_node (n=0x401a6a88,
filename=0x80f9fe8 "<stdin>",
globals=0x401bd79c, locals=0x401bd79c,
flags=0xbffff6cc)
at Python/pythonrun.c:1267
#13 0x080d185c in PyRun_InteractiveOneFlags
(fp=0x4019cf60,
filename=0x80f9fe8 "<stdin>", flags=0xbffff6cc) at
Python/pythonrun.c:757
#14 0x080d164e in PyRun_InteractiveLoopFlags
(fp=0x4019cf60,
filename=0x80f9fe8 "<stdin>", flags=0xbffff6cc) at
Python/pythonrun.c:690
#15 0x080d2c08 in PyRun_AnyFileExFlags (fp=0x4019cf60,
filename=0x80f9fe8 "<stdin>", closeit=0,
flags=0xbffff6cc)
at Python/pythonrun.c:653
#16 0x0805508c in Py_Main (argc=1, argv=0xbffff7a4) at
Modules/main.c:415
#17 0x08054a99 in main (argc=1, argv=0xbffff7a4) at
Modules/python.c:23
#18 0x400825b0 in __libc_start_main () from /lib/libc.so.6

ldd tells me this is how it's linked:

% ldd build/lib.linux-i686-2.3/_bsddb.so
libdb-3.3.so => /lib/libdb-3.3.so (0x40022000)
libgcc_s.so.1 => /usr/local/lib/libgcc_s.so.1 (0x400b2000)
libc.so.6 => /lib/libc.so.6 (0x400ba000)
/lib/ld-linux.so.2 => /lib/ld-linux.so.2 (0x80000000)

What's the oldest Berkeley DB usable with the
bsddb package? According to the README file
versions 3.1 through 4.1 are supported. According
to Modules/_bsddb.c versions 3.2 through 4.2 are
supported. According to comments in setup.py 3.1
is only partially supported. I'm wondering if maybe
the bar
has been raised even higher and 3.3 is no longer supported.

Skip

[smontanaro]

Logged In: YES
user_id=44345

I downloaded and installed BerkDB 4.2.52, then rebuilt bsddb
using it. The
results are more promising:

% ./python
Python 2.3.4 (#1, Jul 19 2004, 16:02:09) 
[GCC 3.0.1] on linux2
Type "help", "copyright", "credits" or "license" for more
information.
>>> import bsddb
>>> bsddb.__file__
'/home/skip/src/Python-2.3.4/Lib/bsddb/__init__.pyc'
>>> bsddb.hashopen("junk.db", "c")
{}
>>> 
You have new mail in /var/spool/mail/skip
% ldd build/lib.linux-i686-2.3/_bsddb.so 
	libdb-4.2.so => /usr/local/BerkeleyDB.4.2/lib/libdb-4.2.so
(0x40016000)
	libgcc_s.so.1 => /usr/local/lib/libgcc_s.so.1 (0x400d4000)
	libc.so.6 => /lib/libc.so.6 (0x400dc000)
	/lib/ld-linux.so.2 => /lib/ld-linux.so.2 (0x80000000)

[gpshead]

Logged In: YES
user_id=413

it -should- work with 3.2 and later. that said, its been a
while since i've tested using 3.2 or 3.3. Can you do one
more thing for me against your _bsddb.so that appearred to
be linked with 3.3?

    from bsddb import db
    print db.DB_VERSION_STRING
    print 'bsddb.db.version():   %s' % (db.version(), )
    print 'bsddb.db.__version__: %s' % db.__version__
    print 'bsddb.db.cvsid:       %s' % db.cvsid

just to make sure that the header files it compiled with
match the version the dynamic loader decided to link it
with. i've seen problems with that in the past.

i'll built it against 3.2 and 3.3 later tonight and run the
testsuites locally. if it doesn't work with those without
an obvious reason+fix i'll just mark them as unsupported.
4.0 has been around for a long time now.

[smontanaro]

Logged In: YES
user_id=44345

I rebuilt it with 3.3, but noticed something I had missed when
it compiled before:

gcc -pthread -DNDEBUG -g -O3 -Wall -Wstrict-prototypes -fPIC \
-fno-strict-aliasing -I/usr/local/BerkeleyDB.3.2/include \
-I. -I/home/skip/src/Python-2.3.4/./Include -I/usr/local/include \
-I/home/skip/src/Python-2.3.4/Include \
-I/home/skip/src/Python-2.3.4 -c \
/home/skip/src/Python-2.3.4/Modules/_bsddb.c -o \
build/temp.linux-i686-2.3/_bsddb.o

gcc -pthread -shared build/temp.linux-i686-2.3/_bsddb.o \
-L/usr/local/lib -ldb-3.3 -o build/lib.linux-i686-2.3/_bsddb.so

Note that it searches /usr/local/BerkeleyDB.3.2/include for include
files, but fails to search /usr/local/BerkeleyDB.3.2/lib for so files.
(There are none there, just a .a file.) Consequently, it includes
the 3.2 header files and links against the 3.3 so files. Definitely
a recipe for disaster. Here's the proof:

>>> from bsddb import db
>>> print db.DB_VERSION_STRING
Sleepycat Software: Berkeley DB 3.2.9: (January 24, 2001)
>>> print 'bsddb.db.version():   %s' % (db.version(), )
bsddb.db.version():   (3, 3, 11)
>>> print 'bsddb.db.__version__: %s' % db.__version__
bsddb.db.__version__: 4.2.0.2
>>> print 'bsddb.db.cvsid:       %s' % db.cvsid
bsddb.db.cvsid:       $Id: _bsddb.c,v 1.17.6.4 2004/03/16 07:47:45 
greg Exp $

I manually compiled without the -I flag and relinked. Opening
a new db file worked like a charm.

It would thus appear to be a combination of an incomplete 3.2
installation coupled with maybe some subtle breakage in
setup.py.

I'll leave it for you to close. I don't know if it's worth comparing
information from the header files and the shared library during
import. It certainly would have helped here though.

[gpshead]

Logged In: YES
user_id=413

I just committed a rewrite of the bsddb library+include file finding portion of python's setup.py. Could you try python CVS head on the system you had this problem on? It should now pick a properly paired header file and library version.

[nnorwitz]

Logged In: YES
user_id=33168

Skip, is this still a problem?

[smontanaro]

Logged In: YES
user_id=44345

It's been ages. I'll assume Greg just didn't close it...