New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
automembership rebuild task can hang server #845
Comments
Comment from mreynolds (@mreynolds389) at 2013-09-11 22:12:12 This appears to be a race condition with schema-compat plugin. There are two threads 33 and 2. In thread 2 we take the backend serial lock in ldbm_back_modify(). Then thread 33 takes the map_wrlock() when adding the automember rebuild task, then thread 2 tries to take the map_rdlock(), and its blocked by thread 33. But thread 33 is waiting on the backend serial lock that thread 2 holds, and we deadlock.
|
Comment from mreynolds (@mreynolds389) at 2013-09-12 23:38:22 The automember rebuild task was being performed outside a transaction, and it appears this is the cause of the deadlock. linked attribute plugin also does its fixup task outside a transaction. Still testing... |
Comment from mreynolds (@mreynolds389) at 2013-09-14 01:21:55 Hang still occurs with automember task being done in a txn. |
Comment from nkinder (@nkinder) at 2013-09-29 09:04:43 This turned out to be a problem in slapi-nis, not in 389 DS. Closing this ticket. |
Comment from mreynolds (@mreynolds389) at 2017-02-11 22:49:23 Metadata Update from @mreynolds389:
|
Cloned from Pagure issue: https://pagure.io/389-ds-base/issue/47508
Thread 33 (Thread 0x7ff017fff700 (LWP 6572)):
0 0x00007ff037105575 in pthread_cond_wait@@GLIBC_2.3.2 () from /usr/lib64/libpthread.so.0
1 0x00007ff031c9af63 in __db_hybrid_mutex_suspend () from /usr/lib64/libdb-5.3.so
2 0x00007ff031c9a440 in __db_tas_mutex_lock () from /usr/lib64/libdb-5.3.so
3 0x00007ff031d3f85a in __lock_get_internal () from /usr/lib64/libdb-5.3.so
4 0x00007ff031d40460 in __lock_get () from /usr/lib64/libdb-5.3.so
5 0x00007ff031d6ac2a in __db_lget () from /usr/lib64/libdb-5.3.so
6 0x00007ff031cb74f4 in __bam_search () from /usr/lib64/libdb-5.3.so
7 0x00007ff031ca2996 in __bamc_search () from /usr/lib64/libdb-5.3.so
8 0x00007ff031ca43af in __bamc_get () from /usr/lib64/libdb-5.3.so
9 0x00007ff031d57ff6 in __dbc_iget () from /usr/lib64/libdb-5.3.so
10 0x00007ff031d66704 in __dbc_get_pp () from /usr/lib64/libdb-5.3.so
11 0x00007ff02debf2a0 in idl_new_fetch (be=0x7ff03b61f550, db=, inkey=0x7ff017fed180, txn=0x0, a=0x7ff03b6dbaa0, flag_err=0x7ff017ff3dec,
allidslimit=100000) at ldap/servers/slapd/back-ldbm/idl_new.c:231
12 0x00007ff02debef2d in idl_fetch_ext (be=be@entry=0x7ff03b61f550, db=, key=key@entry=0x7ff017fed180, txn=txn@entry=0x0, a=,
err=err@entry=0x7ff017ff3dec, allidslimit=allidslimit@entry=100000) at ldap/servers/slapd/back-ldbm/idl_shim.c:130
13 0x00007ff02decce3c in index_read_ext_allids (be=be@entry=0x7ff03b61f550, type=type@entry=0x7ff03b7bdc80 "member",
indextype=indextype@entry=0x7ff02df0d05b "eq", val=, txn=txn@entry=0x7ff017ff1420, err=err@entry=0x7ff017ff3dec,
unindexed=unindexed@entry=0x7ff017ff1414, allidslimit=allidslimit@entry=100000) at ldap/servers/slapd/back-ldbm/index.c:1023
14 0x00007ff02deb8354 in keys2idl (be=be@entry=0x7ff03b61f550, type=0x7ff03b7bdc80 "member", indextype=indextype@entry=0x7ff02df0d05b "eq",
ivals=, err=err@entry=0x7ff017ff3dec, unindexed=unindexed@entry=0x7ff017ff1414, txn=txn@entry=0x7ff017ff1420,
allidslimit=allidslimit@entry=100000) at ldap/servers/slapd/back-ldbm/filterindex.c:964
15 0x00007ff02deb8a73 in ava_candidates (pb=pb@entry=0x7ff03b9dc300, be=be@entry=0x7ff03b61f550, f=f@entry=0x7ff03b9dc760, ftype=,
err=0x7ff017ff3dec, allidslimit=100000, range=0, nextf=0x0) at ldap/servers/slapd/back-ldbm/filterindex.c:316
16 0x00007ff02deb910d in filter_candidates_ext (pb=pb@entry=0x7ff03b9dc300, be=be@entry=0x7ff03b61f550,
base=base@entry=0x7ff03b9e6030 "cn=groups,cn=accounts,dc=idm,dc=lab,dc=eng,dc=brq,dc=redhat,dc=com", f=f@entry=0x7ff03b9dc760, nextf=nextf@entry=0x0,
range=range@entry=0, err=err@entry=0x7ff017ff3dec, allidslimit=allidslimit@entry=100000) at ldap/servers/slapd/back-ldbm/filterindex.c:139
17 0x00007ff02deb9feb in list_candidates (pb=pb@entry=0x7ff03b9dc300, be=be@entry=0x7ff03b61f550,
base=base@entry=0x7ff03b9e6030 "cn=groups,cn=accounts,dc=idm,dc=lab,dc=eng,dc=brq,dc=redhat,dc=com", flist=flist@entry=0x7ff03b9dc8d0, ftype=,
err=0x7ff017ff3dec, allidslimit=100000) at ldap/servers/slapd/back-ldbm/filterindex.c:805
18 0x00007ff02deb8fdb in filter_candidates_ext (pb=pb@entry=0x7ff03b9dc300, be=be@entry=0x7ff03b61f550,
base=base@entry=0x7ff03b9e6030 "cn=groups,cn=accounts,dc=idm,dc=lab,dc=eng,dc=brq,dc=redhat,dc=com", f=f@entry=0x7ff03b9dc8d0, nextf=nextf@entry=0x0,
range=range@entry=0, err=err@entry=0x7ff017ff3dec, allidslimit=allidslimit@entry=100000) at ldap/servers/slapd/back-ldbm/filterindex.c:172
19 0x00007ff02deb9feb in list_candidates (pb=pb@entry=0x7ff03b9dc300, be=be@entry=0x7ff03b61f550,
base=base@entry=0x7ff03b9e6030 "cn=groups,cn=accounts,dc=idm,dc=lab,dc=eng,dc=brq,dc=redhat,dc=com", flist=flist@entry=0x7ff03b9dc9f0, ftype=,
err=0x7ff017ff3dec, allidslimit=100000) at ldap/servers/slapd/back-ldbm/filterindex.c:805
20 0x00007ff02deb8fdb in filter_candidates_ext (pb=pb@entry=0x7ff03b9dc300, be=be@entry=0x7ff03b61f550,
base=base@entry=0x7ff03b9e6030 "cn=groups,cn=accounts,dc=idm,dc=lab,dc=eng,dc=brq,dc=redhat,dc=com", f=0x7ff03b9dc9f0, nextf=nextf@entry=0x0,
range=range@entry=0, err=err@entry=0x7ff017ff3dec, allidslimit=100000, allidslimit@entry=0) at ldap/servers/slapd/back-ldbm/filterindex.c:172
21 0x00007ff02deba4ca in filter_candidates (pb=pb@entry=0x7ff03b9dc300, be=be@entry=0x7ff03b61f550,
base=base@entry=0x7ff03b9e6030 "cn=groups,cn=accounts,dc=idm,dc=lab,dc=eng,dc=brq,dc=redhat,dc=com", f=, nextf=nextf@entry=0x0,
range=range@entry=0, err=err@entry=0x7ff017ff3dec) at ldap/servers/slapd/back-ldbm/filterindex.c:203
22 0x00007ff02def0384 in onelevel_candidates (err=0x7ff017ff3dec, lookup_returned_allidsp=0x7ff017ff3ddc, managedsait=, filter=,
e=0x7ff03b7a93a0, base=0x7ff03b9e6030 "cn=groups,cn=accounts,dc=idm,dc=lab,dc=eng,dc=brq,dc=redhat,dc=com", be=0x7ff03b61f550, pb=0x7ff03b9dc300)
at ldap/servers/slapd/back-ldbm/ldbm_search.c:1116
23 build_candidate_list (candidates=0x7ff017ff3e18, lookup_returned_allidsp=0x7ff017ff3ddc, scope=,
base=0x7ff03b9e6030 "cn=groups,cn=accounts,dc=idm,dc=lab,dc=eng,dc=brq,dc=redhat,dc=com", e=, be=0x7ff03b61f550, pb=0x7ff03b9dc300)
at ldap/servers/slapd/back-ldbm/ldbm_search.c:983
24 ldbm_back_search (pb=0x7ff03b9dc300) at ldap/servers/slapd/back-ldbm/ldbm_search.c:661
25 0x00007ff039165590 in op_shared_search (pb=pb@entry=0x7ff03b9dc300, send_result=send_result@entry=1) at ldap/servers/slapd/opshared.c:803
26 0x00007ff0391727ae in search_internal_callback_pb (pb=0x7ff03b9dc300, callback_data=, prc=0x0,
psec=0x7ff02c14f570 <backend_shr_note_entry_sdn_cb>, prec=0x0) at ldap/servers/slapd/plugin_internal_op.c:812
27 0x00007ff02c1511a1 in backend_shr_update_references_cb () from /usr/lib64/dirsrv/plugins/schemacompat-plugin.so
28 0x00007ff02c15ce3f in map_data_foreach_map () from /usr/lib64/dirsrv/plugins/schemacompat-plugin.so
29 0x00007ff02c14f41b in backend_shr_update_references () from /usr/lib64/dirsrv/plugins/schemacompat-plugin.so
30 0x00007ff02c1502ba in backend_shr_add_cb.part.14 () from /usr/lib64/dirsrv/plugins/schemacompat-plugin.so
31 0x00007ff02c1503d1 in backend_shr_betxn_post_add_cb () from /usr/lib64/dirsrv/plugins/schemacompat-plugin.so
32 0x00007ff03916e7d5 in plugin_call_func (list=0x7ff03b4ee850, operation=operation@entry=560, pb=pb@entry=0x7ff03b9e9780, call_one=call_one@entry=0)
at ldap/servers/slapd/plugin.c:1453
33 0x00007ff03916e93a in plugin_call_list (pb=0x7ff03b9e9780, operation=560, list=) at ldap/servers/slapd/plugin.c:1415
34 plugin_call_plugins (pb=pb@entry=0x7ff03b9e9780, whichfunction=whichfunction@entry=560) at ldap/servers/slapd/plugin.c:398
35 0x00007ff0391333ea in dse_add (pb=0x7ff03b9e9780) at ldap/servers/slapd/dse.c:2235
36 0x00007ff03911ee4a in op_shared_add (pb=pb@entry=0x7ff03b9e9780) at ldap/servers/slapd/add.c:735
37 0x00007ff039120108 in do_add (pb=0x7ff03b9e9780) at ldap/servers/slapd/add.c:258
38 0x00007ff03962cd98 in connection_dispatch_operation (pb=, op=0x7ff03b9e9a30, conn=0x7ff024cf82f0) at ldap/servers/slapd/connection.c:582
39 connection_threadmain () at ldap/servers/slapd/connection.c:2348
40 0x00007ff03775feb1 in _pt_root () from /usr/lib64/libnspr4.so
41 0x00007ff037101c53 in start_thread () from /usr/lib64/libpthread.so.0
42 0x00007ff036e2ed3d in clone () from /usr/lib64/libc.so.6
IPA's schema-compat plugin is in the mix. Not sure if it is related yet or not.
To reproduce:
The text was updated successfully, but these errors were encountered: