SEGFAULT when subscribing to change notifications without importing threading

[P403n1x87]

Answer the following questions:

1. What is your version of Python? Is it 32-bit or 64-bit? Python 3.6.3 (64-bit)

2. What is your cx_Oracle version? 7.1.1

3. What exact command caused the problem (e.g. what command did you try to install with)? Who were you logged in as? N.A.

4. What error(s) you are seeing? SEGFAULT

5. What OS (and version) is Python executing on? Oracle Linux Server 7.6 (under systemd-nspawn on Ubuntu 18.04.2 LTS)

6. What is your version of the Oracle client (e.g. Instant Client)? How was it installed? Where is it installed?
   SQL*Plus: Release 18.0.0.0.0 - Production on Mon Mar 4 12:48:41 2019
   Version 18.4.0.0.0
   Installed from repositories

7. What is your Oracle Database version?
   Oracle XE 18c installed from the OTN RPM

8. What is the PATH environment variable (on Windows) or LD_LIBRARY_PATH (on Linux) set to? On macOS, what is in ~/lib?
   LD_LIBRARY_PATH="/lib64:/opt/oracle/product/18c/dbhomeXE/lib"
   No ~/lib folder

9. What Oracle environment variables did you set? How exactly did you set them?
   ORACLE_SID=XE
   ORACLE_BASE=/opt/oracle
   ORAENV_ASK=NO
   ORACLE_HOME=/opt/oracle/product/18c/dbhomeXE

10. Do you have a small, single Python script that immediately runs to show us the problem?

import cx_Oracle
# import threading  # Uncomment this to "fix" SEGFAULT

def callback(msg):
	print(msg)

db = cx_Oracle.Connection("g", "g", "XE", events=True)

cursor = db.cursor()
try:
	cursor.execute("drop table temp_sub")
except cx_Oracle.DatabaseError:
	pass
cursor = db.cursor()
cursor.execute("create table temp_sub(id number(3))")

sub = db.subscribe(callback=callback, ipAddress="127.0.0.1")
sub.registerquery("select * from temp_sub")

cursor.executemany(
	"insert into temp_sub values (:1)",
	[(i,) for i in range(10)]
)

cursor.execute("drop table temp_sub")
db.close()

[P403n1x87]

I understand from https://github.com/oracle/oracle-db-examples/blob/master/python/DatabaseChangeNotification.py that the threading module should be imported. However, getting a SEGFAULT when the module is not imported could be confusing. Ideally, it would be great if the module could be automatically imported lazily or the SEGFAULT prevented and turned into an informative exception.

[P403n1x87]

From valgrind

==1865== Process terminating with default action of signal 11 (SIGSEGV)
==1865==    at 0x536C49B: raise (in /usr/lib64/libpthread-2.17.so)
==1865==    by 0xA27B511: skgesigOSCrash (in /opt/oracle/product/18c/dbhomeXE/lib/libclntsh.so.18.1)
==1865==    by 0xA913BF4: kpeDbgSignalHandler (in /opt/oracle/product/18c/dbhomeXE/lib/libclntsh.so.18.1)
==1865==    by 0xA27B84F: skgesig_sigactionHandler (in /opt/oracle/product/18c/dbhomeXE/lib/libclntsh.so.18.1)
==1865==    by 0x536C5CF: ??? (in /usr/lib64/libpthread-2.17.so)
==1865==    by 0x5366C2F: ??? (in /usr/lib64/libpthread-2.17.so)
==1865==    by 0x68884B7: dpiOci__collSize (dpiOci.c:916)
==1865==    by 0x689AAFC: dpiSubscr__populateObjectChangeMessage (dpiSubscr.c:345)
==1865==    by 0x689AAFC: dpiSubscr__populateMessage (dpiSubscr.c:410)
==1865==    by 0x689AAFC: dpiSubscr__callback (dpiSubscr.c:49)
==1865==    by 0x8EA23CE: kpunDataCallback (in /opt/oracle/product/18c/dbhomeXE/lib/libclntsh.so.18.1)
==1865==    by 0x94C6415: kpcerecv (in /opt/oracle/product/18c/dbhomeXE/lib/libclntsh.so.18.1)
==1865==    by 0x5364DD4: start_thread (in /usr/lib64/libpthread-2.17.so)
==1865==    by 0x5D7FF6C: clone (in /usr/lib64/libc-2.17.so)

[P403n1x87]

Playing around a bit more with the above snippet, I noticed that even importing threading causes SEGFAULTs. Hence the problem looks more like a race condition (I even reached what seemed to be a deadlock in one of the execution runs).

[anthony-tuininga]

The problem in this case lies in your code. You are closing the connection while activity may still be ongoing. That's the race condition: whether the closing of the database occurs before the message is sent to the callback. So you need to make sure the subscription is destroyed before closing the database. Adding the line

db.unsubscribe(sub)

before calling db.close() should also "resolve" the segfault and the hangs. Without it I was able to get the segfault occasionally and looking at the traceback showed that the database was in the middle of being closed.

I've changed this to an enhancement -- ideally it would be nice to prevent this situation. I'm not sure exactly how at the moment but I'll think about it some more. In the meantime, you have a solution!

[P403n1x87]

Thanks, that makes sense. Although I would have expected that the close method is taking care of any necessary "cleanup". Perhaps the list of current subscription objects could be swept and the unsubscribe method called on each of them as part of Connection.close()?

[anthony-tuininga]

Yeah, that would make sense....but unfortunately the subscriptions live independently of the connections -- if they are pooled. So the subscription is tied to a standalone connection or to a pool of connections, not directly to a single connection in all cases. So its a bit more complex, but probably still doable -- and as you say, you expect things to "just work". :-)