fix: prevent SIGSEGV on second ERPL load in same process (issue #52) - #56
Merged
Conversation
Three interacting problems caused a crash when creating a second DuckDB
connection and loading ERPL within the same Python process:
1. Trampoline (erpl): after connection 1 closed, DuckDB's dlclose reduced
the trampoline's refcount to 0, allowing glibc to release the SAP SDK's
RTLD_GLOBAL entries from RTLD_DEFAULT. Fix: PinSelf() acquires one extra
dlopen(RTLD_NODELETE) reference on first load — subsequent dlclose calls
cannot drop the count to 0 and the SAP SDK stays mapped.
2. erpl_rfc: registers extension options with Value("INFO") etc., whose
StringValueInfo vtable lives inside erpl_rfc's code pages. DuckDB caches
these Values in a thread-local CachedGlobalSettings; if erpl_rfc were
unloaded between connections, destroying the stale cache on the next
connection would dereference dangling vtable pointers → SIGSEGV. Fix:
same PinSelf() pattern applied to erpl_rfc's LoadInternal.
3. Re-saving sub-extension blobs on connection 2 would truncate the
currently-mmap'd extension files (erpl_rfc.duckdb_extension, etc.) before
rewriting them. Accessing unmapped pages beyond the truncated file size
causes SIGBUS/SIGSEGV. Fix: when SapLibsAlreadyLoaded() is true, skip the
SaveToFile calls entirely — the files are already on disk from connection 1,
INSTALL is a NOP, and LOAD reuses the pinned dlopen handles.
Also adds a regression test (trampoline/test/python/test_double_load.py)
and integrates it as Step 5 in all three smoke-test scripts (Linux glibc,
Linux musl, Windows PowerShell) to prevent future regressions.
…uild, quote --only-binary in PS1 - smoke-test.sh: replace `pip3 install` with `python3 -m pip install` so the duckdb package lands in the same interpreter that runs the test (manylinux CI has pip3 and python3 pointing at different installations) - smoke-test.sh: skip Step 5 when OSX_BUILD_ARCH=x86_64 on a Darwin runner — the native Python/duckdb wheel is arm64 and refuses to install an osx_amd64 extension, causing a platform-mismatch IO error - smoke-test.ps1: single-quote '--only-binary=:all:' to prevent PowerShell treating the leading colon in ':all:' as a scope operator (ParserError)
… HOME conflict python3 -m pip install without --target installs to $HOME/.local, but the test runs with HOME="$SMOKE_HOME" so Python cannot find the package. Using --target $PY_PKG_DIR and PYTHONPATH="$PY_PKG_DIR" keeps install and runtime on the same search path regardless of HOME.
…event exit SIGSEGV After the SAP SDK is loaded with RTLD_GLOBAL, its OpenSSL symbols shadow system libssl in RTLD_DEFAULT. At Python interpreter shutdown, libssl.so's __attribute__((destructor)) calls OPENSSL_cleanup() which resolves to the SAP SDK's implementation, corrupting the SDK's global SSL state. Subsequent teardown code that touches SSL (PostHog telemetry thread, DuckDB cleanup) then crashes. glibc: calling dlopen with RTLD_NOLOAD|RTLD_NODELETE on an already-loaded library retroactively sets DF_1_NODELETE in its link-map entry, permanently preventing unload and suppressing the destructor. This fixes the exit-time SIGSEGV seen with duckdb Python 1.5.1/1.5.2 wheels.
…ad test The PostHog background thread makes a live HTTPS call to eu.posthog.com. Without this flag, Python's interpreter teardown races with the SSL thread and the process crashes with SIGSEGV (exit 139) after the test prints PASSED. DATAZOO_DISABLE_TELEMETRY=1 causes PostHogProcess() to return immediately, so no SSL activity occurs and Python exits cleanly.
…t-cause fix posthog-telemetry now sets 2s/3s HTTP timeouts and uses a timed Stop() with detach fallback so the background worker thread always completes well before process teardown begins. The DATAZOO_DISABLE_TELEMETRY=1 env override that was papering over the SIGSEGV in the smoke test is removed.
…fix PS1 em-dash parse error - posthog-telemetry: ProcessQueue exits after the current in-flight HTTP task when stop_processing is set, dropping any remaining queued tasks. This caps the Stop() wait to one HTTP call (8 s max) instead of all pending calls, ensuring the timed join always succeeds. - posthog-telemetry: increase Stop() timed-join deadline from 5 s to 10 s to cover the full 8 s worst-case HTTP duration (2 s connect + 3 s read + 3 s write) with a 2 s safety margin. The thread is now always joined, never detached, eliminating the race between a detached thread and OpenSSL cleanup at process exit. - smoke-test.ps1: replace em-dash character with ASCII hyphen in the Step 5 skip warning. The em-dash is UTF-8 encoded (0xE2 0x80 0x94); PowerShell 5.1 reads the file as Windows-1252 and decodes byte 0x94 as RIGHT DOUBLE QUOTATION MARK (U+201D), which it treats as a string terminator, causing a TerminatorExpectedAtEndOfString parse error.
The std::atexit(Shutdown) registered in each sub-extension's LoadInternal raced with ~PostHogTelemetry(): since Instance() is called AFTER the atexit registration, the singleton's __cxa_atexit destructor runs first (LIFO), freeing the TelemetryTaskQueue. The atexit handler then calls Shutdown() on the already-destroyed object, reading a dangling unique_ptr and triggering a double-free → SIGABRT. Fix: remove the atexit registrations. ~PostHogTelemetry() already joins the worker thread via _queue->Stop(), so no additional cleanup is needed. Also switch PostHogProcess to SSLClient(host, port) to avoid the function- local static Regex in Client(url) that caused the original SIGSEGV.
With $ErrorActionPreference='Stop', using '2>&1 | Out-Null' on pip wraps pip's stderr lines as PowerShell ErrorRecord objects that trigger a terminating error before $LASTEXITCODE can be checked to skip the test gracefully. Fix: use '2>$null | Out-Null' to discard stderr directly (no ErrorRecord objects in the pipeline) and add a try/catch so any unexpected exception still degrades to a skip rather than a hard failure.
PowerShell uses $LASTEXITCODE from the last native command when no explicit exit is given. When pip install fails (no wheel for the DuckDB version on Windows), $LASTEXITCODE stays 1 for the rest of the script — so the runner sees exit code 1 even though '=== Smoke test PASSED ===' was printed.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fixes #52 — SIGSEGV when creating a second DuckDB connection and loading
LOAD erplwithin the same Python process.Three root causes were identified and fixed:
Trampoline self-pinning: After connection 1 closes, DuckDB's
dlclosereduced the trampoline's refcount, letting glibc release the SAP SDK'sRTLD_GLOBALentries fromRTLD_DEFAULT. AddedPinSelf()usingdlopen(RTLD_NODELETE)on first load so subsequentdlclosecalls cannot drop the refcount to 0.erpl_rfc self-pinning:
erpl_rfcregisters extension options withValue("INFO")etc., whoseStringValueInfovtable lives in erpl_rfc's code pages. DuckDB caches these in a thread-localCachedGlobalSettings; unloading erpl_rfc between connections caused dangling vtable dereferences on the next connection. SamePinSelf()fix applied toerpl_rfc.No re-saving mapped files: On connection 2, re-writing sub-extension blobs (
erpl_rfc.duckdb_extensionetc.) would truncate the currently-mmap'd files before rewriting them, causing SIGBUS/SIGSEGV when accessing pages beyond the truncated file size. WhenSapLibsAlreadyLoaded()is true, theSaveToFilecalls are now skipped — the files exist from connection 1,INSTALLis a NOP, andLOADreuses the pinned dlopen handles.Test plan
python3 trampoline/test/python/test_double_load.py build/release/extension/erpl/erpl.duckdb_extensionpasses locally (54 sap_* functions on connection 2)trampoline/test/python/test_double_load.pyadded — tests two consecutive connections in the same processscripts/smoke-test.sh(Linux glibc),scripts/smoke-test-musl.sh(Linux musl/Alpine), andscripts/smoke-test.ps1(Windows) — usespip install duckdb==<version>to run the Python test; skips gracefully if no wheel is available