fix error for initializing Geos#51
Conversation
|
@estebanzimanyi can you help me review and fix the error in this PR. |
|
Thanks Ngoc for the detailed report. The mutex approach in this PR should in principle serialize all MobilityDuck–MEOS calls against each other, which prevents concurrent GEOS legacy-API ( However, your specific query mixes two extensions in the same pipeline: tgeompointSeq(list(tgeompoint(ST_Transform(geom_4326, 'EPSG:4326', 'EPSG:25832', always_xy := true), t) ORDER BY t))
To narrow down the root cause, could you try:
If the error vanishes when |
|
|
Thank you for investigating this! The GEOS initialization segfault turned out to have a deeper root cause that the mutex approach doesn't fully address. Root causeMEOS PR #815 made all mutable MEOS state thread-local ( The initialization in Why the mutex doesn't fix itThe global
The correct fixEach DuckDB worker thread must call inline void EnsureMeosInitializedOnThread() {
thread_local bool initialized = false;
if (!initialized) {
meos_initialize();
meos_initialize_error_handler(&MobilityduckMeosErrorHandler);
initialized = true;
}
}
Verified: Superseding this PR with |
|
Closing in favour of #fix/per-thread-meos-init — see comment above for full analysis. |
Replaces the process-global `std::mutex` (from the now-closed PR #51) that wrapped every MEOS scalar-function call with a per-thread `thread_local bool` guard in `EnsureMeosInitializedOnThread()`. Each DuckDB worker thread calls `meos_initialize()` and `meos_initialize_error_handler()` exactly once on first use; cost is one `bool` check per scalar-function call thereafter — negligible. Full DuckDB parallelism is preserved. Bundles the matching co-commit refinements (3 commits squashed to 1): 1. **Per-thread MEOS init** (was c237f6c) The mutex → thread_local swap. 2. **Cover cast functions** (was 55d4c5a) Cast functions bypass `RegisterSerializedScalarFunction` and would otherwise leave MEOS uninitialised when a cast is the first MEOS call (e.g. `SELECT stbox 'STBOX XT(...)'` before any scalar function). Initialise on the loading thread to cover them. 3. **TZ-neutral test assertions** (was 9dd765a) Make `stbox.test` timestamp assertions timezone-neutral so the test passes under both local and CI session timezones. ## Root cause MEOS [PR #815](MobilityDB/MobilityDB#815) made all mutable MEOS state thread-local (`MEOS_TLS`). Threads that hadn't called `meos_initialize()` had NULL TLS pointers and segfaulted. The mutex was the conservative workaround; per-thread init is the correct fix. Closes #404.
Mutex serialization for MobilityDuck scalar function bodies that ultimately call MEOS / liblwgeom. MEOS uses GEOS's legacy global API (
initGEOS/finishGEOS) internally from worker threads; concurrent legacy GEOS init/teardown from DuckDB's parallelism triggers intermittent GEOS errors and allocator corruption when spatial and MobilityDuck coexist.Wrapping each MEOS-backed ScalarFunction execution prevents overlapping MEOS/GEOS legacy pairs across concurrent pipelines while preserving parallelism elsewhere.