Opening a large number of collections concurrently in one process can crash inside RocksDB's registry lookup rather than in zvec code.
Backtrace of the crashing thread:
EXC_BAD_ACCESS / SIGSEGV, address 0x20
std::__hash_table<std::string, ...>::find(this=..., __k="rocksdb.BuiltinBloomFilter")
This looks like concurrent access to a RocksDB internal registry (the filter-policy / plugin ObjectLibrary) that is populated during DB open. zvec opens a RocksDB instance per FTS and scalar indexer, so creating and opening many collections from many threads at once drives concurrent first-time registry population, and the lookup crashes.
Environment: v0.5.1 (submodule d25f3a4), built from source (RelWithDebInfo, AVX512 off). Reproduced on both x86_64 Linux and aarch64 macOS. A moderate ulimit -n amplifies it.
Reproduction: N threads, each creating, opening, upserting, querying, then dropping a distinct collection.
This seems to need zvec to either serialize RocksDB opens, or warm the registry once before spawning concurrent opens. Is concurrent multi-collection open intended to be supported, and if so, is any caller-side serialization expected?
Separate from the MemForwardStore writer-null crash fixed in #579.
Opening a large number of collections concurrently in one process can crash inside RocksDB's registry lookup rather than in zvec code.
Backtrace of the crashing thread:
This looks like concurrent access to a RocksDB internal registry (the filter-policy / plugin ObjectLibrary) that is populated during DB open. zvec opens a RocksDB instance per FTS and scalar indexer, so creating and opening many collections from many threads at once drives concurrent first-time registry population, and the lookup crashes.
Environment: v0.5.1 (submodule d25f3a4), built from source (RelWithDebInfo, AVX512 off). Reproduced on both x86_64 Linux and aarch64 macOS. A moderate
ulimit -namplifies it.Reproduction: N threads, each creating, opening, upserting, querying, then dropping a distinct collection.
This seems to need zvec to either serialize RocksDB opens, or warm the registry once before spawning concurrent opens. Is concurrent multi-collection open intended to be supported, and if so, is any caller-side serialization expected?
Separate from the MemForwardStore writer-null crash fixed in #579.