Description
LocalBackend fails to initialize on HPC clusters where the home directory resides on a parallel filesystem (GPFS/Spectrum Scale or Lustre), because PRAGMA journal_mode = WAL requires POSIX shared-memory locking via mmap, which these filesystems do not support.
Steps to Reproduce
- Any script on an HPC compute node where the database path is on a parallel filesystem (GPFS, Lustre, or NFS) is blocked if, for instance, the local dashboard is open
Root Cause
SQLite WAL mode creates a shared-memory file (*.db-shm) that uses mmap with POSIX advisory locks. Parallel/distributed filesystems (GPFS, Lustre, NFS) do not fully support these semantics, causing the locking protocol error.
References
Environment
- OS: Linux (HPC cluster)
- Filesystem: GPFS / Lustre / NFS
- SQLite: bundled with Python
Description
LocalBackendfails to initialize on HPC clusters where the home directory resides on a parallel filesystem (GPFS/Spectrum Scale or Lustre), becausePRAGMA journal_mode = WALrequires POSIX shared-memory locking viammap, which these filesystems do not support.Steps to Reproduce
Root Cause
SQLite WAL mode creates a shared-memory file (
*.db-shm) that usesmmapwith POSIX advisory locks. Parallel/distributed filesystems (GPFS, Lustre, NFS) do not fully support these semantics, causing thelocking protocolerror.References
Environment