Skip to content

sqlite3.OperationalError: locking protocol when running on HPC parallel filesystems #407

@RPaolino

Description

Description

LocalBackend fails to initialize on HPC clusters where the home directory resides on a parallel filesystem (GPFS/Spectrum Scale or Lustre), because PRAGMA journal_mode = WAL requires POSIX shared-memory locking via mmap, which these filesystems do not support.

Steps to Reproduce

  1. Any script on an HPC compute node where the database path is on a parallel filesystem (GPFS, Lustre, or NFS) is blocked if, for instance, the local dashboard is open

Root Cause

SQLite WAL mode creates a shared-memory file (*.db-shm) that uses mmap with POSIX advisory locks. Parallel/distributed filesystems (GPFS, Lustre, NFS) do not fully support these semantics, causing the locking protocol error.

References

Environment

  • OS: Linux (HPC cluster)
  • Filesystem: GPFS / Lustre / NFS
  • SQLite: bundled with Python

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions