Add lockfile to prevent concurrent index runs#38
Conversation
There was a problem hiding this comment.
Pull request overview
Adds a PID-based lockfile mechanism to prevent concurrent codemem index runs on the same namespace, addressing CPU exhaustion from parallel indexing processes (issue #37).
Changes:
- New
lockfilemodule with RAII-based lock acquisition and stale-PID detection - Lock guards added to both CLI (
cmd_index) and MCP (tool_index_codebase) entry points libcdependency added for Unix PID liveness checks
Reviewed changes
Copilot reviewed 5 out of 6 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
| crates/codemem/src/lockfile.rs | New module implementing PID-based lockfile with RAII cleanup |
| crates/codemem/src/cli/commands_export.rs | Acquires lock before CLI index operation |
| crates/codemem/src/mcp/tools_graph.rs | Acquires lock before MCP index operation |
| crates/codemem/src/lib.rs | Exports new lockfile module |
| crates/codemem/Cargo.toml | Adds libc dependency for Unix |
| Cargo.lock | Lock file update |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
You can also share your feedback on Copilot code review. Take the survey.
There was a problem hiding this comment.
Pull request overview
Adds a PID-based lockfile mechanism to prevent concurrent codemem index runs on the same namespace, addressing CPU exhaustion from duplicate indexing processes.
Changes:
- New
lockfilemodule with atomic lock creation (O_CREAT|O_EXCL), stale PID detection, and RAII cleanup - Lock acquisition integrated into both CLI (
cmd_index) and MCP (tool_index_codebase) entry points libcdependency added for Unix PID liveness checks
Reviewed changes
Copilot reviewed 5 out of 6 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| crates/codemem/src/lockfile.rs | New module implementing PID-based lockfile with RAII guard |
| crates/codemem/src/lib.rs | Exposes the new lockfile module |
| crates/codemem/src/cli/commands_export.rs | Acquires lock before indexing in CLI path |
| crates/codemem/src/mcp/tools_graph.rs | Acquires lock before indexing in MCP path |
| crates/codemem/Cargo.toml | Adds libc dependency for Unix |
| Cargo.lock | Lock file update |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
You can also share your feedback on Copilot code review. Take the survey.
There was a problem hiding this comment.
Pull request overview
Adds a PID-based lockfile mechanism to prevent concurrent codemem index runs on the same namespace, addressing CPU exhaustion from duplicate indexing processes.
Changes:
- New
lockfilemodule with atomic lock creation (O_CREAT|O_EXCL), stale PID reclamation, and RAII cleanup - Lock acquisition integrated into both CLI (
cmd_index) and MCP (tool_index_codebase) entry points libcdependency added for Unix PID liveness checks
Reviewed changes
Copilot reviewed 5 out of 6 changed files in this pull request and generated no comments.
Show a summary per file
| File | Description |
|---|---|
| crates/codemem/src/lockfile.rs | New lockfile module with IndexLock guard and try_acquire |
| crates/codemem/src/lib.rs | Exports new lockfile module |
| crates/codemem/src/cli/commands_export.rs | Acquires lock before indexing in CLI path |
| crates/codemem/src/mcp/tools_graph.rs | Acquires lock before indexing in MCP path |
| crates/codemem/Cargo.toml | Adds libc dependency for Unix |
| Cargo.lock | Lock file update |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
You can also share your feedback on Copilot code review. Take the survey.
Codecov Report❌ Patch coverage is 📢 Thoughts on this report? Let us know! |
Summary
~/.codemem/locks/{namespace}.lock) to prevent multiplecodemem indexprocesses from running on the same namespace concurrentlycmd_index) and MCP (tool_index_codebase) entry pointsO_CREAT | O_EXCL(create_new) for atomic lock creation — no TOCTOU raceMotivation
Without deduplication, concurrent index runs (from hooks, multiple sessions, or MCP calls) pile up and compete for CPU during the embedding step. Each process slows the others, creating a feedback loop where none can finish. I hit 15 simultaneous processes indexing the same repo at ~60% CPU each (load average 61).
Closes #37
Note
libcis listed as a new direct dependency, but it's already a transitive dependency of the workspace (pulled in by several existing crates). Adding it explicitly adds no new packages to the build and does not affect compile times. If preferred, thekill(pid, 0)call can be replaced withstd::process::Command::new("kill").args(["-0", &pid.to_string()])at the cost of spawning a subprocess on the stale-lock path.Test plan
cargo checkandcargo clippypass