Skip to content

Add lockfile to prevent concurrent index runs#38

Merged
mehrdadpfg merged 3 commits intocogniplex:mainfrom
vfmatzkin:fix/index-lockfile
Mar 13, 2026
Merged

Add lockfile to prevent concurrent index runs#38
mehrdadpfg merged 3 commits intocogniplex:mainfrom
vfmatzkin:fix/index-lockfile

Conversation

@vfmatzkin
Copy link
Copy Markdown
Contributor

@vfmatzkin vfmatzkin commented Mar 13, 2026

Summary

  • Adds a PID-based lockfile (~/.codemem/locks/{namespace}.lock) to prevent multiple codemem index processes from running on the same namespace concurrently
  • Guards both CLI (cmd_index) and MCP (tool_index_codebase) entry points
  • Uses O_CREAT | O_EXCL (create_new) for atomic lock creation — no TOCTOU race
  • Stale locks (dead PID) are reclaimed with a remove-and-retry
  • Namespace is sanitized before use as a filename

Motivation

Without deduplication, concurrent index runs (from hooks, multiple sessions, or MCP calls) pile up and compete for CPU during the embedding step. Each process slows the others, creating a feedback loop where none can finish. I hit 15 simultaneous processes indexing the same repo at ~60% CPU each (load average 61).

Closes #37

Note

libc is listed as a new direct dependency, but it's already a transitive dependency of the workspace (pulled in by several existing crates). Adding it explicitly adds no new packages to the build and does not affect compile times. If preferred, the kill(pid, 0) call can be replaced with std::process::Command::new("kill").args(["-0", &pid.to_string()]) at the cost of spawning a subprocess on the stale-lock path.

Test plan

  • Live PID lock blocks concurrent run with clear error message
  • Dead PID lock is reclaimed and indexing proceeds
  • Lock file is cleaned up on successful completion
  • Lock file is cleaned up on error (RAII Drop guard)
  • Two concurrent processes: one wins, one gets a clean error
  • cargo check and cargo clippy pass

Copilot AI review requested due to automatic review settings March 13, 2026 00:48
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds a PID-based lockfile mechanism to prevent concurrent codemem index runs on the same namespace, addressing CPU exhaustion from parallel indexing processes (issue #37).

Changes:

  • New lockfile module with RAII-based lock acquisition and stale-PID detection
  • Lock guards added to both CLI (cmd_index) and MCP (tool_index_codebase) entry points
  • libc dependency added for Unix PID liveness checks

Reviewed changes

Copilot reviewed 5 out of 6 changed files in this pull request and generated 1 comment.

Show a summary per file
File Description
crates/codemem/src/lockfile.rs New module implementing PID-based lockfile with RAII cleanup
crates/codemem/src/cli/commands_export.rs Acquires lock before CLI index operation
crates/codemem/src/mcp/tools_graph.rs Acquires lock before MCP index operation
crates/codemem/src/lib.rs Exports new lockfile module
crates/codemem/Cargo.toml Adds libc dependency for Unix
Cargo.lock Lock file update

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

You can also share your feedback on Copilot code review. Take the survey.

Comment thread crates/codemem/src/lockfile.rs Outdated
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds a PID-based lockfile mechanism to prevent concurrent codemem index runs on the same namespace, addressing CPU exhaustion from duplicate indexing processes.

Changes:

  • New lockfile module with atomic lock creation (O_CREAT|O_EXCL), stale PID detection, and RAII cleanup
  • Lock acquisition integrated into both CLI (cmd_index) and MCP (tool_index_codebase) entry points
  • libc dependency added for Unix PID liveness checks

Reviewed changes

Copilot reviewed 5 out of 6 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
crates/codemem/src/lockfile.rs New module implementing PID-based lockfile with RAII guard
crates/codemem/src/lib.rs Exposes the new lockfile module
crates/codemem/src/cli/commands_export.rs Acquires lock before indexing in CLI path
crates/codemem/src/mcp/tools_graph.rs Acquires lock before indexing in MCP path
crates/codemem/Cargo.toml Adds libc dependency for Unix
Cargo.lock Lock file update

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

You can also share your feedback on Copilot code review. Take the survey.

Comment thread crates/codemem/src/lockfile.rs Outdated
Comment thread crates/codemem/src/lockfile.rs Outdated
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds a PID-based lockfile mechanism to prevent concurrent codemem index runs on the same namespace, addressing CPU exhaustion from duplicate indexing processes.

Changes:

  • New lockfile module with atomic lock creation (O_CREAT|O_EXCL), stale PID reclamation, and RAII cleanup
  • Lock acquisition integrated into both CLI (cmd_index) and MCP (tool_index_codebase) entry points
  • libc dependency added for Unix PID liveness checks

Reviewed changes

Copilot reviewed 5 out of 6 changed files in this pull request and generated no comments.

Show a summary per file
File Description
crates/codemem/src/lockfile.rs New lockfile module with IndexLock guard and try_acquire
crates/codemem/src/lib.rs Exports new lockfile module
crates/codemem/src/cli/commands_export.rs Acquires lock before indexing in CLI path
crates/codemem/src/mcp/tools_graph.rs Acquires lock before indexing in MCP path
crates/codemem/Cargo.toml Adds libc dependency for Unix
Cargo.lock Lock file update

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

You can also share your feedback on Copilot code review. Take the survey.

@codecov
Copy link
Copy Markdown

codecov Bot commented Mar 13, 2026

Codecov Report

❌ Patch coverage is 56.60377% with 23 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
crates/codemem/src/lockfile.rs 58.33% 20 Missing ⚠️
crates/codemem/src/cli/commands_export.rs 0.00% 2 Missing ⚠️
crates/codemem/src/mcp/tools_graph.rs 66.66% 1 Missing ⚠️

📢 Thoughts on this report? Let us know!

@mehrdadpfg mehrdadpfg merged commit 888d5a9 into cogniplex:main Mar 13, 2026
11 of 15 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Concurrent codemem index processes pile up and exhaust CPU

3 participants