feat: Implement Git worktree-like functionality for multi-agent systems

[zhangfengcdt]

Summary

This PR implements Git worktree-like functionality for ProllyTree's VersionedKvStore, solving the critical race condition problem in multi-agent systems where multiple instances compete for the same Git repository resources.

Problem Solved

Multiple VersionedKvStore instances pointing to the same Git repository would compete for:

• The same HEAD file
• The same refs/heads/ branch references
• The same working directory files
• The same index/staging area

This created race conditions and data corruption in concurrent multi-agent scenarios.

Solution

Implemented a worktree system similar to git worktree add that provides:

• Separate HEAD for each worktree
• Separate working directories for each agent
• Separate index/staging areas per worktree
• Shared Git object database for collaboration
• Locking mechanism to prevent conflicts

Key Features

🏗️ Core Architecture

• WorktreeManager: Manages multiple worktrees for a single Git repository
• WorktreeVersionedKvStore: VersionedKvStore that operates within a specific worktree
• Branch Isolation: Each agent gets separate HEAD, working directory, and branch
• Thread-Safe Operations: Concurrent worktree access with proper locking

🐍 Python Integration

• Full Python API via PyO3 bindings
• PyWorktreeManager and PyWorktreeVersionedKvStore classes
• Seamless integration with existing VersionedKvStore Python API

🔒 Safety & Isolation

• Race Condition Prevention: No more HEAD file competition
• Agent Isolation: Complete context separation between agents
• Collaborative Foundation: Shared object database enables data sharing
• Locking Mechanism: File-based locks prevent concurrent modifications

Files Changed

Implementation (1,359+ lines added)

• ✨ src/git/worktree.rs: Core worktree implementation (622 lines)
• 🔧 src/python.rs: Python bindings (+202 lines)
• 📦 Cargo.toml: Added uuid dependency
• 🔗 src/git/mod.rs: Export worktree types

Testing & Documentation

• 🧪 python/tests/test_worktree_integration.py: Comprehensive integration tests
• 📚 docs/WORKTREE_IMPLEMENTATION.md: Complete architecture documentation
• 📝 tests/README.md: Test organization documentation
• 📖 CLAUDE.md: Usage commands and examples

Test Results

✅ Rust Tests (84/84 passing)

• Basic Operations: Worktree creation, listing, locking
• Manager Functionality: All core WorktreeManager operations
• Thread Safety: Concurrent worktree access validation
• Concept Validation: Multi-agent race condition prevention

✅ Python Tests (100% passing)

• WorktreeManager API: Full Python bindings validation
• Multi-Agent Simulation: 3 concurrent agents with isolation
• Locking Mechanism: Conflict prevention validation
• Architecture Concepts: Race condition solution verification

Usage Examples

Multi-Agent Python Pattern

from prollytree.prollytree import WorktreeManager

# Create manager for shared repository
manager = WorktreeManager("/path/to/shared/repo")

# Each agent gets isolated workspace
agents = ["billing", "support", "analysis"]
for agent in agents:
    info = manager.add_worktree(
        f"/tmp/{agent}_workspace",
        f"session-001-{agent}",
        True  # Create new branch
    )
    print(f"Agent {agent} workspace: {info['path']}")
    print(f"Agent {agent} branch: {info['branch']}")

Rust API

use prollytree::git::{WorktreeManager, WorktreeVersionedKvStore};

// Create manager for existing repository
let mut manager = WorktreeManager::new("/path/to/repo")?;

// Add worktree for agent
let info = manager.add_worktree(
    "/path/to/agent_workspace", 
    "agent-feature-branch", 
    true
)?;

// Create isolated store for agent
let agent_store = WorktreeVersionedKvStore::<32>::from_worktree(
    info, 
    Arc::new(Mutex::new(manager))
)?;

// Agent can now work safely on their branch
agent_store.store_mut().insert(b"key".to_vec(), b"value".to_vec())?;

Architecture Impact

Before (Race Conditions) ❌

Multiple VersionedKvStore → Same .git/HEAD → Race Conditions
Multiple VersionedKvStore → Same refs/heads/ → Conflicts  
Multiple VersionedKvStore → Same working dir → File conflicts

After (Isolated Worktrees) ✅

Agent1 → WorktreeManager → Worktree1 → Separate HEAD + Branch
Agent2 → WorktreeManager → Worktree2 → Separate HEAD + Branch  
Agent3 → WorktreeManager → Worktree3 → Separate HEAD + Branch
              ↓
        Shared Git Objects (collaboration)

Performance Characteristics

• Memory: Minimal overhead per worktree (separate HEAD + metadata only)
• Storage: Shared object database minimizes disk usage
• Concurrency: No locking contention between different worktrees
• Scalability: Linear scaling with number of agents

Breaking Changes

None. This is purely additive functionality that doesn't affect existing APIs.

Migration Path

Existing code continues to work unchanged. Teams can opt-in to worktree functionality for multi-agent scenarios where race condition prevention is needed.

Integration with Multi-Agent Systems

This implementation provides the foundation for safe multi-agent operations:

1. Agent Initialization: Each agent gets their own worktree
2. Isolated Work: Agents work on separate branches without conflicts
3. Validation: Agent work can be validated before merging
4. Collaboration: Agents share data through common object database
5. Audit Trail: All operations tracked with Git commits

Future Enhancements

Potential production improvements:

• Automated merging of agent branches
• Conflict resolution for merge operations
• Garbage collection of abandoned worktrees
• Health monitoring and metrics
• Remote worktree operations

🤖 Generated with Claude Code