track everything in jb ides

[makosblade]

Fix false-positive AI attribution in IntelliJ plugin sweep checkpoints

Problem

The sweep checkpoint mechanism incorrectly attributes human edits to AI agents. When Copilot or Junie edits a file, the plugin "tags" that file in an agentTouchedFiles map. The original implementation then watched for any subsequent documentChanged event on that file where the stack trace didn't contain the AI agent's packages -- and treated the content delta as an AI-attributable change.

The flaw: documentChanged fires for every content modification to the in-memory document, regardless of source. Human typing, IDE refactoring, VCS operations, and format-on-save all trigger it. None of these carry AI agent packages in their stack traces. So after an AI agent touches a file, every human keystroke in that file gets checkpointed as type: "ai_agent" until the tracking entry expires.

Root cause

A single BulkAwareDocumentListener was serving two fundamentally different detection roles:

1. Direct AI edit detection (stack trace analysis) -- works correctly
2. VFS refresh detection (catch AI disk writes that bypass the Document API) -- cannot distinguish human typing from disk refresh within the documentChanged callback

The documentChanged callback has no built-in mechanism to tell you why the content changed. It only tells you what changed.

Solution

Separate the two detection roles into distinct IntelliJ event listeners, each using the correct event source:

• DocumentChangeListener handles role 1: stack-trace-based AI detection for in-memory edits. When no AI agent is detected, it does nothing (guard clause returns early). It never triggers sweep checkpoints.

• VfsRefreshListener (new) handles role 2: implements BulkFileListener, which fires on VFS-level events. It filters for VFileContentChangeEvent where isFromRefresh == true -- meaning the change originated from a disk scan, not from an in-process write. It cross-references against agentTouchedFiles and triggers sweep checkpoints only for tracked files whose disk content differs from the last checkpoint.

The isFromRefresh property is the key discriminator. IntelliJ's VFS distinguishes between:

• In-process changes (isFromRefresh == false): document saves, VFS API writes, IDE refactoring
• External disk changes (isFromRefresh == true): another process wrote to the file and IntelliJ detected it during a VFS refresh

Human typing never produces a BulkFileListener event at all -- it only modifies the in-memory Document model. Auto-save produces isFromRefresh == false. Only actual external disk writes produce isFromRefresh == true.

Architecture

DocumentChangeTrackerService is the coordinator. It owns the shared state and injects it into both listeners via constructor:

DocumentChangeTrackerService (owns shared state)
├── agentTouchedFiles: ConcurrentHashMap<String, TrackedAgent>  (shared)
├── scheduler: ScheduledExecutorService                          (shared)
│
├── DocumentChangeListener (registered on EditorFactory.eventMulticaster)
│   ├── Writes to agentTouchedFiles in beforeDocumentChange (HIGH confidence AI)
│   ├── Updates lastCheckpointContent in documentChanged (HIGH confidence AI)
│   └── Triggers before_edit (Human) and after_edit (AiAgent) checkpoints
│
├── VfsRefreshListener (registered on VirtualFileManager.VFS_CHANGES message bus)
│   ├── Filters: VFileContentChangeEvent + isFromRefresh + file in agentTouchedFiles
│   ├── Debounces sweep per workspace root (5 seconds)
│   └── Groups checkpoints by agent, reads VFS content, compares against last checkpoint
│
└── Periodic eviction (every 5 minutes, removes stale tracking entries)

Neither listener knows about the other. Adding a third detection mechanism would be another constructor injection.

What this eliminates

Event source	Previously triggered sweep?	Now triggers sweep?
AI disk write (VFS refresh)	Yes	Yes
Human typing in editor	Yes (false positive)	No
IDE refactoring (rename, extract)	Yes (false positive)	No
VCS operation (checkout, stash pop)	Yes (false positive)	No
Format-on-save	Yes (false positive)	No
Auto-save to disk	Yes (false positive)	No
External non-AI disk write	Yes (false positive)	Yes (inherent*)

*External non-AI disk writes to a tracked file remain a false positive because the file system doesn't record which process performed a write. This is unsolvable at the application level without OS-level auditing. In practice it requires a human to externally edit the exact same file an AI recently touched, within the 5-minute staleness window.

Additional changes

• StackTraceAnalyzer: Fixed frame collection to only include frames for the first detected agent. Previously, if two agents' packages both appeared in a stack trace (theoretically possible with future agent additions), frames from both would be collected in relevantFrames even though only one agent is reported.

• Dead code removal: Removed a no-op contentBefore conditional block in executeAfterEditCheckpoint that checked for content changes but had an empty body.

• toRelativePath extracted: Moved to a package-level function in TrackedAgent.kt since both listeners need it (was previously a private method on DocumentChangeListener).

• Resource lifecycle: DocumentChangeTrackerService.dispose() now calls scheduler.shutdownNow() for clean shutdown. The message bus connection and editor factory listener both use the service's Disposable for automatic cleanup.

Files

File	Change
listener/TrackedAgent.kt	New -- shared TrackedAgent data class + toRelativePath()
listener/VfsRefreshListener.kt	New -- BulkFileListener with isFromRefresh filtering
listener/DocumentChangeListener.kt	Accepts injected shared state, sweep fallback removed
listener/StackTraceAnalyzer.kt	Frame collection fix
services/DocumentChangeTrackerService.kt	Coordinator: shared state, dual listener registration, periodic eviction

---

[devin-ai-integration]

✅ Devin Review: No Issues Found

Devin Review analyzed this PR and found no potential bugs to report.

View in Devin Review to see 4 additional findings.