Skip to content

[business-logic] bare_edit_streak classifies run_command and git_* as verification — detector bypassed by common agent patterns #141

Description

@topcheer

Problem

bare_edit_streak.go detects "consecutive file mutations without any verification step" and warns the agent to build/test before continuing. But bareStreakIsVerification() (lines 92-106) classifies tools by name alone without inspecting their arguments, causing two false-negative bypasses.

Root Cause

// bare_edit_streak.go lines 92-106
func bareStreakIsVerification(toolName string) bool {
    switch {
    case toolName == "run_command",           // BUG 1: no command content check
        toolName == "start_command",
        toolName == "lsp_diagnostics",
        toolName == "lsp_references",
        toolName == "lsp_definition",
        toolName == "code_health",
        toolName == "review_changes",
        toolName == "verify",
        strings.HasPrefix(toolName, "git_"):   // BUG 2: git_add/git_commit are mutations, not verification
        return true
    default:
        return false
    }
}

The call site (agent.go) passes only the tool name:

a.bareEditStreak.recordToolCall(tc.Name)  // no args available

Bug 1: run_command unconditionally resets streak

Any run_command resets the streak, regardless of what command was run. run_command("echo done"), run_command("ls -la"), run_command("pwd") — none of these verify code correctness, yet they all reset the counter to 0.

Bypass sequence (threshold = 5, never reached):

Step Tool call Streak
1 edit_file("a.go") 1
2 run_command("echo done") 0 (false reset)
3 edit_file("b.go") 1
4 run_command("ls") 0 (false reset)
... repeats forever never reaches 5

Bug 2: git_ prefix misclassifies mutations as verification

strings.HasPrefix(toolName, "git_") matches ALL git tools, including non-verification mutations:

Tool Actually verification?
git_add No — staging files
git_commit No — writing history
git_checkout No — switching branches
git_reset No — destructive workspace change
git_revert No — history mutation
git_stash No — workspace mutation

Bypass sequence (agent commits incrementally — a very common pattern):

Step Tool call Streak
1 edit_file("a.go") 1
2 git_add(["a.go"]) 0 (classified as verification!)
3 edit_file("b.go") 1
4 git_add(["b.go"]) 0
... never reaches 5

This is especially problematic because interleaving edit → git_add is a natural, recommended agent workflow — not an adversarial bypass. The detector is defeated by following best practices.

Contrast with correct implementation

The sibling detector edit_abandon.go in the same package has the correct pattern:

func eaIsVerifyTool(tool, args string) bool {  // takes BOTH tool + args
    case "run_command":
        return isVerificationCommand(eaExtractCommand(args))  // checks content!

edit_abandon.go already provides eaExtractCommand() and isVerificationCommand() — the infrastructure to fix this exists but is not reused.

Impact

  • Severity: Medium — advisory detector silently fails to fire when it should warn
  • Contradicts own design: The file header (lines 3-12) says the goal is detecting mutations "without any verification step (build/test/run/diagnostic)". The warning message (line 73-76) says "Run go build... or go test... NOW" — implying only actual build/test counts. The code includes non-verification tools.
  • Bypass triviality: Extremely easy — common agent patterns (git_add between edits, run_command("ls")) defeat it without intention

Suggested Fix

  1. Change recordToolCall signature to accept args: recordToolCall(toolName, args string)
  2. For run_command/start_command: delegate to isVerificationCommand(extractCommand(args)) (reuse existing helpers from edit_abandon.go)
  3. For git_: remove the blanket prefix; no git tool verifies code correctness. If desired, allowlist only git_diff (reviewing changes).
  4. Update call site in agent.go to pass tc.Args

Verification

Independently verified by subagent (sa-64):

  • Both bugs confirmed with concrete tool-call sequence analysis
  • Call site confirmed: recordToolCall(tc.Name) passes only name, no args
  • Contrast with edit_abandon.go confirmed: correct pattern exists in same package

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions