Skip to content

fix: classify str_replace and insert tools as mutating edits#1

Merged
scoootscooob merged 1 commit into
openclaw:mainfrom
MiltonHeYan:fix/str-replace-tool-classification
Apr 17, 2026
Merged

fix: classify str_replace and insert tools as mutating edits#1
scoootscooob merged 1 commit into
openclaw:mainfrom
MiltonHeYan:fix/str-replace-tool-classification

Conversation

@MiltonHeYan
Copy link
Copy Markdown
Contributor

What's wrong

classify_tool_call identifies the tool family from the tool name using a set of regex patterns. The pattern for the "edit" family is:

if re.search(r"write|edit|patch|apply|create|delete|rename", name):
    return "edit", True

"replace" and "insert" are missing. As a result, tools like str_replace, replace_in_file, insert_text, and insert_at_line fall through every check and return ("unknown", False) — classified as non-mutating, unknown family.

Impact

For any agent that edits files via str_replace:

  • distinct_mutation_targets is always empty → min_distinct_mutation_targets requirement always fails
  • read_before_write_ratio is 1.0 for the wrong reason — no mutations are detected, so the denominator collapses to 1
  • "edit" never appears in distinct_familiesrequired_families always reports it as missing
  • Trajectory score is silently wrong without any error

Fix

Extend the edit pattern:

# before
if re.search(r"write|edit|patch|apply|create|delete|rename", name):
# after
if re.search(r"write|edit|patch|apply|create|delete|rename|replace|insert", name):

Tests

Added two new tests in test_trajectory.py:

  • test_replace_and_insert_tools_are_classified_as_edit — unit test for classify_tool_call directly
  • test_str_replace_mutation_is_detected_in_trajectory — end-to-end trajectory test verifying that mutations via str_replace are properly detected

All 7 tests pass.

classify_tool_call matched tool names against a fixed set of verb
patterns. The pattern for the "edit" family was:

    r"write|edit|patch|apply|create|delete|rename"

This omitted "replace" and "insert", so tools like str_replace,
replace_in_file, insert_text, and insert_at_line all fell through
every check and were returned as ("unknown", False) – classified as
non-mutating with unknown family.

Consequences for any agent that edits via str_replace:
- distinct_mutation_targets stayed empty → min_distinct_mutation_targets
  requirement always failed
- read_before_write_ratio was 1.0 for the wrong reason (no mutations
  detected, so denominator collapsed to 1)
- "edit" never appeared in distinct_families → required_families check
  always reported it as missing

Fix: extend the edit pattern with "replace" and "insert".

Tests added: unit test for classify_tool_call directly and an end-to-end
trajectory test using a str_replace-based edit transcript.
@scoootscooob scoootscooob merged commit a2757e6 into openclaw:main Apr 17, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants