[BUG] apply_diff fails complaining about insufficient match when the match is 100% perfect

[rkfg]

Problem (one or two sentences)

Sometimes apply_diff starts refusing to work even though the content matches 100%

Context (who is affected and when)

Happens after a while when the model's context fills up. I'd suspect attention issues but the code fragments in the error message fully match.

Reproduction steps

Just running the usual editing loop. Here's the message:

<error_details>
No sufficiently similar match found (92% similar, needs 100%)

Debug Info:
- Similarity Score: 92%
- Required Threshold: 100%
- Search Range: start to end
- Tried both standard and aggressive line number stripping
- Tip: Use the read_file tool to get the latest content of the file before attempting to use the apply_diff tool again, as the file content may have changed

Search Content:
:start_line=18
-------
// MatrixConfig holds Matrix connection settings.
type MatrixConfig struct {
	HomeserverURL  string   `yaml:"homeserver_url"`
	AccessToken    string   `yaml:"access_token"`
	UserID         string   `yaml:"user_id"`
	BotDisplayName string   `yaml:"bot_display_name"`
	Rooms          []string `yaml:"rooms"`
}

Best Match Found:
16 | }
17 | 
18 | // MatrixConfig holds Matrix connection settings.
19 | type MatrixConfig struct {
20 | 	HomeserverURL  string   `yaml:"homeserver_url"`
21 | 	AccessToken    string   `yaml:"access_token"`
22 | 	UserID         string   `yaml:"user_id"`
23 | 	BotDisplayName string   `yaml:"bot_display_name"`
24 | 	Rooms          []string `yaml:"rooms"`
25 | }


Original Content:
 1 | package config
 2 | 
 3 | import (
 4 | 	"fmt"
 5 | 	"os"
 6 | 
 7 | 	"gopkg.in/yaml.v3"
 8 | )
 9 | 
10 | // Config holds all configuration for the bot.
11 | type Config struct {
12 | 	Matrix      MatrixConfig      `yaml:"matrix"`
13 | 	LLM         LLMConfig         `yaml:"llm"`
14 | 	Personality PersonalityConfig `yaml:"personality"`
15 | 	Context     ContextConfig     `yaml:"context"`
16 | }
17 | 
18 | // MatrixConfig holds Matrix connection settings.
19 | type MatrixConfig struct {
20 | 	HomeserverURL  string   `yaml:"homeserver_url"`
21 | 	AccessToken    string   `yaml:"access_token"`
22 | 	UserID         string   `yaml:"user_id"`
23 | 	BotDisplayName string   `yaml:"bot_display_name"`
24 | 	Rooms          []string `yaml:"rooms"`
25 | }
26 | 
27 | // LLMConfig holds LLM API settings.
28 | type LLMConfig struct {
29 | 	APIKey      string  `yaml:"api_key"`
30 | 	BaseURL     string  `yaml:"base_url"`      // Optional custom base URL
31 | 	Model       string  `yaml:"model"`
32 | 	MaxTokens   int     `yaml:"max_tokens"`
33 | 	Temperature float64 `yaml:"temperature"`
34 | }
35 | 
36 | // PersonalityConfig holds bot personality settings.
37 | type PersonalityConfig struct {
38 | 	Name           string   `yaml:"name"`
39 | 	Description    string   `yaml:"description"`
40 | 	Traits         []string `yaml:"traits"`
41 | 	Tone           string   `yaml:"tone"`
42 | 	LanguagePref   string   `yaml:"language_preference"`
43 | }
44 | 
45 | // ContextConfig holds chat context settings.
46 | type ContextConfig struct {
47 | 	MaxHistoryMessages int `yaml:"max_history_messages"`
48 | 	MaxContextTokens   int `yaml:"max_context_tokens"`
49 | }

</error_details>

And another one:

<error_details>
No sufficiently similar match found (83% similar, needs 100%)

Debug Info:
- Similarity Score: 83%
- Required Threshold: 100%
- Search Range: start to end
- Tried both standard and aggressive line number stripping
- Tip: Use the read_file tool to get the latest content of the file before attempting to use the apply_diff tool again, as the file content may have changed

Search Content:
:start_line=195
-------
	// Check for /reset command
	if strings.HasPrefix(body, "/reset") {
		if evt.Sender.String() == cfg.Matrix.UserID {

Best Match Found:
193 | 	}
194 | 
195 | 	// Check for /reset command
196 | 	if strings.HasPrefix(body, "/reset") {
197 | 		if evt.Sender.String() == cfg.Matrix.UserID {

Expected result

apply_diff works

Actual result

apply_diff complains about insufficient match

Variations tried (optional)

I would suspect tab/spaces mix up as in Go indentation is strictly tab-based. So I copied the code fragments from the error message to two text files, carefully removed the line numbers with an extra space after the pipe symbol, and then calculated sha256sum on both. It matched.

Another idea is there might be some kind of wrong escaping being applied internally. For example, another time the model couldn't produce an HTML escaping function with simple string replacement because it looked like strings.ReplaceAll("&", "&") instead of strings.ReplaceAll("&", "&"). It really tried hard and even sed command failed, though in finish_task report the replacement was reported correctly. Eventually it managed to implement it as "&"+"amp;". This issue is (probably) NOT directly related to the main apply_diff issue but it looked weird enough to also mention here because the model also struggled with editing where it shouldn't have.

App Version

Version: 3.53.0 (44f6bba)

API Provider (optional)

OpenAI Compatible

Model Used (optional)

Qwen3.6 35B A3B APEX

Roo Code Task Links (optional)

No response

Relevant logs or errors (optional)

[roomote-v0]

I've analyzed this issue and identified the root cause. Here's what's happening and my proposed fix:

Root Cause

The apply_diff regex in src/core/diff/strategies/multi-search-replace.ts expects the :start_line: marker with a colon delimiter (e.g., :start_line:18), but some models (especially smaller/quantized ones like the Qwen model mentioned) output :start_line=18 with an equals sign instead.

When the regex fails to match the :start_line=18 pattern as metadata, that line (plus the ------- separator) gets captured as part of the search content itself. So the tool ends up searching for:

:start_line=18
-------
// MatrixConfig holds Matrix connection settings.
type MatrixConfig struct {
...

...instead of just the actual code. The extra lines at the top reduce the Levenshtein similarity score (92% in the first example, 83% in the second), causing the "no sufficiently similar match" error even though the code portion matches perfectly.

Proposed Fix

1. Update the main diff-parsing regex in MultiSearchReplaceDiffStrategy.applyDiff() to accept both : and = as delimiters for :start_line: and :end_line: markers. Change \:start_line:\s*(\d+) to \:start_line[:=]\s*(\d+) (and same for :end_line:).

2. Update validateMarkerSequencing() to also recognize =-delimited markers when checking for invalid markers in REPLACE sections (lines 194-199).

3. Add test cases covering the = delimiter variant to confirm the fix works.

This is a small, targeted change (just the regex patterns) that makes the parser more tolerant of model output variations without changing any behavioral semantics.

Please mention @roomote in a comment saying that you approve this plan, and I'll proceed with the implementation.

[rkfg]

While this is true, the core issue wouldn't be solved. According to

Roo-Code/src/core/prompts/tools/native-tools/apply_diff.ts

Line 5 in ad25634

const DIFF_PARAMETER_DESCRIPTION = `A string containing one or more search/replace blocks defining the changes. The ':start_line:' is required and indicates the starting line number of the original content. You must not add a start line for the replacement content. Each block must follow this format:

:start_line: is a required parameter, if it's not found the tool should clearly report that as an error instead of ignoring the missing parameter and using the whole block as a code fragment to search. Such an error would let the model to quickly recover instead of getting lost. Even I didn't notice the argument format error!

[rkfg]

Here's what I propose instead, @roomote tell me what you think.

Plan: Fix :start_line: Validation in ApplyDiffTool

1. Problem Description

The apply_diff tool accepts a diff parameter that uses a multi-search-replace format with <<<<<<< SEARCH, =======, and >>>>>>> REPLACE markers. The :start_line: directive (specifying the exact line number in the original file where the search block starts) is documented as required in the tool prompt (src/core/prompts/tools/native-tools/apply_diff.ts:5: "The :start_line: is required and indicates the starting line number of the original content"), but the implementation does not enforce this requirement.

The regex in src/core/diff/strategies/multi-search-replace.ts:292 makes :start_line: optional via the group ((?:\:start_line:\s*(\d+)\s*\n))? (with trailing ?). When :start_line: is absent:

• The startLine defaults to 0 (src/core/diff/strategies/multi-search-replace.ts:310)
• The search content may be empty or match incorrect lines
• Fuzzy matching occurs across the entire file without a starting position
• No clear error message informs the user that :start_line: is missing

Note: All file references in this document use backtick format (path:line) instead of clickable links, for compatibility with GitHub issue rendering.

2. Problem Location in Code

File	Lines	Issue
src/core/prompts/tools/native-tools/apply_diff.ts	5	Prompt documents :start_line: as required, but enforcement is missing in implementation
src/core/diff/strategies/multi-search-replace.ts	292	Regex pattern has ? making :start_line: group optional
src/core/diff/strategies/multi-search-replace.ts	268-269	Comment says "Optionally matches" for :start_line:
src/core/diff/strategies/multi-search-replace.ts	310	startLine defaults to 0 when :start_line: is missing
src/core/tools/ApplyDiffTool.ts	77	parseInt(params.diff.match(/:start_line:(\d+)/)?.[1] ?? "") — silently falls back to empty string
src/core/diff/strategies/multi-search-replace.ts	101-243	validateMarkerSequencing does not track or enforce :start_line: presence

3. Fix Plan

3.1. Add Explicit :start_line: Check in src/core/tools/ApplyDiffTool.ts

In src/core/tools/ApplyDiffTool.ts:71-81, before calling task.diffStrategy?.applyDiff(), add validation that :start_line: is present in the diff content:

// Before line 74, add:
if (!/:start_line:\s*\d+/.test(diffContent)) {
    task.consecutiveMistakeCount++
    task.recordToolError("apply_diff")
    pushToolResult(await task.sayAndCreateMissingStartLineError(diffContent))
    return
}

This requires creating a new helper method sayAndCreateMissingStartLineError on the Task class (see 3.4).

Note on error format: The existing error messages in src/core/diff/strategies/multi-search-replace.ts:121,145,160 use the ERROR: prefix pattern. However, other tools in the codebase (e.g., src/core/tools/ApplyDiffTool.ts:64, src/core/tools/EditFileTool.ts:242) use <error_details> XML tags. For consistency with the apply_diff tool's existing error format, the new error should use <error_details> tags instead of ERROR: prefix.

3.2. Add :start_line: Presence Tracking in validateMarkerSequencing

In src/core/diff/strategies/multi-search-replace.ts:101-243, update the validateMarkerSequencing method:

1. Add a new state tracking variable:

   let foundStartLine = false

2. In the State.AFTER_SEARCH case (around line 214-220), detect :start_line::

   case State.AFTER_SEARCH:
       if (SEARCH_PATTERN.test(marker)) return reportInvalidDiffError(SEARCH_PATTERN.source, SEP)
       if (marker.startsWith(SEARCH_PREFIX)) return reportMergeConflictError(marker, SEARCH)
       if (marker === REPLACE) return reportInvalidDiffError(REPLACE, SEP)
       if (marker.startsWith(REPLACE_PREFIX)) return reportMergeConflictError(marker, SEARCH)
       if (/:start_line:\s*\d+/.test(marker.trim())) foundStartLine = true
       if (marker === SEP) {
           if (!foundStartLine) {
               return reportInvalidDiffError(SEP, ":start_line: <line_number>")
           }
           state.current = State.AFTER_SEPARATOR
       }
       break

Note: Instead of a custom error message with ERROR: prefix, use the existing reportInvalidDiffError helper which already includes the "CORRECT FORMAT" section with :start_line: documentation (src/core/diff/strategies/multi-search-replace.ts:148-154).

3.3. Update Regex in applyDiff To Make :start_line: Mandatory

In src/core/diff/strategies/multi-search-replace.ts:292, change the regex pattern:

Before:

/(?:^|\n)(?<!\\)<<<<<<< SEARCH>?\s*\n((?:\:start_line:\s*(\d+)\s*\n))?((?:\:end_line:\s*(\d+)\s*\n))?((?<!\\)-------\s*\n)?([\s\S]*?)(?:\n)?(?:(?<=\n)(?<!\\)=======\s*\n)([\s\S]*?)(?:\n)?(?:(?<=\n)(?<!\\)>>>>>>> REPLACE)(?=\n|$)/g

After:

/(?:^|\n)(?<!\\)<<<<<<< SEARCH>?\s*\n\:start_line:\s*(\d+)\s*\n((?:\:end_line:\s*(\d+)\s*\n))?((?<!\\)-------\s*\n)?([\s\S]*?)(?:\n)?(?:(?<=\n)(?<!\\)=======\s*\n)([\s\S]*?)(?:\n)?(?:(?<=\n)(?<!\\)>>>>>>> REPLACE)(?=\n|$)/g

Key changes:

• Remove ? from ((?:\:start_line:\s*(\d+)\s*\n))? → make it \:start_line:\s*(\d+)\s*\n (mandatory)
• Update capture group indices: match[1] is now the start line number (was match[2])
• Update all subsequent capture group references (match[6] → match[5], match[7] → match[6])

Also update the comment at lines 268-269:

// 3. \:start_line:\s*(\d+)\s*\n
//    Matches the required ":start_line:" line. Group 1 is the line number.

3.4. Add sayAndCreateMissingStartLineError Helper Method

Add a new method to the Task class (src/core/task/Task.ts) following the pattern of existing src/core/task/Task.ts:1869 method (sayAndCreateMissingParamError):

async sayAndCreateMissingStartLineError(diffContent: string) {
    await this.say(
        "error",
        "Roo tried to use apply_diff without value for required parameter ':start_line:' in diff. Retrying...",
    )
    
    const formattedError = `<error_details>\n:start_line: is required in your diff content.\n\n` +
        "The ':start_line:' directive specifies the exact line number in the original file where the search block starts.\n\n" +
        "CORRECT FORMAT:\n\n" +
        "<<<<<<< SEARCH\n" +
        ":start_line:42\n" +
        "-------\n" +
        "[exact content to find including whitespace]\n" +
        "=======\n" +
        "[new content to replace with]\n" +
        ">>>>>>> REPLACE\n" +
        "\n" +
        "Your diff content:\n" +
        "```\n" +
        diffContent.slice(0, 500) + (diffContent.length > 500 ? "..." : "") + "\n" +
        "```\n" +
        "\n</error_details>"
    
    return formattedError
}

3.5. Update Error Message at Line 299

In src/core/diff/strategies/multi-search-replace.ts:299, update the error message to reflect that :start_line: is now required:

Before:

error: `Invalid diff format - missing required sections\n\nDebug Info:\n- Expected Format: <<<<<<< SEARCH\\n:start_line: start line\\n-------\\n[search content]\\n=======\\n[replace content]\\n>>>>>>> REPLACE\n- Tip: Make sure to include start_line/SEARCH/=======/REPLACE sections with correct markers on new lines`,

After:

error: `Invalid diff format - missing required sections\n\nDebug Info:\n- Expected Format: <<<<<<< SEARCH\\n:start_line: <line_number>\\n-------\\n[search content]\\n=======\\n[replace content]\\n>>>>>>> REPLACE\n- Tip: Make sure to include :start_line:, SEARCH, -------, SEARCH content, =======, REPLACE content, and REPLACE sections with correct markers on new lines`,

4. Expected Error Message

When :start_line: is missing, the user should see (using <error_details> format consistent with src/core/tools/ApplyDiffTool.ts:64 and src/core/tools/EditFileTool.ts:242):

<error_details>
:start_line: is required in your diff content.

The ':start_line:' directive specifies the exact line number in the original file where the search block starts.

CORRECT FORMAT:

<<<<<<< SEARCH
:start_line:42
-------
[exact content to find including whitespace]
=======
[new content to replace with]
>>>>>>> REPLACE

Your diff content:

[actual diff content snippet]

</error_details>

5. Test Coverage

Add tests in src/core/diff/strategies/__tests__/ for:

1. validateMarkerSequencing rejects diff without :start_line:
2. applyDiff rejects diff without :start_line: (regex returns no matches)
3. ApplyDiffTool emits proper error message when :start_line: is missing
4. Valid diff with :start_line: continues to work correctly

6. Files to Modify

File	Changes
src/core/tools/ApplyDiffTool.ts	Add :start_line: validation before applyDiff call
src/core/diff/strategies/multi-search-replace.ts	Update regex, add tracking in validateMarkerSequencing, update error messages
src/core/task/Task.ts	Add sayAndCreateMissingStartLineError helper (following src/core/task/Task.ts:1869 pattern)
src/core/diff/strategies/__tests__/multi-search-replace.spec.ts	Add tests for mandatory :start_line: validation

7. Reference: Tool Prompt

The tool prompt at src/core/prompts/tools/native-tools/apply_diff.ts already documents :start_line: as required:

const DIFF_PARAMETER_DESCRIPTION = `A string containing one or more search/replace blocks defining the changes. The ':start_line:' is required and indicates the starting line number of the original content. You must not add a start line for the replacement content. Each block must follow this format:
<<<<<<< SEARCH
:start_line:[line_number]
-------
[exact content to find]
=======
[new content to replace with]
>>>>>>> REPLACE`

[roomote-v0]

I've seen the plan and will implement it. I'll open a PR to share progress shortly.

[roomote-v0]

Opened a draft PR with my attempt at implementing the plan: #12200. Feedback welcome!

[rkfg]

@roomote built the extension with this patch and tried it in the session where I had this problem. After a couple of retries Roo was able to recover and use the correct format! However, this patch changes many tests where :start_line: was missing so a human review is needed to decide whether this parameter should be mandatory (as the prompt suggests) or not (as the tests suggest). In any case a botched parameter use should be handled and reported properly because it confuses even humans.

[roomote-v0]

I've seen the feedback from @rkfg and will look into improving the handling of :start_line: - making sure a missing or botched :start_line: is handled gracefully with a clear error message rather than a confusing similarity mismatch. I'll open a PR to share progress.

[roomote-v0]

Opened a draft PR with an alternative approach at #12201. Instead of making :start_line: mandatory (as in PR #12200), this takes a tolerant parsing approach -- the regex now accepts both :start_line: and :start_line= formats, and a fallback strips leaked directives from search content. This keeps backward compatibility and handles botched directives gracefully. Feedback welcome!