Inconsistent Generation of Tool Parameters (XML Special Characters, Diff Markers) Causing Confusion and Tool Failures

[ElemTran]

App Version

3.14.3

API Provider

OpenRouter

Model Used

Gemini 2.5 Pro

Actual vs. Expected Behavior

Actual Behavior:

1. XML Special Character Handling: When the AI assistant (Roo) presents code or text containing XML special characters (specifically <, >, &, ", ') within the XML structure of tool calls (e.g., as parameters for write_to_file, apply_diff, search_files, or even within ask_followup_question), these characters are sometimes displayed in their escaped forms (e.g., <, >, &, ", ').
2. This escaping causes confusion for the user, making it unclear whether this is expected behavior, a parsing requirement, or an error.
3. Diff Marker Inclusion: The AI sometimes incorrectly includes diff markers (like a + prefix) in the REPLACE content generated for the apply_diff tool, indicating a potential misunderstanding of the required parameter format.
4. Tool Failures: Tools (especially apply_diff, but potentially others like search_files) seem more prone to failure when dealing with these special characters or incorrect markers (e.g., apply_diff failing to find a match), even when the target file content appears largely correct. It is suspected that inconsistencies in character handling or marker inclusion lead to matching or parsing failures.

Expected Behavior:

1. There should be clear documentation or AI operational rules/tool usage instructions specifying whether XML special characters (<, >, &, ", ') must be escaped when included in tool parameters (especially for code blocks).
2. The system should handle these characters consistently. Either:
   • Escaping is strictly required, and all tools correctly interpret the escaped forms.
   • Escaping is not required (or handled automatically by the system before parsing), and tools expect literal characters.
   • For code clarity, the presentation layer should ideally display literal < and > characters, unless escaping is absolutely necessary.
3. Tools (like apply_diff, search_files) should reliably handle content containing these special characters. For example, the diffing mechanism for apply_diff needs to correctly account for potential escaping inconsistencies between the provided SEARCH block and the actual file content.
4. The AI should never include any diff markers (like + or -) in the REPLACE content generated for apply_diff; it should only contain the plain text lines intended for the final file.

Detailed Steps to Reproduce

• (For Character Handling):
  1. Instruct the AI assistant (Roo) to perform a task requiring it to include code or text with XML special characters (<, >, &, ", ') within a tool call parameter (e.g., in the content or diff parameter of ask_followup_question, write_to_file, apply_diff).
  2. Observe the AI's response and the tool call format. Note whether the special characters are escaped.
  3. If tools like apply_diff or search_files are used, observe if they fail or produce incorrect results due to inconsistent character handling.
• (For apply_diff '+' Marker):
  1. Instruct the AI assistant (Roo) to use the apply_diff tool to modify a file, specifically adding new lines.
  2. Inspect the >>>>>>> REPLACE section of the apply_diff call generated by the AI.
  3. Observe if the REPLACE section incorrectly contains lines prefixed with +.

Relevant API Request Output

Gemini 2.5 Pro Explanation:
Hello, regarding your question:

What are < and >?

Sorry for the confusion. When I display code, since the code is contained within an XML structure (required by the format of the tool), in order to prevent the < and > symbols in the code from conflicting with the XML tag symbols, I used their HTML entity representations:
&lt; represents the less-than sign <
&gt; represents the greater-than sign >
In an actual C++ code file, you should use the standard < and > symbols, such as #include <QObject> and std::function<void()>.

Additional Context

Initially, I regarded the escaping of < / > and other characters as a potential bug in the AI's output generation.

The frequent failures of the apply_diff tool (and potential issues with other tools) severely hinder the development process when dealing with code or text that contains special characters.

The AI mistakenly includes the + marker when generating the REPLACE content for apply_diff, indicating that there may be a deviation in its understanding of the format of tool parameters.

Currently, it is unclear whether the XML parser at the underlying layer of the Roo Code system requires escaping, or whether the escaping occurs at other stages and interferes with the functionality of the tools.

Improving the tool usage guidelines or prompts regarding character handling and the format of tool parameters will be beneficial both for users and for the AI model itself.

[dosubot]

The issues you're experiencing with XML special character handling and diff marker inclusion in the Roo Code system are not fully addressed in the available documentation or codebase. However, here are some insights based on the information I could find:

1. XML Special Character Handling: There is no specific documentation or code comments explaining how the Roo Code system handles XML special characters in tool parameters. This might be a gap in the documentation that needs addressing .

2. Diff Marker Inclusion: The documentation specifies that diff markers like + should not be included in the REPLACE content for the apply_diff tool. The REPLACE content should only contain the plain text lines intended for the final file ^[1][2].

3. Tool Failures: The failures you're encountering with tools like apply_diff might be due to inconsistencies in character handling or marker inclusion. The documentation does not provide specific guidelines on handling these inconsistencies, which could be contributing to the tool failures .

To address these issues, it might be beneficial to check for any open issues or submit a bug report in the Roo Code repository to highlight these gaps and seek further clarification or improvements in the documentation.

To continue talking to Dosu, mention @dosu.

---

^{Help Dosu learn! Give it feedback: Great Response | Irrelevant Answer | Incorrect Sources | Too Verbose | Hallucination | Bug Report | Other}