Validate to_replace in edit_file_by_replace AgentSkill

[li-boxuan]

What is the problem that this fixes or functionality that this introduces? Does it fix any open issues?

Issue 1

I was running a small subset of SWE-bench-lite and saw LLM doing some dumb stuff:

Diff checker told me the to_replace and new_content are exactly the same. This edit is meaningless, wastes tokens, and increases context length. For this specific task, the LLM kept repeating this dumb edit and eventually our loop detector terminated the task - not sure if it's related to the fact that the edit doesn't make "any progress".

Issue 2

Again running SWE-bench-lite, I noticed that sometimes there are more than one occurrence of to_replace in the same file. Although the prompt instructs LLM to include enough context to be able to uniquely identify a location, sometimes LLM just forgets about the full context of the entire file, OR even hasn't seen the full file yet. When this happens, there's a chance that we end up editing the wrong place.

---

Give a summary of what the PR does, explaining any non-trivial design decisions

Add sanity checks for to_replace parameter. Tell LLM the edit is wrong and hope it can fix the error.

---

Other references

My intuition is this won't change evaluation scores (beyond statistical error), but does more help than harm in general.

[xingyaoww]

lgtm!