fix(agent): inject user-provided value into fillForm observe results

[tiwalayo]

Summary

• Fixes the fillForm agent tool dropping the value parameter, which caused LLMs to hallucinate placeholder values
• Injects the user-provided value into observe results before calling act()

Problem

The fillForm tool receives fields with both action and value:

fillForm({ action: "type email into the textbox", value: "user@example.com" })

But the execute function only used action when calling observe(), causing the LLM to guess what value to fill in. This resulted in hallucinated placeholders like test@example.com instead of the actual user-provided value.

This bug caused flaky behavior in:

• Login forms
• Search queries
• Data entry
• 2FA/OTP flows (when using custom tools that return codes)

Solution

After receiving observe results, inject the user-provided value into the result's arguments array for fill operations:

if (res.method === "fill" && fields[i]?.value) {
  res.arguments = [fields[i].value];
}

This ensures the correct value is passed to act() rather than the LLM-hallucinated placeholder.

Fixes #1789

---

Generated with Claude Code

---

Summary by cubic

Ensure fillForm uses the user-provided value by injecting it into observe results before act(), preventing hallucinated placeholders. This fixes incorrect inputs in login, search, data entry, and OTP flows.

• Bug Fixes
  • For observeResults with method "fill", set res.arguments = [fields[i].value] so act() types the correct value.

^{Written for commit 0e73368. Summary will update on new commits. Review in cubic}

[changeset-bot]

⚠️ No Changeset found

Latest commit: 0e73368

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

[greptile-apps]

Greptile Summary

This PR fixes a real and impactful bug where fillForm discarded user-supplied values and let the LLM hallucinate what to enter. For example, when asked to fill an email field with user@example.com, the LLM might guess test@example.com instead. The fix correctly injects the user-provided value into observe results before calling act(), ensuring the correct value is passed through to the actual form interaction. This resolves flaky behavior in login forms, search queries, data entry, and OTP flows. The implementation uses index-based correlation between observe results and fields, which is a reasonable approach for the intended use case.

Confidence Score: 4/5

• This PR is safe to merge. It fixes a verified bug with a straightforward and correct implementation.
• The PR addresses a real bug where user-supplied form values were being discarded, causing the LLM to hallucinate placeholder values instead of using the intended values. The fix is correct: it injects the provided value into observe results before calling act(), ensuring the right data flows to the form interaction. The code is well-guarded with a check for res.method === "fill" before overwriting arguments. The implementation uses index-based correlation between observe results and fields, which is a reasonable approach for matching results to input entries. The score is 4/5 (not higher) because the tests mentioned in the PR description remain unchecked, though the fix itself is sound.
• No files require special attention. The single changed file implements the stated fix correctly.

_{Last reviewed commit: 0e73368}

[cubic-dev-ai]

1 issue found across 1 file

Confidence score: 3/5

• There is a concrete user-facing risk in packages/core/lib/v3/agent/tools/fillform.ts: mapping form values by observe-result index can bind values to the wrong fill action when observe output order or count changes.
• Given the reported severity (7/10) and strong confidence (8/10), this is more than a minor edge case and could cause incorrect form submissions, so the merge risk is moderate.
• Pay close attention to packages/core/lib/v3/agent/tools/fillform.ts - index-based binding logic may misassign values when observation results are not aligned with input fields.

Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="packages/core/lib/v3/agent/tools/fillform.ts">

<violation number="1" location="packages/core/lib/v3/agent/tools/fillform.ts:62">
P1: Form values are injected by observe-result index, which can misbind values to the wrong fill action when observe output ordering/count differs from input fields.</violation>
</file>

_{Reply with feedback, questions, or to request a fix. Tag @cubic-dev-ai to re-run a review.}

[cubic-dev-ai]

P1: Form values are injected by observe-result index, which can misbind values to the wrong fill action when observe output ordering/count differs from input fields.

Prompt for AI agents

Check if this issue is valid — if so, understand the root cause and fix it. At packages/core/lib/v3/agent/tools/fillform.ts, line 62:

<comment>Form values are injected by observe-result index, which can misbind values to the wrong fill action when observe output ordering/count differs from input fields.</comment>

<file context>
@@ -56,7 +56,13 @@ export const fillFormTool = (
+        for (let i = 0; i < observeResults.length; i++) {
+          const res = observeResults[i];
+
+          if (res.method === "fill" && fields[i] !== undefined) {
+            res.arguments = [fields[i].value];
+          }
</file context>

[tiwalayo]

This is theoretically valid but not a practical concern. The instruction sent to observe() lists actions in explicit order, and the LLM returns results corresponding to that ordered list.

The original code already had this same implicit assumption—it looped over observeResults and called act() without any correlation mechanism. This fix does not introduce a new ordering assumption.

There is no ID or correlation mechanism between fields and observe results in the existing API, so index-based matching is the only option without a larger architectural refactor, which is out of scope for this bug fix.