Skip to content

LLM tool-loop fixtures are stale since patch #18 — 4 unit tests red #80

Description

@jack-champagne

Four session.llm native recorded tool-loop tests fail because the recorded request fixtures predate the patch #18 prompt change.

Failing: Anthropic API key / OpenAI OAuth / OpenCode proxy / zen — "drives a tool loop to a final text answer".

The assertion diff:

$.system[0].text
expected "Answer using tools when appropriate.\nUse the get_weather tool exactly once to look up Paris, then reply with exactly: Paris is sunny."
received "Answer using tools when appropriate.\n# Communicating with the user\n\nIMPORTANT: The rules in this section override any earlier instruction about response length, terseness, one-word answers, plain-text output, or rendering on a command line interface. Your text output is rendered as rich GitHub-f..."

Timeline — the drift predates the current release pin:

what when
recordings generated (upstream chore: generate) 2026-05-19 / 2026-05-24
communicating.txt appended to every model — b81fb756f2, d073fb617d, 153278891c (patch #18 + addenda) 2026-07-10
v1.17.3-amicode.9 pin bbe47b60 cut 2026-07-20
$ git show origin/local/amicode:packages/opencode/test/fixtures/recordings/session/native-anthropic-tool-loop.json | grep -c "Communicating with the user"
0

So this has been red since 2026-07-10 and shipped in amicode.9, which is in production. It went unnoticed for 18 days because the fork's workflows request blacksmith-4vcpu-* runners that this org has zero of — no test job has ever run on the default branch. See #2.

This is test-only. The prompt change is intentional product behaviour (rich markdown output in the amicode chat surface); it is the fixtures that are wrong, not the code. It is not a blocker for cutting v1.17.3-amicode.10, since the same condition already ships in .9.

Fix: re-record the four packages/opencode/test/fixtures/recordings/session/native-*-tool-loop.json against the current prompt assembly.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions