fix(agent-platform): read the janitor's args_schema in tool schemas - #69202
Merged
Conversation
dmarticus
reviewed
Jul 8, 2026
dmarticus
left a comment
Contributor
There was a problem hiding this comment.
thanks for this fix! I left some comments around just committing to a single direction for this change rather than supporting the legacy behavior, since we don't have any bundles in production that are actually malformed. I'll approve once that's addressed!
The one production writer of custom-tool schema.json is the janitor's
typed pipeline (typed-bundle.ts writeToolSourceAndSchema), which writes
`{ description, args_schema }`; agent-shared's readTypedBundle already
reads `args_schema`. The runner's loadCustomSchema was the only reader
still parsing the legacy `args` key, so it silently fell back to a bare
`{ type: 'object' }` — the model never saw the parameter schema and could
only guess arguments from prose (a transcript shows it concluding the
tool has no parameters).
The runner now reads `args_schema` and nothing else, matching
readTypedBundle and the sole production writer's key. No production bundle
uses the legacy `args` key (confirmed against the production buckets), so
this commits to a single direction rather than two ways to spell the same
contract. The silent degradation is still guarded on both paths: an
unreadable schema.json (`custom_tool.schema_unreadable`, with the error)
and a parsed schema with no `args_schema`
(`custom_tool.schema_missing_args_schema`) — the latter being the exact
failure mode of this bug had the key drifted again, and now unambiguously
wrong.
Test fixtures use the production `args_schema` shape (agent-runner +
agent-tests). A single regression test welds the writer to the reader: it
authors through the real writer (writeToolSourceAndSchema) and asserts the
schema survives the writeToolSourceAndSchema → loadCustomSchema round-trip
(it collapses to a bare object schema without the fix).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Aidan945
force-pushed
the
fix/custom-tool-schema-key
branch
from
July 9, 2026 00:50
6597332 to
2a60baa
Compare
Contributor
Author
|
Makes sense. Good to get ahead of it. Dropped the fallback and the legacy test, so |
2 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Models see custom tools with no parameters. The janitor writes
tools/<id>/schema.jsonas{description, args_schema}(writeToolSourceAndSchema), but the runner'sloadCustomSchemaparses{description, args}— so every typed-pipeline tool's schema silently resolves to a bare{type: "object"}and the model can only guess arguments from prose. A session transcript shows the model concluding "the tool's schema shows no explicit input parameters".Both sides pass their own tests because the runner's fixtures hand-write
schema.jsonwith the runner's key — two hand-maintained copies of one wire contract. Among the readers of this file, the runner was the only one parsingargs;readTypedBundlealready readsargs_schema.Changes
agent-runner/src/loop/build-agent-tools.tsloadCustomSchemareadsargs_schemaargskey (confirmed by the maintainer), so there is a single path, no fallbackcustom_tool.schema_unreadable, with the error) and a parsed schema missing both keys (custom_tool.schema_missing_args_schema)build-agent-tools.test.tswriteToolSourceAndSchemaagent-tests/src/cases/argsfixtures →args_schemaThe key rename is the direct fix; the two warn logs and fixture flips are the durable half — they make this class of drift visible and tested rather than silent. Deliberately not in this PR: single-sourcing the schema.json codec in agent-shared (natural follow-up; kept out to avoid colliding with in-flight storage-boundary work).
Sibling fix in the same authoring loop: #69201.
How did you test this code?
Ran locally against a full dev stack (all four agent services + Postgres/Redis/Kafka/SeaweedFS):
custom tool parameters survive the writeToolSourceAndSchema → loadCustomSchema round-trip, authored through the real writer against the real test bundle store. With the fix reverted it fails (parameters collapse to the bare{type:"object"}fallback) — the writer→reader weld no existing test provided.pnpm vitest run src/loop/build-agent-tools.test.ts→ 25 passed; full@posthog/agent-runnersuite → all assertions pass;typescript:check,oxlint,oxfmt --check→ clean on agent-runner and agent-tests.{}; after, it passes correct structured arguments. Reproduced both directions.Automatic notifications
Docs update
No docs change needed.
🤖 Agent context
Autonomy: Human-driven (agent-assisted)
I hit this while building agents against a local stack — a model calling my custom tool with empty arguments — and traced it from the transcript to the key mismatch; the fix and tests were developed in a Claude Code session (Fable 5) under my direction, with every executable claim above verified by running it. Key decision: fix the reader, not the writer — a census of every schema.json consumer showed the runner was the single outlier, and frozen bundles are immutable, so only the reader side heals revisions that already exist.
🤖 Generated with Claude Code