Adopt Codex-style tool exposure and structured tool contracts

[dcramer]

Problem

Our prompt already tells Junior to stay on task, use tools, ask only when blocked, and continue until complete. The Codex codebase suggests the bigger difference is not a better paragraph in the system prompt. Codex moves many of those guardrails into runtime/tool design:

• typed model-visible context fragments instead of broad string assembly
• first-class tool exposure states (direct, deferred, model-only, hidden)
• search-loaded/deferred tools for large tool catalogs
• structured tool outputs with status, truncation, and continuation handles
• constrained ask-user and edit flows
• explicit continuation/completion audits for long-running work

Junior has much of the policy intent in packages/junior/src/chat/prompt.ts, but the current tool surface in packages/junior/src/chat/tools/index.ts and ToolDefinition appears more direct/prompt-driven. This makes behavior depend too much on model compliance.

Codex references

• Tool exposure is a runtime enum: codex-rs/tools/src/tool_executor.rs
• Tool planning decides what the model sees per turn: codex-rs/core/src/tools/spec_plan.rs
• Deferred search only appears when deferred tools exist: codex-rs/core/src/tools/spec_plan.rs
• Shell tool has explicit input and output schemas: codex-rs/core/src/tools/handlers/shell_spec.rs
• Ask-user is a constrained tool, not freeform text: codex-rs/core/src/tools/handlers/request_user_input_spec.rs
• Context is assembled from typed fragments: codex-rs/core/src/context/mod.rs
• Continuation/completion audit prevents shrinking the objective: codex-rs/prompts/templates/goals/continuation.md
• AGENTS.md loading is scoped, ordered, and budgeted: codex-rs/core/src/agents_md.rs

OpenAI Codex repo inspected locally at /home/dcramer/src/openai-codex, commit beca198b8a89f903d9dabc636c96c36ed281dcb2.

Proposed work

1. Add tool exposure metadata to ToolDefinition.

   • Suggested values: direct, deferred, modelOnly, hidden.
   • Keep direct as the default for compatibility.
   • Make the tool assembly path explicitly choose which tools are model-visible each turn.

2. Expand deferred discovery beyond MCP where appropriate.

   • Use searchable metadata derived from tool name, description, parameter descriptions, and owner/source.
   • Consider deferring plugin tools, low-frequency Slack tools, and specialized provider tools.
   • Keep high-frequency core tools direct.

3. Add structured output contracts for high-leverage tools.

   • Start with bash, file read/write/edit, MCP call, and Slack side-effect tools.
   • Include status, exact target, error classification, truncation info, artifact/session IDs, and continuation handles where relevant.
   • Avoid returning only freeform strings for workflows the model must continue from.

4. Add a first-class requestUserInput/elicitation tool.

   • Support one focused blocking question.
   • Support non-blocking auto-resolution for helpful-but-not-required input.
   • Make normal assistant text a poor path for asking operational questions.

5. Add a continuation/completion audit for resumed or long-running tasks.

   • Derive requirements from the active objective.
   • Require evidence for completion.
   • Prevent final answers that redefine success around a smaller completed subset.

6. Shift prompt assembly toward typed fragments.

   • Treat permissions, runtime capabilities, skills, tool catalogs, plugin context, and current instruction as bounded, testable fragments.
   • Prefer fragment snapshot/eval tests over more prompt prose.

Acceptance criteria

• ToolDefinition or its replacement supports explicit exposure.
• At least one non-MCP tool class can be deferred and discovered by metadata.
• Core high-leverage tools return typed results with enough information for reliable continuation.
• There is a first-class ask-user tool or equivalent elicitation primitive.
• Resumed/long-running tasks have a completion audit mechanism covered by evals or integration tests.
• Existing prompt rules are simplified where runtime mechanisms now enforce the behavior.

Out of scope

• A broad rewrite of Junior's personality or tone.
• Copying Codex prompts verbatim.
• Making every tool deferred.

[dcramer]

I think the narrow version of this is to make the deferred-tool mechanism generic, rather than starting with a broad Codex-style rewrite.

Proposed first slice:

1. Add an explicit tool exposure contract on ToolDefinition:

   • direct as the default
   • deferred for tools that should be searchable/callable but not native model tools
   • reserve modelOnly / hidden only if they are immediately needed

2. Add a small planning layer that separates:

   • direct native tools
   • deferred tool metadata
   • excluded tools

3. Introduce generic primitives:

   • searchTools: searches deferred tool metadata and returns exact callable names plus owner/source, description, schema summary, and relevant call notes
   • executeTool: invokes only deferred tools by exact returned name and arguments

4. Prove it with one non-MCP tool class first, likely plugin tools. Core tools stay direct. MCP stays on searchMcpTools / callMcpTool for the first slice, and can become a backend later if the generic path proves out.

5. Reuse the existing tool execution path for executeTool so validation, hooks, auth, tracing, error handling, and result normalization do not fork.

Acceptance should be behavior-level, not unit-test churn:

• deferred plugin tools are absent from the native model tool list
• searchTools can discover them from name, description, parameter metadata, and owner/source
• executeTool can invoke a discovered deferred tool through the normal executor path
• executeTool rejects unknown, hidden, or direct tools
• existing direct tools and MCP behavior remain unchanged

Verification should mostly be a focused integration-style wiring test plus an eval/manual local-agent check if model tool-choice behavior changes. Unit tests are only useful for tiny pure helpers like metadata extraction or exposure planning, not as the main proof.