Skip to content

P0: prove actual agent-to-tool bindings#268

Merged
pengfei-threemoonslab merged 3 commits into
mainfrom
codex/p0-agent-tool-bindings
Jul 12, 2026
Merged

P0: prove actual agent-to-tool bindings#268
pengfei-threemoonslab merged 3 commits into
mainfrom
codex/p0-agent-tool-bindings

Conversation

@pengfei-threemoonslab

Copy link
Copy Markdown
Contributor

What changed

This implements the P0 binding milestone: Agents Shipgate no longer assumes that every extracted tool declaration belongs to the configured agent.

  • Adds one deterministic AgentBindingGraphAssessment rooted at the configured agent.
  • Separates the complete tool_catalog from proven-reachable tool_inventory capabilities.
  • Normalizes direct tools, tool nodes, toolsets, workflows, subagents, and handoffs into canonical binding edges with provenance and completeness.
  • Adds reviewed, closed-world agent_bindings declarations with exact canonical selectors.
  • Makes semantic assessments, controls, action facts, capability facts, locks, diffs, and release decisions consume only the root-reachable surface.
  • Keeps unbound catalog tools visible for audit without allowing them to create capability findings or affect the verdict.
  • Routes missing, partial, dynamic, ambiguous, malformed, and conflicting binding evidence to insufficient_evidence outside the suppressible Finding model.

Framework normalization covers OpenAI Agents SDK, LangChain/LangGraph, CrewAI, Google ADK, n8n, and exact Codex plugin MCP inventories. MCP, OpenAPI, OpenAI tool JSON, and Anthropic artifacts remain catalogs unless structural wiring or a reviewed declaration binds them. Conductor remains non-pass-eligible until it emits the same normalized graph contract.

Why

The previous pipeline conflated extraction with reachability: every catalog declaration was assigned to one synthetic agent. That could both attribute unrelated provider tools to an agent and evaluate controls against capabilities the agent could not actually call. Conversely, it could not defend a passed result with evidence of the deployed agent's complete reachable tool and handoff graph.

This change makes binding a first-class static evidence dimension. A pass now requires an unambiguous root, a complete reachable graph, and pass-eligible identity, binding, effect, and authority evidence for every reachable capability. Static analysis still does not prove runtime behavior.

Public contract and migration

  • Package: 0.16.0b2
  • Runtime contract: 13
  • Report schema: 0.31
  • Packet schema: 0.10
  • Capability standard: 0.4
  • Capability lock / diff: 0.5 / 0.6
  • Action snapshot: 0.3
  • Safety corpus, receipt index, and qualification formats: v2

Pre-0.31 binding data and mixed lock versions fail closed with regeneration guidance. There is no legacy catalog-as-agent runtime switch; users requiring that behavior must pin 0.16.0b1.

Validation

  • Full pytest suite passed.
  • Existing 64/64 evidence-backed semantic canaries passed.
  • New 48/48 exact binding canaries passed and run as a separate blocking CI step.
  • All latency gates passed without changing the 10-second large-scan budget.
  • The large-scan gate also passed under -n auto plus coverage instrumentation.
  • Ruff and git diff --check passed.
  • Generated-schema drift check passed.
  • Wheel and sdist builds passed twine check.
  • The committed governance benchmark replay is green.

The large multi-framework sample now exposes 63 catalog tools and only five structurally bound SDK tools as reachable.

Review boundary

Shipgate preflight correctly routes this PR to a human because it changes protected schemas, policy semantics, CI, plugin/skill contracts, sample manifests, and reviewed binding declarations. No agent self-approval, suppression, baseline expansion, or policy weakening was used.

The four-week / three-design-partner rollout remains an external promotion requirement. This PR does not claim that rollout evidence is code-enforced or that the independent 100-case beta corpus has already been collected.

Copy link
Copy Markdown
Contributor Author

Addressed the binding trust-boundary review in commit 8e0e1e4:

  • MCP annotations and OpenAPI x-agents-shipgate now strip the reserved binding keys before constructing tools, while retaining non-binding annotations and emitting deterministic warnings.
  • Every possible_tool_id now produces partial_binding_evidence; release coverage also synthesizes a defensive gap if a graph ever omits that issue, and passed consistency rejects non-empty possible surfaces.
  • OpenAI Agents SDK wiring is now represented as typed agent-level observations rather than tool-anchored annotations, so pure routers with tools=[] retain handoffs.
  • Added strict end-to-end hostile MCP and OpenAPI canaries, incomplete-edge/handoff regressions, and router coverage.

Validation: exact CI aggregate command passed (3163 passed, 5 skipped, 88.84% coverage), all binding/semantic canaries passed, schema generation is clean, and the unchanged 10-second large-scan gate passed. Shipgate self-verification still correctly requires maintainer review for this trust-root change; no self-approval is claimed.

@pengfei-threemoonslab
pengfei-threemoonslab marked this pull request as ready for review July 12, 2026 05:16
@pengfei-threemoonslab
pengfei-threemoonslab merged commit ecd7d12 into main Jul 12, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant