Repository navigation
External query-tool adapter: separate job outcome from receipt verification #24209
Replies: 1 comment
|
The separation you are drawing between job outcome and receipt verification is the one that matters, and it is worth being strict about: verification is not a field on the outcome. It is a second, independent pass over the saved record, and it has its own outcomes. We maintain AER-1 (IETF draft-zambo-aer1-26), an open format for agent execution receipts, and this is the model we landed on once independent implementations started checking each other's records:
Your lost-acknowledgement case maps cleanly onto this: submission outcome unknown, verification not checked, and the adapter's job is to keep those two facts visible and separate until the backend reconciles. The failure mode to design against is the quiet merge, where "completed" absorbs "and we checked it" without anyone having checked. I run ops for Zambo, the hosted implementation of AER-1, so discount accordingly. The draft and its conformance corpus (143 vectors, 146 checks) are public if the vocabulary is useful for the adapter's evidence field. |
Uh oh!
There was an error while loading. Please reload this page.
Proposed independent integration: explicit query-job and receipt states
I am scoping an independently maintained LlamaIndex tool adapter for ODAP Gateway, an Apache-2.0 preview. This is an integration-design discussion, not an announcement of a working adapter or a request to add another integration package to this repository.
The use case is an agent requesting a governed analytical query whose external review and warehouse execution can outlive a tool call. LlamaIndex already supports human-in-the-loop workflows and async
FunctionToolimplementations. I am not proposing a replacement for either.Smallest adapter boundary
The proposed tool surface would separate preparing a structured query request, reading an existing job's state, and retrieving an evidence reference. Those are logical operations for the proposed adapter, not published SDK methods.
approve=trueargument or an approval-granting tool.awaiting_review,running,completed, andoutcome_unknown. Receipt verification is a separate field, so completed execution with missing or invalid evidence is not represented as verified delivery.ToolOutput.raw_output, but an in-memory tool output is not a durable audit store or an authorization boundary. Full results and access-controlled evidence stay in the application/backend.Initial validation scope
A first prototype would use the existing async
FunctionToolAPI and a mocked backend, with no warehouse credentials or paid queries. Proposed cases are changed query arguments after review, review refusal, repeated observation calls, lost submission acknowledgement, and a successful job whose receipt remains unverified. This would validate adapter behavior only, not warehouse execution or security certification.I reviewed the human-in-the-loop guide, the async/callback support in
FunctionTool, #22559 on replay, and discussion #21778 on pre-action authority. The narrower question here is how to represent a long-running query's observed outcome and separately verified evidence, not how to introduce another authority protocol.Does this independent, tool-level boundary fit existing LlamaIndex patterns, or is there a maintained external-job example it should follow? Feedback on the state contract and minimal test cases would be useful before implementation. No core changes or upstream package PR are proposed.
AI assisted this source review and draft. The adapter and tests described above are not implemented yet, and no production pilot is claimed.
All reactions