Repository navigation
Prototype an AI Run Review App
Use this checklist when an internal-tools team needs to connect an AI run investigation to a human review decision. Read From AI run logs to an internal review app for the architecture and tradeoffs.
This is a proposed prototype built from documented platform capabilities. It is not a ready-made template or a claim that the complete application has been runtime-tested.
- Complete one synthetic request and inspect its matching run using the gateway evaluation checklist
- Record the installed version and exact supported interfaces
- Use synthetic data and an account authorized for the test application
- Choose what success means: a reviewer can inspect a run, save a linked decision and read it back
The v0.4.1 README documents Data Models, CRUD APIs, workflow-backed APIs, React/TSX blocks and MCP. The newer log-structure note describes a development schema snapshot, not a stable API contract. In particular, do not assume its task-level grouping exists in v0.4.1.
- Match a synthetic prompt marker and time to a run
- Inspect the available model, tool and final-output evidence
- Identify the authorized API or built-in view that supplies the required reference and fields
- Record missing fields and version differences
The v0.4.1 console log handlers require console authorization. The application key used for model requests is not a grant to inspect administration logs.
Do not write directly into internal log tables or invent an endpoint from a table name. If the installed version lacks the required supported read interface, use manual inspection and record that integration gap.
Create a small Data Model for the review process. Suggested fields:
| Field | Purpose |
|---|---|
| Application reference | Identify the owning application |
| Run reference | Link to original execution evidence |
| Review status | New, investigating or resolved |
| Issue category | Group the team's own investigation categories |
| Reviewer note | Store the human conclusion |
| Timestamps | Track creation and latest update |
These are your proposed business fields, not built-in log columns. Prefer references over copied conversation content. Define required values, who can edit them and what happens when the referenced run is removed by retention cleanup.
- Generated CRUD operations can create and read the review record
- A React/TSX list and detail view display the correct record
- A status change persists and survives a fresh read
- The original evidence is accessible only to an authorized reviewer
- A second user without the required access cannot retrieve another application's evidence
- Unavailable or still-processing evidence is labeled accurately
Keep human review status separate from execution status and answer quality. A successful request can contain a poor answer; a resolved review does not change the original execution outcome.
If an agent needs to assist, begin with the documented progressive mcp_list, mcp_get, mcp_call discovery pattern. Verify what the installed version actually exposes, authenticated identity and permissions.
- Expose only capabilities needed by the agent
- Start with read-only inspection
- Review write operations independently before enabling them
- Keep the project context available for complex application work
MCP tooling and ready-to-use application templates are still being refined. Do not treat this checklist as a no-code, no-context application generator.
Choose retention, backups and access policies for both the source evidence and review records. Keep private prompts, credentials and full logs out of public issues; report only sanitized reproduction details through the project issue tracker.
Persistent run records support inspection. They do not alone prove durable execution, automatic crash recovery, safe retries or exactly-once external effects. Test such requirements separately against the installed version.
For deeper manual investigation, use Token BOM. Keep a short prototype receipt with passed, failed and untested checks before expanding the application.