Skip to content

Add ATIF v1.7 and full transcript artifacts for regular TRACE debug runs#378

Merged
mmprotest merged 7 commits into
mainfrom
mmprotest/add-full-transcript-and-atif-v1.7-artifacts
Jun 7, 2026
Merged

Add ATIF v1.7 and full transcript artifacts for regular TRACE debug runs#378
mmprotest merged 7 commits into
mainfrom
mmprotest/add-full-transcript-and-atif-v1.7-artifacts

Conversation

@mmprotest

Copy link
Copy Markdown
Owner

Motivation

  • Provide two additional debug artifacts for regular (non-Villani) TRACE-mode runs: a full transcript (transcript.full.json) and an ATIF v1.7 trajectory (trajectory.json).
  • Keep the change minimal and non-invasive by reusing existing in-memory transcript/message captures and the repository’s redaction behavior, without changing runtime behaviour, prompts, tool logic, benchmarking, or autonomous flows.

Description

  • New exporter module villani_code/atif_export.py with pure, side-effect-free builders build_full_transcript_artifact(...) and build_atif_trajectory(...) that serialize existing transcript and messages data and reuse maybe_redact_payload/token-normalization logic. Redaction is applied consistently when requested.
  • DebugRecorder.write_regular_trace_artifacts(...) added to call the pure exporters and safely write transcript.full.json and trajectory.json into the existing debug run directory using the recorder’s safe-write isolation.
  • Runner (villani_code/state.py) gains a single narrow eligibility predicate _should_write_regular_trace_artifacts() and terminal calls to _write_regular_trace_artifacts(...) at the existing finalisation points for regular-run completions/failures; this guard enforces: TRACE mode, runtime execution (not planning), not small-model, not villani/autonomous, not benchmark-enabled, and presence of a debug recorder.
  • Tests: added tests/test_regular_trace_artifacts.py with focused cases validating eligibility rules, ATIF structure and ordering, tool-call ↔ tool-result linkage, token metrics aggregation, non-duplication of cumulative requests in ATIF steps, redaction application, non-fatal export errors, and that execution results remain unchanged except for the presence of the two new files.

Testing

  • Code compiled: python -m compileall -q villani_code tests/test_regular_trace_artifacts.py (succeeded).
  • Focused unit tests for the new behaviour: pytest -q tests/test_regular_trace_artifacts.py — 12 passed (all tests in that file passed).
  • Larger subset run: pytest -q tests/test_regular_trace_artifacts.py tests/test_debug_mode_and_recorder.py tests/test_trace_runtime_path.py tests/test_trace_summary.py tests/test_state_runtime.py — 87 passed across that set.
  • Full test suite: pytest -q — the run showed unrelated pre-existing failures in other areas (11 failing, 607 passed, 11 skipped); the failing tests are outside the scope of this change (examples observed: tests/test_hardening_pass.py::test_interactive_plan_approval_still_works, tests/test_interactive_tokens.py::test_interactive_shell_is_textual_wrapper, several tests/test_loop.py cases, tests/test_mission_state_runtime.py::test_autonomous_summary_mirrors_to_mission_state, planning/TUI contract tests, and tests/test_villani_hygiene.py::test_low_authority_paths_are_blocked_from_mutation).

Notes / confirmations

  • Exact files added/modified: villani_code/atif_export.py (new), villani_code/debug_recorder.py (edit), villani_code/state.py (edit), and tests/test_regular_trace_artifacts.py (new test file).
  • The change does not modify benchmark, autonomous/Villani, small-model production codepaths, prompts, CLI/config surfaces, tool execution logic, approvals, retries, timeouts, validation, termination logic, message-history construction, or existing transcript generation; the only externally visible change is that eligible regular TRACE debug runs now write transcript.full.json and trajectory.json into the debug run directory.
  • ATIF fields intentionally left absent/null if not captured by the current runner: provider may be null when not supplied, model_name may be null if unavailable, step-level timestamps/durations/costs are not fabricated, and message fields may be null when the corresponding response has no text blocks.

Codex Task

@mmprotest mmprotest merged commit 5d29950 into main Jun 7, 2026
8 of 10 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant