Add ATIF v1.7 and full transcript artifacts for regular TRACE debug runs#378
Merged
mmprotest merged 7 commits intoJun 7, 2026
Merged
Conversation
…tch-to-version-1.7 Fix regular trace ATIF trajectory shape
…-for-terminal-bench-compatibility Make debug artifacts ATIF-compatible
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation
transcript.full.json) and an ATIF v1.7 trajectory (trajectory.json).Description
villani_code/atif_export.pywith pure, side-effect-free buildersbuild_full_transcript_artifact(...)andbuild_atif_trajectory(...)that serialize existingtranscriptandmessagesdata and reusemaybe_redact_payload/token-normalization logic. Redaction is applied consistently when requested.DebugRecorder.write_regular_trace_artifacts(...)added to call the pure exporters and safely writetranscript.full.jsonandtrajectory.jsoninto the existing debug run directory using the recorder’s safe-write isolation.Runner(villani_code/state.py) gains a single narrow eligibility predicate_should_write_regular_trace_artifacts()and terminal calls to_write_regular_trace_artifacts(...)at the existing finalisation points for regular-run completions/failures; this guard enforces: TRACE mode, runtime execution (not planning), not small-model, not villani/autonomous, not benchmark-enabled, and presence of a debug recorder.tests/test_regular_trace_artifacts.pywith focused cases validating eligibility rules, ATIF structure and ordering, tool-call ↔ tool-result linkage, token metrics aggregation, non-duplication of cumulative requests in ATIF steps, redaction application, non-fatal export errors, and that execution results remain unchanged except for the presence of the two new files.Testing
python -m compileall -q villani_code tests/test_regular_trace_artifacts.py(succeeded).pytest -q tests/test_regular_trace_artifacts.py— 12 passed (all tests in that file passed).pytest -q tests/test_regular_trace_artifacts.py tests/test_debug_mode_and_recorder.py tests/test_trace_runtime_path.py tests/test_trace_summary.py tests/test_state_runtime.py— 87 passed across that set.pytest -q— the run showed unrelated pre-existing failures in other areas (11 failing, 607 passed, 11 skipped); the failing tests are outside the scope of this change (examples observed:tests/test_hardening_pass.py::test_interactive_plan_approval_still_works,tests/test_interactive_tokens.py::test_interactive_shell_is_textual_wrapper, severaltests/test_loop.pycases,tests/test_mission_state_runtime.py::test_autonomous_summary_mirrors_to_mission_state, planning/TUI contract tests, andtests/test_villani_hygiene.py::test_low_authority_paths_are_blocked_from_mutation).Notes / confirmations
villani_code/atif_export.py(new),villani_code/debug_recorder.py(edit),villani_code/state.py(edit), andtests/test_regular_trace_artifacts.py(new test file).transcript.full.jsonandtrajectory.jsoninto the debug run directory.providermay be null when not supplied,model_namemay be null if unavailable, step-level timestamps/durations/costs are not fabricated, andmessagefields may be null when the corresponding response has no text blocks.Codex Task