[Feature]: Make traces replayable and useful for step-by-step debugging

[GenTang]

已有 Issue / Existing Issues

• 我已搜索过现有 issues，没有找到重复项。 / I searched existing issues and found no duplicate.

痛点与背景 / Problem and Context

Currently, Yada traces show run summaries and event timelines, but they fail to answer the most critical questions when debugging the Agent:

• What exactly did the model receive at a specific step?
• Why did the model choose these specific tools?
• What were the complete inputs and outputs of the tools?
• How did the previous round's assistant message, reasoning_content, and tool results feed into the next request?
• Why did two runs of the exact same task yield different results?

Reproduction command:

uv run yada eval \
  --case benchmarks/swebench_verified/pytest-10051 \
  --agent yada \
  --yes

An example run contains:

• 21 model turns
• 26 tool results
• 117 trace events

However, yada-trace only displays something like this:

step=8 model_request messages=19 chars=19514
step=8 assistant finish=tool_calls tools=search_code,run_command
step=8 tool_call tool=run_command args=argv,purpose,timeout_seconds
step=8 tool_result tool=run_command status=ok exit=4

From this report, it is impossible to know:

• What command was executed in step 8.
• What the stdout and stderr were.
• Why the command returned exit code 4.
• The actual arguments and results for search/read/apply_patch.
• The complete messages, tools, and model parameters sent to DeepSeek in that turn.
  The raw JSONL contains more tool data than the yada-trace report, but model_request still only stores the message count and character count:

{
  "step": 8,
  "context": {
    "message_count": 19,
    "serialized_chars": 19514
  }
}

Thus, there are currently two primary issues:

1. The collection layer does not save the complete model request; the context is irretrievably lost.
2. The Viewer hides the tool parameters and results that already exist in the JSONL.

The current trace acts more like a runtime statistic rather than an execution trace usable for debugging and auditing.

期望的解决方案 / Proposed Solution

Introduce an explicit debug trace mode while retaining the current lightweight default output. Proposed minimal CLI:

yada eval ... --trace-level debug

yada-trace <trace.jsonl> --step 8
yada-trace <trace.jsonl> --verbose

--trace-level summary will remain the default; detailed data is recorded only when debug mode is explicitly enabled.

The Debug trace should be able to reconstruct the following pipeline:

model request
    ↓
assistant response
    ↓
planner decision
    ↓
tool calls
    ↓
tool results
    ↓
next model request

Suggested records:

1. Complete and sanitized model requests:

• model
• messages
• tools / tool schemas
• thinking
• reasoning_effort
• max_tokens
• request/step correlation ID

2. Complete normalized responses:

• content
• reasoning_content
• tool_calls
• finish_reason
• usage
• response ID
• system fingerprint

3. Complete tool interactions:

• Actual tool argument values (not just keys)
• stdout / stderr
• exit code
• read/search outputs
• apply_patch diffs and execution results

4. Reproducibility metadata:• Yada version and git commit• benchmark/case ID

• workspace base commit
• model config
• trace schema version

5. Viewer enhancements:• --step N to expand a single step in detail.

• --verbose to expand the full execution timeline.
• Default mode continues to display concise summaries only.

Regarding reasoning_content:

• Default: Continue storing only the length and SHA-256.
• Explicit: Store raw content only when --trace-reasoning is used.
• Constraint: Writing to disk must not interfere with passing reasoning_content back to DeepSeek during runtime.

File size optimization:

To control file size, any of the following approaches can be used:

• Save full payload per turn (if manageable).
• Initial full snapshot + message deltas for subsequent turns.
• Content-addressed references for the system prompt and tool schemas.

Regardless of the internal format, the viewer must be able to deterministically reconstruct the exact payload sent in each turn.

验收标准 / Acceptance Criteria

•[ ] Add --trace-level summary|debug, with summary as the default.
•[ ] Debug trace can reconstruct the complete JSON payload sent to the model for any given turn.
•[ ] yada-trace --step N displays the model input, model output, and full tool interactions for that step.
•[ ] yada-trace --verbose displays the complete execution timeline.
•[ ] Shows argv, stdout, stderr, and exit_code for run_command.
•[ ] Shows patch content and modification results for apply_patch.
•[ ] Does not save raw reasoning_content by default.
•[ ] --trace-reasoning works with debug trace to save and restore reasoning_content.
•[ ] Authorization tokens, API keys, and common secrets are strictly redacted from traces.
•[ ] Automated tests verify that the reconstructed payload is functionally equivalent to the payload passed to the completion client.
•[ ] Traces include Yada commit, case ID, workspace base commit, and model configuration.
•[ ] Legacy trace schemas remain readable or return a clear schema compatibility error.
•[ ] Disk overhead and standard output for the default summary mode remain practically unchanged.

评测、Token 与延迟影响 / Evaluation, Token, and Latency Impact

This feature must function as passive observation and should NOT alter:

• The prompt sent to the model.
• The Agent's tool selection logic.
• Benchmark resolved rates.
• Model token usage.

The following tests/measurements should be added:

1. Payload parity check: Using a fake completion client to assert actual provider payload == reconstructed trace payload.
2. Behavior parity on pytest-10051: Comparing summary vs debug modes must yield the same resolved result, token usage, and tool call sequence.
3. Performance metrics for a 30-step run: Measure trace file size, JSON serialization overhead, and total run latency impact.

Note: If using full request snapshots, file size may bloat due to conversation history duplication. Prefer delta or content-addressed designs, provided they do not sacrifice deterministic reconstructability.