Propose durable agent state schema 2.0 contract

[Tamir Dresher (tamirdresher)]

Contract discussion, not runtime activation

This schema-only proposal separates execution/delivery completion from evictable
conversation transcript state. It changes only schemas/ JSON and Markdown:
no .NET/Python implementation, DTOs, converters, runtime activation, samples,
test-project links, or migrations. Positive and negative validation cases are
language-neutral JSON test data, not runtime code.

Python and .NET must not emit schema 2.0 until joint agreement and compatible
readers/consumers exist. Agreement on compatible-reader/consumer semantics and
the enforced rollout floor is a prerequisite to merging this contract proposal.
Any 2.0 writes require separately reviewed implementations and deployment gates;
schema validation or merging this PR does not authorize them.

Proposed wire concepts

• Immutable terminal response/error envelopes keyed by request correlation ID,
  outside conversationHistory.
• Independent completion receipts/tombstones that survive result payload expiry
  and transcript pruning; explicit available versus completed-but-unavailable
  results, distinct from pending/unknown requests.
• Optional, separately versioned runtime-profile data under the existing
  historyBinding name, preserved by compatible writers and validated only by
  runtimes relying on it. No shared binding shape or effective-owner policy.
• V2-only lossless developer-role/string-argument/URI message shapes, safe opaque JSON
  content, and a named optional terminal response value that distinguishes
  absence from explicit null/false/zero/empty values.
• Opaque session continuation, existing whole-entity TTL, ingestion watermarks,
  truncation evidence, extension-data preservation, and known-discriminator rules.

The JSON Schema checks structural constraints. Shared documentation specifies
cross-map consistency, immutable completion, full entity-local atomic commits, lazy-expiry
polling, trust boundaries, migration limits, and unresolved deployment policy.
Fixtures are synthetic review data, not captured production output; the
Python-shaped 1.2 fixture is explicitly not byte-for-byte Python serializer
output. No runtime's support for proposed 1.2/2.0 is inferred from them.

Historical 1.0/1.1/1.2 message/content validation remains unchanged from the
pre-widening proposal. Explicit versioned definitions and full-envelope
positive/negative cases keep the expanded mappings v2-only. Existing opaque
unknown.content shapes and all four persisted fixture blobs remain unchanged.

Compatibility and rollout risks

This is proposed as a major semantic revision because completion/replay authority
moves away from transcript presence. Is major 2 the right mechanism, or is an
enforceable same-major rollout gate preferable?

At base 62afdbac03f0a81b0917abe6328203b706a2f294, Python's reader checks that a
version is present but does not gate its value, and its data reader does not
preserve unknown data fields. The revised C# preview is fail-closed. This is a
cross-runtime compatibility issue to resolve together: a major number or unknown
field preservation alone does not protect against duplicate execution or lost
completion state.

The rollout floor must cover readers, duplicate lookups, pollers, writers,
rollback writers, hosting consumers, and scheduler/dashboard tooling. Participants
must behave compatibly or reject 2.0 before processing/mutation and be isolated
from it. A rollback writer must not downgrade or discard completion evidence.
Changing only the version or adding empty maps to old pruned state is not a
migration. Whole-entity TTL, session recreation, late duplicates, receipt storage
growth, and continuation/numeric interoperability also need agreement.

Feedback requested

Ahmed/Python maintainers and .NET/Durable Task maintainers:

1. Schema shape, request identity scope, and versioning mechanism.
2. Separately versioned runtime-profile definitions and validation for runtimes
   relying on them. Fixed-owner policy remains runtime-specific; the common
   contract does not standardize a binding object or per-run transition policy.
3. Success/error envelope, cancellation, and continuation-byte compatibility.
4. Result expiry/removal, lazy-cleanup pollers, receipt lifetime, whole-entity TTL,
   session generation, and late duplicate semantics. Expired lookup exposes
   retained success/failure with completed-but-result-unavailable and no payload.
   Older receipts lacking authoritative outcome must not invent one or reopen
   execution; their migration/lookup representation still needs agreement.
   The agreed behavior requires aligned ADR wording and Python receipt/lookup updates.
5. Numeric/Unicode/timestamp limits, unknown-field preservation, and discriminator policy.
6. Enforceable rollout floor, safe migration, and compatible rollback consumers.

Related context: #88 (ADR 0032).
That ADR motivates compaction/retention discussion; it does not mandate this
exact schema. This PR is ready for cross-runtime review, but the contract remains
proposed and does not authorize schema 2.0 activation. Dependent implementation
PRs will remain drafts until the contract and rollout requirements are agreed.

[Copilot]

Copilot review overview

🟡 Changes recommended

The 2.0 schema currently allows unconstrained correlationId values in typed transcript entries, undermining the stricter identifier contract used by the receipt/result maps and the stated invariants.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Review tier: Lite
Findings: 1

New issues introduced by this change (1)

Severity	Finding
	schemas/​durable-agent-entity-state.json — In schema 2.0, conversationHistory entries are constrained to…

What changed in this PR

This PR proposes a durable agent state schema 2.0 contract (plus legacy review snapshots) intended for joint cross-runtime review, explicitly without authorizing any runtime to emit 2.0. It adds documentation and synthetic fixtures to discuss new “terminal completion outside transcript” concepts and rollout/compatibility semantics.

Changes:

• Added a Draft 2020-12 JSON Schema that enumerates snapshot versions (1.0.0/1.1.0/proposed 1.2.0/proposed 2.0.0) and introduces 2.0 concepts like terminalResults, completionReceipts, and historyBinding.
• Added documentation describing semantic invariants, compatibility concerns, and rollout gating expectations.
• Added synthetic JSON fixtures for reviewing 1.2 (Python-shaped) and 2.0 state shapes, including a pruned-transcript example.

File	Description
schemas/​README.md	Documents the proposed 2.0 contract semantics, invariants, and rollout/compatibility constraints.
schemas/​durable-agent-entity-state.json	Draft 2020-12 schema defining legacy snapshots and proposed 2.0 structure (receipts/results/binding).
schemas/​fixtures/​README.md	Explains fixture provenance and how to interpret the synthetic examples.
schemas/​fixtures/​shared-durable-agent-state-1.2-python-shape.json	Synthetic 1.2 “Python-shaped” review example emphasizing unknown-field preservation discussion.
schemas/​fixtures/​shared-durable-agent-state-2.0.json	Synthetic 2.0 example showing an available success and an expired/unavailable failure receipt.
schemas/​fixtures/​shared-durable-agent-state-2.0-pruned.json	Synthetic 2.0 example with empty transcript but retained completion evidence and TTL/truncation diagnostics.

Suppressed comments (2)

schemas/durable-agent-entity-state.json:218

• For schema 2.0, agentResponse should also constrain correlationId to the shared identifier definition so transcript entries can’t contain correlation IDs that would be invalid as terminalResults/completionReceipts keys.

      "properties": {
        "$type": { "type": "string", "const": "response" },
        "usage": {

schemas/durable-agent-entity-state.json:231

• For schema 2.0, agentErrorResponse should constrain correlationId to identifier as well, to keep correlation IDs consistent across transcript entries and the receipt/result maps.

      "properties": {
        "$type": { "type": "string", "const": "errorResponse" },
        "usage": { "$ref": "#/$defs/usage" }

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[Tamir Dresher (tamirdresher)]

Thanks for the feedback. I am going to update the schema proposal based on these threads. I am marking this PR ready for review now so we can agree on the shared contract while I start opening the dependent C# implementation PRs as drafts. Those implementation PRs will remain drafts and will not activate schema 2.0 until this contract and the rollout requirements are agreed.

[Copilot]

Copilot review overview

🔵 Needs a closer look

Cross-runtime schema semantics remain unresolved, and the moderate legacy-version scoping issue requires correction and human review.

Review tier: Lite
Findings: None

Issues resolved since last review (1)

Severity	Finding
	schemas/​durable-agent-entity-state.json — In schema 2.0, conversationHistory entries are constrained to… View resolved comment

[Copilot]

Copilot review overview

🔵 Needs a closer look

Two moderate schema issues remain unresolved, and the cross-runtime contract and rollout risks require human review.

Review tier: Lite
Findings: None

[Ahmed Muhsin (ahmedmuhsin)]

Approved for the schema-only contract at eff12f4. The review concerns are addressed. The documented reader/consumer agreement remains required before merge, and runtime implementations and v2 activation need separate review and deployment gates.