fix: Decimal JSON serialization for Usage.cost_usd — required for M2 cost stamping

[Ken Chau (kenotron-ms)]

What broke

M2 cost-tracking added cost_usd: Decimal | None to Usage and SessionStatus. Every orchestrator (loop-streaming, loop-basic, loop-events) calls model_dump() to serialize these models into events. With no field-level serializer, model_dump() emits raw Decimal values, which then crash json.dumps() at downstream JSONL write sites.

The milestone-wide sweep identified at least 5 such write sites across 4 modules:

• amplifier-module-hooks-logging (fix(core): Handle encoding errors in exception logging for Windows #5 attempted a per-module fix via mode='json')
• amplifier-module-hooks-backup (unpatched)
• amplifier-module-hook-shell (unpatched)
• amplifier-module-context-persistent (unpatched)
• amplifier-foundation/serialization.py (#190 attempted a module-level Decimal branch)

A "patch every consumer" approach doesn't scale to N orchestrators or future hook modules.

Root cause

The Decimal field had no @field_serializer, so model_dump() returned the raw Decimal object — JSON-incompatible. Adding a field-level serializer that converts Decimal -> str on every dump path is the correct boundary defense.

Solution

Define @field_serializer("cost_usd", when_used="always") on both Usage.cost_usd and SessionStatus.cost_usd. when_used="always" is explicit (it matches Pydantic 2.x's default for @field_serializer, but stating it makes the contract unambiguous and survives any future Pydantic default change without surprise).

• Usage.cost_usd — serializer returns str(v) if v is not None else None
• SessionStatus.cost_usd — same shape
• ToolResult.get_serialized_output() — adds _json_default(obj) handler + default= argument to json.dumps() for the non-Pydantic path (tool outputs that may carry Decimal values e.g. {"price": Decimal("9.99")})

Invariants preserved:

• Direct usage.cost_usd access still returns Decimal for in-memory math
• None stays as None (preserves the None != Decimal("0") invariant for "unknown" vs "free")
• Precision is preserved as a string ("0.0000123", not rounded or truncated)
• Existing field_validator rejecting float still fires
• Plain model_dump() and model_dump(mode='json') produce identical output

Verified behaviors (10 test cases, all PASS)

• ✓ model_dump() returns string for cost_usd
• ✓ json.dumps(usage.model_dump()) doesn't crash
• ✓ Direct usage.cost_usd access still returns Decimal
• ✓ Existing field_validator rejecting float still fires
• ✓ extra="allow" still works for provider-specific fields
• ✓ None survives as None (not "None")
• ✓ Round-trip model_validate(model_dump()) rebuilds with Decimal
• ✓ Precision preserved across scales: 0.0000123, 12345.6789, 0, 1E-9
• ✓ Plain model_dump() and mode='json' produce identical output
• ✓ int coerces to Decimal then serializes to string

Test coverage

• New: tests/test_cost_usd_serialization.py — 25 tests, all passing
• Adjacent tests: 129 tests still passing (test_message_models, test_session, test_tool_result_autopop, test_interfaces, test_generated_equivalence)
• E2E smoke test: PASSED (run by user per release-mandate Pre-Merge Gate)

Version bump

1.5.0 -> 1.5.1 (PATCH — bug fix)

• pyproject.toml
• crates/amplifier-core/Cargo.toml
• bindings/python/Cargo.toml
• python/amplifier_core/__init__.py

Files changed

File	Changes
python/amplifier_core/message_models.py	+13 lines: @field_serializer on Usage.cost_usd
python/amplifier_core/models.py	+34 lines: @field_serializer on SessionStatus.cost_usd + _json_default(obj) + default= in ToolResult.get_serialized_output()
tests/test_cost_usd_serialization.py	+234 lines: new test file covering all 10 behaviors
Version files	+7 lines: 1.5.0 -> 1.5.1
Cargo.lock	regenerated by Rust toolchain

Total: 286 insertions, 7 deletions across 8 files.

Unblocks M2/M3 milestone (#225)

This PR is a prerequisite for the M2/M3 cost-tracking milestone. Once merged, the following 11 PRs can drop per-consumer Decimal sanitizer code:

Foundation & framework:

• amplifier-foundation #190 — can drop the Decimal -> str branch from serialization.py (will be cleanup commit on existing PR after this merges)

Hooks:

• amplifier-module-hooks-logging fix(core): Handle encoding errors in exception logging for Windows #5 — close as superseded (mode='json' switch becomes redundant)

Provider PRs (M2):

• amplifier-module-provider-anthropic feat: add --local-source to E2E smoke test + fix input_schema test #52
• amplifier-module-provider-openai feat: polyglot contracts — gRPC bridges, native transport, WASM, proto conversions #32
• amplifier-module-provider-azure-openai feat: add retry_after and delay_multiplier to LLMError for generalized backoff #24
• amplifier-module-provider-gemini fix: resolve fast class by cost_tier=low #28
• amplifier-module-provider-vllm feat: add input_schema to Tool protocol to match ecosystem convention #22
• amplifier-module-provider-ollama docs: add metadata field to context compaction event contracts #14
• amplifier-module-provider-chat-completions fix(core): Handle encoding errors in exception logging for Windows #5

App/consumer PRs (M3a/M3b):

• amplifier-app-cli #171 (unchanged)
• amplifier-module-hooks-streaming-ui fix: catch BaseException in cleanup loops to survive asyncio.CancelledError #11 (unchanged)

Modules no longer needing PRs (automatic safety from this fix):

• amplifier-module-hooks-backup — no follow-up PR needed
• amplifier-module-hook-shell — no follow-up PR needed
• amplifier-module-context-persistent — no follow-up PR needed

Tracked in: microsoft-amplifier/amplifier-support#225

Side finding (future work)

The version bump script scripts/bump_version.py has a regex bug: the pattern ^((?:__)?version\s*=\s*\") treats __ as an optional prefix but doesn't account for the suffix in __version__ = "...". The fix is one-line (^((?:__)?version(?:__)?\s*=\s*\")), tracked for a follow-up PR.

---

Relates to: microsoft-amplifier/amplifier-support#225

[Ken Chau (kenotron-ms)]

Reopened with refinement: switched from default field_serializer (fires only on mode='json') to when_used='always' so plain model_dump() (used by every orchestrator: loop-streaming, loop-basic, loop-events) also produces JSON-safe output. Verified across 25 new tests + 129 adjacent tests + E2E smoke test PASSED. Version bumped 1.5.0 → 1.5.1 per release-mandate.