ModelOutputThunk structural cleanup — ._meta partitioning, raw responses, _thinking

[ajbozarth]

Background

During the design discussion for #793, @jakelorocco flagged that the ModelOutputThunk structure has broader issues worth addressing as a separate issue. The mot.generation sub-object pattern introduced in #793 is the first chunk; this issue covers the rest.

Current state

ModelOutputThunk still has several rough edges:

• ._meta is a catch-all dict inherited from CBlock. It currently holds a mix of things: raw provider responses (e.g. oai_streaming_usage, litellm_streaming_usage, chat_response), the telemetry_span, and other backend-specific data. There's no contract for what lives there.
• Computation logistics fields (_async_queue, _first_chunk_received, etc.) are flat private attributes with no grouping.
• _thinking is a private field but arguably user-facing content (like content) — it may belong public.

Proposed areas of work

1. Raw provider responses — give backend-specific raw responses a dedicated typed home instead of splatting into ._meta
2. telemetry_span — if span data lives in ._meta, move it to a sub-structure with a clear contract
3. Computation logistics — consider grouping async/streaming internal fields (_async_queue, _first_chunk_received, etc.) similarly to how generation groups metadata
4. _thinking — evaluate whether this should be public

Notes

• Each area can likely be a self-contained chunk, but it's worth agreeing on the full picture first so nothing conflicts
• The ._meta rename/cleanup is the thorniest since CBlock owns it — any change there touches the whole inheritance chain

[ajbozarth]

@jakelorocco I'm unsure if this fits into the deliverable for 0.5 or not, if you think I should tackle it this sprint I'll assign it to myself and add it to the iteration. Otherwise this will just sit until other work is done

[psschwei]

I'm not sure if the areas that would be changed under this (or #793 for that matter) would overlap with changes coming out of #891 , but in the case that there were overlaps, I would think that #891 is a higher priority to land first.

[ajbozarth]

Flagging design tension with #891 before this gets assigned.

Three specific collision points:

1. _async_queue / computation logistics fields
Sub-issue #901 (stream_with_chunking()) is explicitly built around the single-consumer constraint on _async_queue, including cancellation behavior. If #909 restructures those internal fields first, the stream_with_chunking() implementation will need to adapt to whatever new shape they land in — doing them in the wrong order creates rework.

2. telemetry_span in ._meta
If #909 relocates telemetry_span, it intersects with an open concern from the #891 review: when does GENERATION_POST_CALL fire in the streaming path, and how does TokenMetricsPlugin see it? The stream_with_chunking() path bypasses BaseSamplingStrategy entirely, so where the span lives affects whether that concern is resolved cleanly or left dangling.

3. Phase 2 of #891 vs. logistics grouping
Epic #891 scopes a second phase where the MOT owns chunking via a new stream_parsed_repr method, driven by the need to relax the single-consumer constraint. Issue #909's "computation logistics" grouping touches the same internals. If #909 lands first without considering that shape, it may make the Phase 2 MOT API harder to design.

Recommendation: hold #909 until #891's core sub-issues (#898–#901) are landed. Once stream_with_chunking() is stable, the scope of #909's logistics grouping will be much clearer — and some of it may naturally align with or be superseded by the Phase 2 design.