Semantic convention proposal: a span for context compaction, and an attribute for the served context ceiling

[behrnt-slatgng]

Disclosure: I build Grunz, a hosted chat and coding agent on open weights. Posting a semantic-convention proposal, since OpenLLMetry is where these conventions actually get settled.

There is no span for context compaction, and it is the operation that most often explains a bad run

The current conventions cover the LLM call well: model, prompts, completions, token counts, cost. What they do not cover is the operation that happens between calls in any long-running agent — when the context window fills and the history is compressed.

That compression is a lossy write performed by a model and validated by nothing. It happens inside a framework's memory layer, below the level anything instruments, and it emits no telemetry.

The practical consequence is that a trace shows every call in perfect detail and says nothing about the moment the run went wrong. The moment it went wrong is rarely the call with visibly bad output — it is the compaction several steps earlier, where a constraint was summarised away. Constraints are dropped preferentially in any summarisation, because they are short, low-frequency and look unimportant to whatever is compressing. So the failure is systematic rather than random, and completely invisible.

Proposed: a gen_ai.context.compaction span

Attributes, kept minimal so frameworks will actually emit them:

• gen_ai.context.compaction.tokens_before / tokens_after
• gen_ai.context.compaction.strategy — summarize, truncate, sliding_window, evict
• gen_ai.context.compaction.messages_dropped
• gen_ai.context.compaction.system_prompt_retained (boolean) — cheap, and it catches the most common real failure
• optionally the dropped content itself, gated behind the same content-capture flag that already governs prompts and completions

Two derived signals fall straight out and are useful on their own: compactions per run, and time-to-first-compaction. In my experience a run that compacts in its first third rarely ends well, and that is knowable long before the output is.

Related: the served context ceiling is not the model card's number, and it is not currently observable

The window advertised on a model card is a property of the weights. The window an endpoint actually serves is whatever the provider configured. Most hosted endpoints serve around 32K tokens regardless of what the card claims, a few reach 256K, and I have not found one genuinely serving the 1M figures that get quoted.

It fails silently — no 413, no truncation notice, the front of the context is evicted and the model answers confidently from the remainder. Since the front is where the system prompt lives, the symptom is "the model stopped following instructions", which nobody attributes to a window.

There is currently no attribute for it. gen_ai.request.max_context_tokens (declared) alongside an observed or probed value would let a backend flag "this prompt exceeded the ceiling this endpoint has been seen to honour" — which today produces no signal at all, in any tool.

This one is arguably more important than the compaction span, because it is a property of the provider rather than of the application, and providers change it without announcement. An agent that silently degrades because an endpoint's serving config changed is indistinguishable, from inside current telemetry, from an agent that got unlucky.

Why here

Frameworks have little incentive to make their memory layer's lossiness legible — it is the part that looks like magic. Conventions are the lever: once the span exists in the spec, emitting it becomes the obvious thing to do, the way the generation span did.

Happy to help draft the attribute set properly, or to contribute traces that demonstrate the failure if a concrete example is useful.