v0.21.0-beta — the discriminating-power release

The discriminating-power release. A benchmark cannot resolve a difference smaller than its own noise, cannot compare an arm to a ceiling nobody else can build, and cannot test a capability its corpus has no questions for.

No default changes what a run selects, injects, or scores, and no judge prompt was added or edited — the judge-prompt fingerprint is unchanged, so baselines sealed against it stay comparable.

Pin the answer model

JudgeTemperature pinned the grader; nothing pinned the call being graded, so the answer model ran at the provider default. That self-disagreement is the floor beneath which no memory improvement is detectable, and it is invisible in a result: repeats of one configuration can flip verdicts with byte-identical retrieval.

• AnswerTemperature and AnswerSeed on ExternalBenchmarkOptions.
• IAnswerSamplingConfigurableAgent — how the values reach an agent AgentEval does not own. ChatClientAgentAdapter and LongMemEvalOracleReader implement it.
• ExternalBenchmarkResult.AnswerSampling — each parameter's fate per question: NotSupportedByAgent, DeclinedByAgent, SentUnverified, SentAndEchoed, EchoedDifferentValue, RejectedByProvider.

Values pass through as given — no assumption that 0 works — and a provider that rejects one fails the question rather than being silently retried without it. A successful call earns SentUnverified and nothing stronger: only a provider echo upgrades it, because a seed a provider ignores is worse than no seed at all.

A public, controllable oracle arm

The ceiling is a property of the dataset, not of any memory system, and a ceiling each caller re-implements is a different number per caller.

• LongMemEvalOracleProjector and LongMemEvalOracleReader are public, and RunOracleAsync returns the ordinary result shape — a QuestionResult per question plus SampleComposition.
• DistractorSessions adds non-evidence sessions from the question's own haystack; sessions borrowed from elsewhere are trivially ignorable and would measure a strawman.
• GoldSessionFraction keeps part of the evidence, rounding up and never below one session. 0 is rejected: it would make every question unanswerable by construction and score it anyway.
• ExternalBenchmarkResult.OracleProjection reports realised counts, because a level that degraded nothing and a level whose degradation did not matter look identical in a score.

A time-grounded corpus variant

Conversation dates live in metadata and in rendered text, and nothing forces an ingesting system to place messages in time — so a system that stamps everything with ingestion time still scores well on temporal questions, and the benchmark cannot tell it from one with real bitemporal memory.

• TemporalGrounding delivers session dates as real instants through the new ITimestampedHistoryInjectableAgent, and under TimestampsOnly removes the harness's in-text date scaffolding. Run it against TimestampsAndText: the difference between the two scores is the measurement.
• Any mode other than None requires the interface and fails before the first provider call otherwise. A text fallback would answer temporal questions from exactly the scaffolding the mode removes.
• LongMemEvalTimeGroundedCorpus — 12 authored questions embedded in the package: temporal-as-of, temporal-current, prospective-memory, four each. Not LongMemEval and not comparable with it. No message content contains an absolute date or a four-digit year, so every temporal expression is relative and resolving one requires the session's own timestamp — enforced by test.

Full notes in CHANGELOG.md, docs at LongMemEval getting started and Time-grounded probe.