MemKraft 3.0.2
MemKraft 3.0.2
Release date: 2026-07-17. This patch packages the evidence-context and benchmark
reliability work completed after 3.0.1.
Gym gate evidence: docs/bench/baselines/3.0.2.json.
What changed
- Compiled evidence now retains validated UTC
source_timeprovenance for
temporal interval reasoning. Missing, invalid, and weekday-mismatched source
dates fail closed. - Hard context budgets preserve evidence text before optional timestamps.
- Selective context compilation applies only to standard and temporal routes;
aggregation, preference, ordinal, and empty-compiler cases keep the legacy
full-context fallback. - LongMemEval evaluation preserves historical exact/contains metrics and adds a
conservative canonical metric for surface-form equivalence only. - LongMemEval artifacts now record sample-level routing, latency, context size,
and output size. MK_SPLIT_RECOMMENDATION_ROUTE=1optionally sends explicit past
recommendation recall to the factual route. New recommendations and
meta-reminders stay on the preference route. The default remains disabled.
External benchmark evidence
The paired runs below used the same N=50, seed 42, top_k=15, endpoint,
provider, and gpt-5.6-sol model. They are benchmark-harness evidence, not a
full LongMemEval claim.
- 3.0.1 control: exact 8%, contains 64%, canonical 80%, p50 4,067 ms,
p95 11,069 ms, execution errors 0. - Selective evidence candidate: contains 66%, canonical 80%, paired
canonical loss 0, p50 3,779 ms. Its p95 was 12,579 ms, so this release does
not claim a p95 improvement. - Recommendation split candidate: exactly three explicit past-recall cases
changed route. In the paired runs, those cases showed output length of 996
versus 145 characters (-85.4%) and median latency of 2,961 versus 2,701 ms
(-8.8%), with canonical loss 0. These are observed run differences, not an
isolated causal estimate of routing alone. The whole-run contains change on
an unaffected sample is treated as model/run variance, not a routing
regression.
Reproduction details and artifact names are recorded in
docs/bench/longmemeval-3.0.1.md. Benchmark
result JSON remains local and is not claimed as a public release artifact.
Compatibility and migration
- No destructive migration is required.
- Existing exact/contains evaluator outputs remain available.
- Selective evidence compilation fails open to legacy context when it cannot
produce safe evidence. - Recommendation route splitting is opt-in; existing default behavior is
unchanged.
Validation
- Full test suite: 2,031 passed, 3 skipped, 0 failed.
- Memory Gym: 11/11 scenarios passed; see the committed baseline above.
twine check: passed for wheel and sdist.- Fresh isolated wheel install: CLI version/init/doctor, entity write/search,
search_v2content retrieval, and evidence/numeric API checks passed. - Artifact SHA-256:
- wheel:
a66c3408a55376511bc4e4c5caa2217ad34e383c9577a41a33241b6b128357c2 - sdist:
b3521ab6ff305f757bcfe2c51e1300842478a78cf7e1ed3ee7af9dd46c1b165d
- wheel:
- Hermes provider smoke is performed against the published PyPI artifact after
upload; the result is reported in the GitHub release.