Skip to content

Kindex F26 benchmark build

Pre-release
Pre-release

Choose a tag to compare

@alexu8007 alexu8007 released this 23 Oct 01:37
· 92 commits to main since this release

This tag marks the exact code behind the F26 benchmark report (docs/benchmarks/kin-ask-f26-report.html) and the Labs post on Kindex.

It is not a PyPI release. Its package metadata still reads 0.48.1. Nothing is published from this tag.

Which version to use

  • Default settings (LoCoMo 95.13%, LongMemEval-S 97.80%, BEAM-1M 80.69%): the default answer path behind these scores shipped unchanged in Kindex 0.49.0. Later releases move it to ask.preset: benchmark (#93). Pass each question's date with --as-of.

  • BEAM with the dataset's annotations:

    Split Score
    100K 91.66
    500K 89.57
    1M 90.61
    10M 84.15

    These runs need this build. They pass each question's annotations through caller hints: answer_question(hints=..., hint_queries=...) with ask.hint_mode: lead_both. That interface was removed before 0.49.0 (91616af).

    • The answer model was gpt-5.6-luna with reasoning off.
    • The shared answer settings and per-category overrides are in #78.