Skip to content

feat(competition): challenger kit restoring the auto backend - #565

Merged
github-actions[bot] merged 1 commit into
koth-ladderfrom
kit/restore-auto-backend
Jul 27, 2026
Merged

feat(competition): challenger kit restoring the auto backend#565
github-actions[bot] merged 1 commit into
koth-ladderfrom
kit/restore-auto-backend

Conversation

@plind-junior

Copy link
Copy Markdown
Member

round-1 ladder entry: restore backend: auto over the round-0 canary champion (substring).

local precheck with the gate's own scorer (.github/scripts/koth_score.py) at base sha 7243bd5, utc 2026-07-27: champion mean 0.20, challenger mean 0.55, paired diff +0.35 against a band of max(0.007, 1.96 x SE) = 0.0295 — dethrones on all 12 derived seeds.

reproduce:

python .github/scripts/koth_score.py \
  --champion competition/kits/current/kit.yaml \
  --challenger kit-from-this-pr.yaml \
  --base-sha <base tip> --date <utc date>

kit-only change, as the ladder requires. expected gate behaviour: classify as ladder PR, validate against the allowlist, run paired scoring, post the scorecard comment, verdict dethroned, enable squash auto-merge (base is the KOTH_LADDER_BASE branch).

round-1 ladder entry: the round-0 canary champion pins the substring
scan; this kit restores backend auto (fts5 with rrf fusion when
embeddings exist). measured locally with the gate's own scorer on the
day's derived seeds, the restore clears the dethrone band by a wide
margin. kit-only change, as the ladder requires.
@coderabbitai

coderabbitai Bot commented Jul 27, 2026

Copy link
Copy Markdown

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

🗂️ Base branches to auto review (2)
  • test
  • main

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: ea2b4954-4db6-42ce-9edb-b8a9cbc691e4

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch kit/restore-auto-backend

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@github-actions github-actions Bot added size: XS less than 50 changed non-doc lines ci: passing ci is green labels Jul 27, 2026
@plind-junior plind-junior reopened this Jul 27, 2026
@plind-junior plind-junior reopened this Jul 27, 2026
@github-actions

Copy link
Copy Markdown

koth ladder - dethroned

{
 "date": "2026-07-27",
 "base_sha": "881127250ba3a680615d99bf73ab4376acfdcb69",
 "seeds": [
  90813253984371,
  164084902193554,
  261359132532710,
  255327968308077,
  88737840794468,
  76942358585422,
  142842187877729,
  97581596516071,
  157100008129172,
  46217966697507,
  46815146570875,
  10927681347214
 ],
 "champion": {
  "scores": [
   0.2,
   0.2,
   0.2,
   0.2,
   0.2,
   0.2,
   0.2,
   0.2,
   0.2,
   0.2,
   0.2,
   0.2
  ],
  "mean": 0.2
 },
 "challenger": {
  "scores": [
   0.5,
   0.6,
   0.5,
   0.6,
   0.5,
   0.6,
   0.5,
   0.6,
   0.5,
   0.6,
   0.6,
   0.7
  ],
  "mean": 0.5666666666666667
 },
 "mean_diff": 0.36666666666666664,
 "se": 0.01880253582725887,
 "band": 0.03685297022142739,
 "dethroned": true
}

auto-merge on dethrone (provisional ladder branch).

seeds derive from base sha + utc date; rerun locally with
python .github/scripts/koth_score.py --date <utc-date> to reproduce.
the daily result is provisional and overfittable - payout rank is
settled monthly on sealed commit-reveal seeds (docs/vouchbench-seasons.md).

@github-actions
github-actions Bot merged commit 3f286c9 into koth-ladder Jul 27, 2026
32 of 34 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ci: passing ci is green size: XS less than 50 changed non-doc lines

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant