promote 0.21.0 to prod (P1.3.3 challenge canon parity)

[klappy]

Promotes oddkit 0.21.0 from main to prod.

What ships

• oddkit_challenge D5 stemmed prereq matcher (Item 1, PR feat(challenge): D5 stemmed prereq matcher + D9 cache removal (0.21.0) #120)
• oddkit_challenge D9 cache removal (Item 2, PR feat(challenge): D5 stemmed prereq matcher + D9 cache removal (0.21.0) #120)
• New canon principle klappy://canon/principles/cache-fetches-and-parses (klappy.dev#125, already live)

Attestation

Stage	Result
Preview branch smoke @ challenge-d5-d9-parity-oddkit	170/170 × 3 consecutive (after 1 cold-start flake)
CI Test CF Preview on PR #120	success
Main-preview smoke @ main-oddkit	170/170 × 3 consecutive

Same Option A smoke-heavy attestation pattern as P1.3.2 (no Sonnet 4.6 validator dispatch this round — scope is small, no envelope changes, no new governance files).

Sweep status after promotion

Tool	Version	Matcher	BM25 cached?
oddkit_encode	0.18.0	regex (P8 follow-up)	parse product cached
oddkit_challenge	0.21.0	stemmed set intersection + structural	rebuild inline
oddkit_gate	0.20.0	BM25 + stemmed set intersection	rebuild inline

Challenge has full canon parity with gate's matcher and caching discipline.

Refs

• Feature PR: feat(challenge): D5 stemmed prereq matcher + D9 cache removal (0.21.0) #120 (merged 33ca5bf)
• Canon PR: klappy.dev#125 (merged 3726073)
• Handoff: klappy://odd/handoffs/2026-04-20-p1-3-3-challenge-revisit

---

Note

Medium Risk
Changes oddkit_challenge prerequisite evaluation and type detection internals; although intended to be strictly additive, it alters text-matching behavior and could affect which gaps are surfaced in edge cases.

Overview
Bumps oddkit to 0.21.0 and documents the release in CHANGELOG.md.

Updates oddkit_challenge prerequisite checking to use parse-time stemmed token sets + runtime set intersection (with URL/numeric/proper-noun/citation structural tests and the non-trivial-input fallback preserved), and removes the module-level BM25 type-detection index cache in favor of rebuilding the index inline per request.

Adds a shared PrereqMatchVocab shape used by both base prerequisites and per-type prerequisite overlays, and expands the live smoke test to assert the new stemmed/structural prereq behavior and BM25 rebuild stability.

^{Reviewed by Cursor Bugbot for commit 33ca5bf. Bugbot is set up for automated code reviews on this repo. Configure here.}

[cloudflare-workers-and-pages]

Deploying with    Cloudflare Workers

The latest updates on your project. Learn more about integrating Git with Workers.

Status	Name	Latest Commit	Preview URL	Updated (UTC)
✅ Deployment successful! View logs	oddkit	33ca5bf	Commit Preview URL Branch Preview URL	Apr 20 2026, 04:16 AM

[cursor]

Cursor Bugbot has reviewed your changes and found 1 potential issue.

Bugbot Autofix prepared a fix for the issue found in the latest run.

• ✅ Fixed: Stop-word keywords silently dropped breaking additive invariant
  • Disabled stop-word filtering in both parseCheckColumn (vocabulary side) and inputStems (input side) by passing an empty Set to tokenize(), so canon keywords like "from" that happen to be stop words survive on both sides and participate in stemmed set intersection, restoring the strictly-additive invariant vs. the prior regex evaluator.

Preview (33ca5bfc98)

diff --git a/workers/src/orchestrate.ts b/workers/src/orchestrate.ts
--- a/workers/src/orchestrate.ts
+++ b/workers/src/orchestrate.ts
@@ -2145,7 +2145,10 @@
   // the loop, stemmedTokens differ per prereq. Per PRD D3 (P1.3.3): stemmed
   // set intersection at runtime, structural tests preserved, no regex compile
   // per check. This is the fit-to-problem matcher per D5.
-  const inputStems = new Set(tokenize(input));
+  // Disable stop-word filtering to preserve canon keywords that happen to be
+  // stop words (e.g. `from` in source-named vocab). Must match the tokenize()
+  // call in parseCheckColumn so vocab and input stems share the same shape.
+  const inputStems = new Set(tokenize(input, new Set()));
   const missing: string[] = [];
   for (const p of prereqMap.values()) {
     const passed = evaluatePrerequisiteCheck(inputStems, input, p);
@@ -2323,10 +2326,16 @@
   let m: RegExpExecArray | null;
   while ((m = quotedRegex.exec(check)) !== null) {
     // Tokenize each quoted keyword or phrase — multi-word phrases like
-    // "according to" contribute multiple stems; stop-words are dropped
-    // by tokenize(). This preserves semantic coverage while normalizing
-    // morphology (problems → problem, considered → consid, etc.).
-    for (const stem of tokenize(m[1])) {
+    // "according to" contribute multiple stems. Disable stop-word filtering
+    // (empty Set) because canon vocab includes stop-word keywords — e.g.
+    // `from` in the source-named prereq, `by` / `as` / `with` elsewhere —
+    // and dropping them here would silently remove them from the matcher,
+    // breaking the strictly-additive invariant vs. the prior regex evaluator
+    // which tested each quoted keyword as a raw \b...\b regex against input.
+    // The input side (inputStems) uses the same empty stop-word set so the
+    // two sides share shape. Stemming still applies: morphology is normalized
+    // (problems → problem, considered → consid, etc.).
+    for (const stem of tokenize(m[1], new Set())) {
       stemmedTokens.add(stem);
     }
   }

_{You can send follow-ups to the cloud agent here.}

^{Reviewed by Cursor Bugbot for commit 33ca5bf. Configure here.}

[cursor]

Stop-word keywords silently dropped breaking additive invariant

Medium Severity

parseCheckColumn calls tokenize(m[1]) on each quoted keyword, which applies the default STOP_WORDS filter. Quoted canon keywords that happen to be stop words — like "from" (explicitly listed in the comment at line 2351 as part of the source-named vocab) — are silently dropped from stemmedTokens. The same filtering on tokenize(input) at the call site removes them from inputStems too. The old regex evaluator matched "from" via new RegExp("\\bfrom\\b", "i") against the raw input, so an input like "I got this from my colleague" would pass source-named before but fails now. This breaks the "strictly additive" invariant claimed by the PR and CHANGELOG.

Additional Locations (1)

• workers/src/orchestrate.ts#L2147-L2148

 

^{Reviewed by Cursor Bugbot for commit 33ca5bf. Configure here.}