Post-rename cleanup from doc audit (4 fixes, 10 files, ±21) by willgitdata · Pull Request #1 · willgitdata/augur

willgitdata · 2026-05-07T20:54:07Z

Summary

Walked the docs end-to-end as a fresh user (only the README + EXAMPLES + API_REFERENCE + DEVELOPMENT_GUIDE) and found four issues. This PR fixes all four. No semantic / behavioral changes — only doc accuracy and brand consistency.

Changes

pnpm test was broken on Node ≤22.5 (packages/core/package.json). The test script used node --experimental-strip-types, which was introduced in 22.6, but engines says ≥20. The flag is redundant when --import tsx is present, so drop it. CI passed either way because the workflow uses explicit args, but the documented pnpm test command now works on the supported Node range.
EXAMPLES.md §1 doc bug (EXAMPLES.md). Comment claimed trace.decision.strategy → "hybrid" for query "vector database", but two-token queries correctly route to keyword. The router is right; the example query was wrong. Replaced with "best vector database for production" (6 tokens, routes hybrid live).
docker-compose.yml referenced the pre-rebrand dashboard package name (docker-compose.yml#L32) — pnpm --filter dashboard dev no longer matches anything. Updated to @augur/dashboard.
Brand consistency: QUERYBRAIN_* env vars survived the rebrand. The earlier sed sweep matched QueryBrain (PascalCase) and querybrain (lowercase) but missed QUERYBRAIN_API_KEY, QUERYBRAIN_ADAPTER, QUERYBRAIN_EMBEDDER, QUERYBRAIN_URL across:
- packages/server/src/cli.ts (3 vars)
- apps/dashboard/next.config.mjs, layout.tsx, page.tsx, traces/page.tsx (AUGUR_URL)
- docker-compose.yml (3 vars)
- API_REFERENCE.md, DEVELOPMENT_GUIDE.md (docs)
All defaults are unchanged, so the in-memory / hash-embedder happy path was working both before and after — but anyone setting an API key or pointing at a real adapter would have been confused by the brand-vs-env-var mismatch.

Routing impact

None. No code in packages/core/src/routing/ is touched. Doc fix #2 changes the example query to one whose routing matches the existing comment.

Testing

pnpm -r --filter './packages/*' build clean
pnpm typecheck clean (already passing in CI)
pnpm test — now passes (was broken pre-PR): 12/12 router + augur tests green
Live verification: "best vector database for production" returns decision.strategy === "hybrid" from the running server with the seeded corpus
Live verification: a server started with AUGUR_API_KEY=test-key returns 401 without the header and 200 with x-api-key: test-key — exactly as the renamed docs say

Notes for reviewer

Diff is 21+/21− across 10 files. Most lines are single-token env-var renames.
I deliberately did not run pnpm format — that produces a 43-file pure-formatting commit (Prettier vs. the repo's existing style), which felt orthogonal to this audit. Worth doing as its own follow-up commit if you want a clean baseline.
I also did not rename the qb variable in README/example code (legacy abbreviation for "QueryBrain"). It still works; rename it if you want full brand consistency in code samples.

🤖 Generated with Claude Code

Found four issues during a fresh-user docs walkthrough: 1. `pnpm test` failed on Node 20 / 22.0–22.5 because the test script used `--experimental-strip-types` (introduced in 22.6). The flag is redundant when tsx is already loaded via `--import tsx`, so drop it. CI passes either way (it bypasses the script with explicit args), but the documented `pnpm test` now works on the supported Node range. 2. EXAMPLES.md §1 "Hello Augur" claimed `trace.decision.strategy → "hybrid"` for query `"vector database"`, but two-token queries correctly route to keyword. Bumping the example to a 6-token natural query that does route to hybrid as advertised. 3. docker-compose.yml referenced `pnpm --filter dashboard dev` after the rebrand renamed the package to `@augur/dashboard`. Fixed. 4. Brand consistency: my earlier rename swept `QueryBrain` and `querybrain` but missed UPPERCASE env vars `QUERYBRAIN_API_KEY`, `QUERYBRAIN_ADAPTER`, `QUERYBRAIN_EMBEDDER`, `QUERYBRAIN_URL` across server/cli.ts, the dashboard, docker-compose, API_REFERENCE.md, and DEVELOPMENT_GUIDE.md. Renamed to AUGUR_*. Defaults still apply, so old deployments work until they upgrade. Verified: - pnpm -r --filter './packages/*' build clean - pnpm test now passes (12/12) - "best vector database for production" routes hybrid live - AUGUR_API_KEY enforces auth: 401 without, 200 with correct key Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

Per follow-up: prefer `const augr = new Augur()` over `const qb = ...` (which was a leftover abbreviation from QueryBrain). Sed `\bqb\b` → `augr` across: - README.md, EXAMPLES.md, API_REFERENCE.md, packages/core/README.md - packages/core/src/augur.ts (doctring), augur.test.ts - examples/basic-search/index.ts, custom-adapter/index.ts - packages/server/src/server.ts (internal var) - DEVELOPMENT_GUIDE.md (just the one variable use, line 125) Intentionally NOT changed: - DEVELOPMENT_GUIDE.md DB name `postgres://localhost/qb` (different category) - DEVELOPMENT_GUIDE.md reverse-proxy paths `/api/qb/*` and `/qb/*` (full word `augur` reads better there; punt as a separate cleanup) Verified: - pnpm -r build clean - pnpm test passes 12/12 - example-basic-search produces same output Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

willgitdata · 2026-05-07T21:13:15Z

 import { TraceView } from "@/components/TraceView";

-const QB = process.env.QUERYBRAIN_URL ?? "http://localhost:3001";
+const QB = process.env.AUGUR_URL ?? "http://localhost:3001";


you forgot to change the "QB" variable. Fix this for augur's setup and abbreviations.

Fixed in 9475fad — renamed QB → AUGR in this file. (Earlier sed was case-sensitive lowercase, missed the SCREAMING_CASE const.)

willgitdata · 2026-05-07T21:13:32Z

 import { ResultList } from "@/components/ResultList";

-const QB = process.env.QUERYBRAIN_URL ?? "http://localhost:3001";
+const QB = process.env.AUGUR_URL ?? "http://localhost:3001";


forgot this QB as well.

Fixed in 9475fad — renamed QB → AUGR in this file. (Earlier sed was case-sensitive lowercase, missed the SCREAMING_CASE const.)

Per PR review feedback. Earlier sed used \bqb\b (lowercase) and missed the SCREAMING_CASE QB const used as a base-URL holder in the two client components. Renaming to AUGR to mirror the new augr variable convention while keeping the const casing. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

…ranker Implements 10 fixes from a critical code review on the publish-ready SDK. Each item is independently shippable; this lands them as one coordinated bump because several feed each other (the eval smoke test exercises the new fusion module; the explicit-reranker breaking change needs MIGRATING.md to land at the same time). #1 Smoke eval harness — 16-doc / 12-query synthetic fixture with a regression floor (NDCG@10 > 0.65 on the stub stack). Runs in <50 ms as part of `pnpm test`. The full BEIR + 504-query eval stays where it already lives — git history — for "did this tweak win?" measurement. #2 Extract pure helpers to `fusion.ts` (composeFilter, pickVectorWeight, weightedRrfFuse, adaptWeightByConfidence, topGapNormalized, clamp) + 31 unit tests. Each empirical threshold is now annotated with what it was tuned against. #3 autoLanguageFilter integration tests — OFF default, ON for non-English, user-filter override wins, soft-fallback when the filtered pool empties. #4 Basic-search example wires the recommended stack: LocalEmbedder + LocalReranker + MetadataChunker(SentenceChunker) + InMemoryAdapter({ useStemming: true }). Matches the README's headline configuration so users copying from "hello world" land on the auto path that produces NDCG@10 = 0.920. #5 PineconeAdapter mocked-fetch tests (13 tests). Pins the wire format (URL, method, auth header, body shape, response decode) so refactors can't silently regress one of the three production adapters. Previously had zero coverage. #6 Ad-hoc scratch adapter cache — bounded LRU keyed by a deterministic fingerprint of (id, content). Repeat searches over the same `req.documents` skip re-chunking + re-embedding. Tunable via `adHocCacheSize` (default 8; set 0 to disable). #7 **BREAKING** Drop `HeuristicReranker` as silent default. The previous default did almost nothing while emitting a "yes I rerank" line in the trace. Default is now `null`; pass `new LocalReranker()` (or any provider's reranker) explicitly to keep cross-encoder voting on. Documented in MIGRATING.md. #8 MIGRATING.md — covers every BREAKING change in 0.2 with smallest-diff examples; non-breaking adoptions documented separately. #9 SemanticChunker tests (8 tests) — boundary detection, maxSize cap, async-only API, metadata propagation. Was the only chunker without coverage. #10 Magic-number provenance documented in `router.ts` and `fusion.ts`. Every threshold (≤2 / ≤6 word counts, 0.6 ambiguity floor, 800 ms latency floor, k=60 RRF, ±0.20 shift, 0.10–0.90 weight clamp, 0.3/0.4/0.5/0.7 priors) now records what it's tuned against and what's load-bearing vs negotiable. Test count: 100 → 163. Build green; full eval still ships against the published packages, not against the smoke fixture. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

Round 1 left two critical lying-trace bugs and a handful of architectural papercuts. This commit clears the table. #1 (critical) Router no longer lies about reranking when no reranker is configured. `Router.decide` gained an optional `hasReranker` parameter; `HeuristicRouter` plumbs it through to `shouldRerank` and forces `reranked: false` when absent. The trace now records "reranking skipped (no reranker configured)" instead of pretending the cross-encoder fired. Augur passes `this.reranker !== null`. Default is `true` so existing third-party Router implementations keep working. #2 (critical) `SearchTrace` declares the four fields that augur.ts attaches at runtime: `adHoc`, `adHocCacheHit`, `autoLanguageFilter`, `autoLanguageFilterDropped`. `Tracer.finish` opts widened to `Omit<SearchTrace, "id"|"query"|"startedAt"|"totalMs"|"spans">` so adding a SearchTrace field propagates automatically. Tests drop their `as unknown as` casts. #3 (high) PgVectorAdapter mocked-fetch tests (14 tests). Pin SQL shape, INSERT batching at 200/round-trip, parameter renumbering across filter clauses, identifier-validation guard against `; DROP TABLE`, **and a filter-key SQL-injection regression test** — the JSON-path quote-doubling defense gets explicit coverage. #4 (high) Adapter trace-string format change reverted. `trace.adapter` is always the bare adapter name; ad-hoc / cache-hit signals surface as the new structured boolean fields from #2. No more "in-memory (ad-hoc, cached)" string parsing. #5 (high) `fingerprintDocs` extracted to `fingerprint.ts` with 10 direct unit tests covering reorder, byte-change, prefix-equal corpora, id|content boundary, doc-record boundary, empty list, unicode, and the output format contract. #6 (medium) Async chunkers no longer pretend to be `Chunker`s. Introduced an explicit `AsyncChunker` interface; `SemanticChunker`, `Doc2QueryChunker`, `ContextualChunker` implement it (no longer Chunker). The runtime traps in their throwing `chunk()` methods are gone — the type system catches misuse at compile time. APIs that accept either flavor (`AugurOptions.chunker`, all chunker `base` fields, `chunkDocument`) now use `Chunker | AsyncChunker`. `MetadataChunker` keeps its dual sync+async path with a runtime guard for the user-opted-in case where its base is async. #7 (medium) `StubEmbedder` consolidated into `packages/core/src/test-fixtures.ts`. Excluded from the published package via tsconfig. Three duplicated copies dropped. #8 (low) `eval-smoke.test.ts` header explicitly distinguishes the synthetic-fixture smoke test (structural) from the BEIR / 504-query eval that produced the README's NDCG@10 = 0.920 numbers (preserved at git `feffc73^`, runs in ~30 min). #9 (low) `BaseAdapter` JSDoc rewritten as the canonical "starting point for custom adapters" comment, including the RRF / capability / `searchHybrid` override guidance. `AsyncChunker` added to public exports. #10 (low) `examples/basic-search/index.ts` header documents the `npm i @huggingface/transformers` requirement for users copying the file out of the repo. Test count: 163 → 191 (+28). Build green; published-package contents verified clean (`find dist -name "test-fixtures*"` → empty, `find dist -name "*.test.*"` → empty). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

Post-rename cleanup from doc audit (4 fixes, 10 files, ±21)

…ranker Implements 10 fixes from a critical code review on the publish-ready SDK. Each item is independently shippable; this lands them as one coordinated bump because several feed each other (the eval smoke test exercises the new fusion module; the explicit-reranker breaking change needs MIGRATING.md to land at the same time). #1 Smoke eval harness — 16-doc / 12-query synthetic fixture with a regression floor (NDCG@10 > 0.65 on the stub stack). Runs in <50 ms as part of `pnpm test`. The full BEIR + 504-query eval stays where it already lives — git history — for "did this tweak win?" measurement. #2 Extract pure helpers to `fusion.ts` (composeFilter, pickVectorWeight, weightedRrfFuse, adaptWeightByConfidence, topGapNormalized, clamp) + 31 unit tests. Each empirical threshold is now annotated with what it was tuned against. #3 autoLanguageFilter integration tests — OFF default, ON for non-English, user-filter override wins, soft-fallback when the filtered pool empties. #4 Basic-search example wires the recommended stack: LocalEmbedder + LocalReranker + MetadataChunker(SentenceChunker) + InMemoryAdapter({ useStemming: true }). Matches the README's headline configuration so users copying from "hello world" land on the auto path that produces NDCG@10 = 0.920. #5 PineconeAdapter mocked-fetch tests (13 tests). Pins the wire format (URL, method, auth header, body shape, response decode) so refactors can't silently regress one of the three production adapters. Previously had zero coverage. #6 Ad-hoc scratch adapter cache — bounded LRU keyed by a deterministic fingerprint of (id, content). Repeat searches over the same `req.documents` skip re-chunking + re-embedding. Tunable via `adHocCacheSize` (default 8; set 0 to disable). #7 **BREAKING** Drop `HeuristicReranker` as silent default. The previous default did almost nothing while emitting a "yes I rerank" line in the trace. Default is now `null`; pass `new LocalReranker()` (or any provider's reranker) explicitly to keep cross-encoder voting on. Documented in MIGRATING.md. #8 MIGRATING.md — covers every BREAKING change in 0.2 with smallest-diff examples; non-breaking adoptions documented separately. #9 SemanticChunker tests (8 tests) — boundary detection, maxSize cap, async-only API, metadata propagation. Was the only chunker without coverage. #10 Magic-number provenance documented in `router.ts` and `fusion.ts`. Every threshold (≤2 / ≤6 word counts, 0.6 ambiguity floor, 800 ms latency floor, k=60 RRF, ±0.20 shift, 0.10–0.90 weight clamp, 0.3/0.4/0.5/0.7 priors) now records what it's tuned against and what's load-bearing vs negotiable. Test count: 100 → 163. Build green; full eval still ships against the published packages, not against the smoke fixture. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

Round 1 left two critical lying-trace bugs and a handful of architectural papercuts. This commit clears the table. #1 (critical) Router no longer lies about reranking when no reranker is configured. `Router.decide` gained an optional `hasReranker` parameter; `HeuristicRouter` plumbs it through to `shouldRerank` and forces `reranked: false` when absent. The trace now records "reranking skipped (no reranker configured)" instead of pretending the cross-encoder fired. Augur passes `this.reranker !== null`. Default is `true` so existing third-party Router implementations keep working. #2 (critical) `SearchTrace` declares the four fields that augur.ts attaches at runtime: `adHoc`, `adHocCacheHit`, `autoLanguageFilter`, `autoLanguageFilterDropped`. `Tracer.finish` opts widened to `Omit<SearchTrace, "id"|"query"|"startedAt"|"totalMs"|"spans">` so adding a SearchTrace field propagates automatically. Tests drop their `as unknown as` casts. #3 (high) PgVectorAdapter mocked-fetch tests (14 tests). Pin SQL shape, INSERT batching at 200/round-trip, parameter renumbering across filter clauses, identifier-validation guard against `; DROP TABLE`, **and a filter-key SQL-injection regression test** — the JSON-path quote-doubling defense gets explicit coverage. #4 (high) Adapter trace-string format change reverted. `trace.adapter` is always the bare adapter name; ad-hoc / cache-hit signals surface as the new structured boolean fields from #2. No more "in-memory (ad-hoc, cached)" string parsing. #5 (high) `fingerprintDocs` extracted to `fingerprint.ts` with 10 direct unit tests covering reorder, byte-change, prefix-equal corpora, id|content boundary, doc-record boundary, empty list, unicode, and the output format contract. #6 (medium) Async chunkers no longer pretend to be `Chunker`s. Introduced an explicit `AsyncChunker` interface; `SemanticChunker`, `Doc2QueryChunker`, `ContextualChunker` implement it (no longer Chunker). The runtime traps in their throwing `chunk()` methods are gone — the type system catches misuse at compile time. APIs that accept either flavor (`AugurOptions.chunker`, all chunker `base` fields, `chunkDocument`) now use `Chunker | AsyncChunker`. `MetadataChunker` keeps its dual sync+async path with a runtime guard for the user-opted-in case where its base is async. #7 (medium) `StubEmbedder` consolidated into `packages/core/src/test-fixtures.ts`. Excluded from the published package via tsconfig. Three duplicated copies dropped. #8 (low) `eval-smoke.test.ts` header explicitly distinguishes the synthetic-fixture smoke test (structural) from the BEIR / 504-query eval that produced the README's NDCG@10 = 0.920 numbers (preserved at git `4d52844^`, runs in ~30 min). #9 (low) `BaseAdapter` JSDoc rewritten as the canonical "starting point for custom adapters" comment, including the RRF / capability / `searchHybrid` override guidance. `AsyncChunker` added to public exports. #10 (low) `examples/basic-search/index.ts` header documents the `npm i @huggingface/transformers` requirement for users copying the file out of the repo. Test count: 163 → 191 (+28). Build green; published-package contents verified clean (`find dist -name "test-fixtures*"` → empty, `find dist -name "*.test.*"` → empty). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

willgitdata and others added 2 commits May 7, 2026 13:53

willgitdata commented May 7, 2026

View reviewed changes

willgitdata merged commit 62f885c into main May 7, 2026
1 check passed

willgitdata mentioned this pull request May 7, 2026

Eval harness, richer query signals, two new rerankers #2

Merged

5 tasks

willgitdata deleted the chore/post-rename-cleanup branch May 8, 2026 01:15

willgitdata added a commit that referenced this pull request May 11, 2026

Merge pull request #1 from willgitdata/chore/post-rename-cleanup

dc2ef4b

Post-rename cleanup from doc audit (4 fixes, 10 files, ±21)

willgitdata mentioned this pull request May 12, 2026

fix(server): safe-by-default HTTP boundary + tests (CRITICAL) #28

Merged

6 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Post-rename cleanup from doc audit (4 fixes, 10 files, ±21)#1

Post-rename cleanup from doc audit (4 fixes, 10 files, ±21)#1
willgitdata merged 3 commits into
mainfrom
chore/post-rename-cleanup

willgitdata commented May 7, 2026

Uh oh!

willgitdata May 7, 2026

Uh oh!

willgitdata May 7, 2026

Uh oh!

willgitdata May 7, 2026

Uh oh!

willgitdata May 7, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

willgitdata commented May 7, 2026

Summary

Changes

Routing impact

Testing

Notes for reviewer

Uh oh!

willgitdata May 7, 2026

Choose a reason for hiding this comment

Uh oh!

willgitdata May 7, 2026

Choose a reason for hiding this comment

Uh oh!

willgitdata May 7, 2026

Choose a reason for hiding this comment

Uh oh!

willgitdata May 7, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant