Releases: hyper-light/vorpal
Releases · hyper-light/vorpal
Release list
v0.8.3
Full Changelog: v0.8.2...v0.8.3
v0.8.2
v0.8.1
v0.8.0
v0.7.1
v0.7.1 — definitions the parser lost are back in the index
Index coverage
- Parser swallow recovery. tree-sitter's C parser sometimes fails to close a function after an error in its body (bare statement-position macros, kernel idioms) and keeps that node open to end-of-file: the rest of the file parses fine but one level down, where the outline never looked, so every later definition was silently missing. vorpal now diagnoses that shape structurally (errors inside + span reaching the tree's true end + a nested function found by a pruned walk) and lifts the recovered definitions back as top-level items, spans intact — no grammar edit, byte-identical output on every unaffected file (verified across all 75,954 kernel products). Linux
net/core/dev.c+431 definitions (netif_receive_skbresolves),kernel/time/hrtimer.c+59, CPythonObjects/object.c+100 (PyObject_GetAttrnow has its 16 callers). Kernel 8,890,840 → 8,891,771 nodes. Cold-build cost: none (interleaved A/B). Product format v20 — existing indexes rebuild on first use. vorpal index --parse-health/healthno longer hide damaged files behind a pack-root path mismatch (CPython reported "clean" with 591 damaged files).
Search quality, measured honestly
- Label sets expanded 8 / 6 / 10 → 54 / 54 / 55 evidence-backed graded queries per corpus (
xtask/labels/*.evidence.mdcites the source line behind every grade). The README's quality table is restamped to them; the tier ordering holds — the learned tier beats lexical on every corpus, the encoder helps CPython and this repo and lowers the kernel's identifier class — but the old headline numbers had been flattered by single answers, and the table now says so. - Two-field dense-sidecar surfaces: the doc-side sidecar embeds name + doc comment + body head while the rerank keeps the name surface, at unchanged query latency. Paraphrase queries score above zero for the first time (targets in the dense top-100: 3 → 11 of 28); the remaining ceiling is single-list evidence in the fusion, recorded as the next lever.
- Size-aware fusion study: eight derived fusion laws for the dense list measured across kernel coverage checkpoints; the "coverage slide" seen on the 8-query set does not exist on 54 queries, and no law beats the shipped fusion without regressing elsewhere.
Flatstays; the seam ships for future sweeps.
Full Changelog: v0.7.0...v0.7.1
v0.7.0
v0.7.0 — the encoder becomes a candidate generator, on every platform
Search quality
- Always-on doc-side dense channel. Every index with the encoder enabled now embeds its referenced definitions at warm time — in the background, resumable across warms, never blocking a search — and fuses them as a fifth ranked list. The encoder stops being a reranker that can only reorder what the lexical channels found: Linux kernel encoder-enabled search 0.222 → 0.31–0.35 NDCG@10 (0.313 without the encoder), this repository 0.585 → 0.705. The fill's stop rule is structural (definitions something references; measured against the "or exported" variant and pinned on the data).
--dense-budget-timeout 5m30scaps a fill round;<index>/dense.channel = offopts out. - Learned tier trains on a balanced kind set (every definition kind capped at the largest callable population, derived from the corpus): kernel learned tier 0.295 → 0.313, the first time it beats the lexical tier on the 8.9 M-node graph; CPython unchanged. Existing learned tiers retrain automatically (the policy label rides in the tier record).
vorpal tunemeasures the dense channel too and can pin a per-index verdict.
Encoder throughput, everywhere
- macOS: the doc-side path runs through Accelerate
sgemm— 232 → 935–1,928 GFLOPS (AMX-bound, proven by shard sweep), embeddings identical to the reference lanes, bit-identical across thread counts. - x86-64 (Linux / Windows): owned AVX-512F and AVX2+FMA kernels, CPUID-detected — ~45× the previous portable path on CI's AVX2 runner (2 → 107–113 GFLOPS, bit-equal). AVX-512 compiles clean and awaits hardware for its first measurement.
- GPUs (NVIDIA, AMD, Intel iGPU, Apple): one
wgpucompute-shader GEMM over Metal / Vulkan / DX12 — no vendor SDK, ships inside the binary — with the weights resident on the device and the MLP block fused there; a GPU → CPU → reference ladder with typed fallback. On an M5 Max the fill runs 1.83× faster than Accelerate at 3.4× less CPU. Vulkan/DX12 are compile-verified and parity-tested on a software Vulkan device; hardware runs pending. - The query-side rerank stays on the fixed-order reference lanes on every platform — rankings are byte-identical to 0.6.1 wherever the dense channel is off.
- int8 kernels (NEON
sdot, AVX-VNNI, AVX-512-VNNI) ship but stay off: per-row activation quantization misses the retention bar on this encoder (recorded).
Also
- Walk reuse for incremental saves of giant files (0.6.1's headline) is unchanged.
- MSRV is now honestly 1.98 (the tree already relied on 1.88 features); three newer style lints unlocked by the bump are allowed at the workspace level pending hygiene work.
docs/wip/ENCODER_RESEARCH.md: the primary-source survey behind this release, including the measured-and-rejected options (LateOn-Code multi-vector channel, cross-encoder reranking, static distillation, richer candidate surfaces).
Full Changelog: v0.6.1...v0.7.0