v0.6.0

Changed

• Raised MAX_SEQ_LENGTH from 256 to 512 tokens and DEFAULT_CHUNK_SIZE from 1000 to 1500 chars (DEFAULT_MIN_CHUNK_SIZE 250 → 350, DEFAULT_CHUNK_OVERLAP 150 → 200). The previous pair caused silent embed-time truncation on dense chunks: char-1000 chunks averaged 250–330 tokens for code, so the long tail of the distribution had its right edge dropped before pooling. Raising both together closes the truncation gap and lets each chunk carry more cross-cutting context. Measured on 30 ground-truth cases across webpack + prometheus + home-assistant-core: aggregate miss rate 6.7% → 3.3%, mean recall@10 0.89 → 0.93, recall@5 0.86 → 0.89; webpack alone went miss% 20% → 0% and r@10 0.60 → 0.83. Existing indexes continue to work but get the truncation benefit only after a full re-index (codesage index --full); incremental hooks pick it up file-by-file as edits land. Indexing wall-clock rises moderately (per-file token count drops, per-token compute rises with longer sequences). Full A/B at bench/history/cap512-1500-2026-05-04.md. Validates the "raise the cap, not shrink the chunk" thesis from the prior null-result attempt at this same fix (token-200 chunking, reverted).
• discover_files_with_excludes now uses WalkParallel with worker-local SHA-256 hashing and an mpsc drain. The previous serial walk hashed every file in series and was the dominant indexer bottleneck on monorepos. Measured on a 2867-file Laravel checkout (no semantic): wall time ≈ 1.5 s with ~6.5× core utilization (9.8 s user / 1.5 s wall), versus the prior serial walk's near-1× ratio. Behavior is preserved: errors still bubble (first error wins via a shared slot, walk quits early), .h files still get C/C++ project-aware classification, and the result is sorted by path so callers that depend on stable ordering keep working.

Added

• Symbol.rationale[] field on find_symbol and export_context MCP responses, populated for Rust definitions whose attached doc-comments start with WHY:, NOTE:, IMPORTANT:, FIXME:, HACK:, XXX:, or TODO:. Each entry carries the marker kind, the comment body with the marker stripped, and the comment's source-line span. The field is omitted from JSON when empty, so existing consumers see no diff for symbols without rationale comments. Marker-based filtering keeps the table small and high-signal: plain "describes what this function does" doc-comments are not stored — agents recover those from the symbol name and signature anyway. Two limitations the field's doc-comment also calls out: rationale rots when code changes (we extract whatever the source says at index time, can't verify it's still accurate); and rationale is a property of definitions only, not callsites or references. Currently extracted for Rust; other languages return empty rationale[] until per-language extractors land. Schema migration 0007_symbols_rationale adds a rationale column to the symbols table; existing indexes pick up real values on the next file-touch + reindex (no global re-index required).
• [embedding].pooling config field ("mean" or "cls") overrides the model-name heuristic that previously decided pooling strategy from a substring match on bge-. The heuristic is silent and wrong for any non-bge- CLS model (intfloat/e5-*, etc.) and any bge- mean-pooling variant — both produce semantically wrong vectors with no error message. Existing configs keep working: when the field is omitted, the heuristic still applies. Set explicitly when picking a model the heuristic doesn't know about.

Changed

• Tightened protocol types: SearchResult.language is now a Language enum (was a free-form String) and SymbolSummary.kind is a SymbolKind enum (was String). JSON serialization is unchanged (both serialize as the same lowercase strings), but Rust consumers that build SearchResult / SymbolSummary literals or match on the fields directly will need to swap string literals for the enum values. Closes the silent-typo class of bug where a hand-written "functon" would compile.
• Added PartialEq / Eq derives to Symbol, Reference, and FileInfo, and Hash to FileCategory. Lets consumers dedup, assert against, and key hash maps on these types — previously blocked by missing trait derives.
• codesage git-index now passes --since=@<2-years-ago> to git log, capping the history walk to the window where decay weight stays meaningfully above zero. With τ=180d half-life, commits older than 2 years contribute ≤ 0.017 per commit (≤ 0.0025 by 3 years, ~zero by 5 years), so a longer walk just inflates first-onboard time without moving churn or coupling scores. Trims walk by ~7× on long-history projects (php-src: 146k → ~10k commits, ~90s → ~10s on a fresh --full). Slow-coupling pairs that co-change once every 8-12 months still accumulate count >= 3 within the window. Constant HISTORY_WINDOW_DAYS = 730 in crates/graph/src/git_history/indexer.rs.
• codesage install-hooks now wraps the background indexer invocations in nice -n 19 and (Linux-only, gated on command -v ionice) ionice -c 3 so a hook-triggered indexer can't soak foreground CPU or block disk I/O. Existing installs need to re-run codesage install-hooks to pick up the niced template; new installs get it automatically.

Fixed

• Symbol and reference column numbers (col_start, col_end, col) are now UTF-8 character columns instead of tree-sitter's default byte columns. Tree-sitter's Point.column is bytes from the start of the line, so a single CJK ideograph or emoji ahead of a node would shift the byte column past where any editor renders the cursor; goto-definition and editor-jump landed in the wrong place on multi-byte source. The 0.26 Rust binding doesn't expose ts_node_*_point_utf8, so the conversion (walk back to line start, count UTF-8 codepoints in the prefix) is implemented in crates/parser/src/position.rs and used by both extract.rs (symbols) and references.rs (references). ASCII columns are unchanged. Covered by unit tests for ASCII, CJK, emoji, line-reset, and invalid-UTF-8 fallback cases.
• Bounded the Python site-packages probe used for ONNX Runtime / NVIDIA library discovery with a 10 s timeout. Without it, a wedged Python interpreter inside OnceLock::get_or_init would block every thread waiting on Embedder::new / Reranker::new indefinitely. On timeout the child is killed and the probe falls back to the next interpreter or to system paths.
• Moved ONNX Runtime + NVIDIA library discovery into fn main() (codesage_embed::model::init_for_main) before any thread or tokio runtime spawns. The two underlying std::env::set_var calls (LD_LIBRARY_PATH, ORT_DYLIB_PATH) are unsafe under Rust 2024 because concurrent getenv from another thread is UB; pinning them to single-threaded startup eliminates the race. The original Once::call_once-guarded calls inside Embedder::new / Reranker::new remain as a defensive fallback for direct library users.
• Hardened the indexer + MCP output paths against three latent OOM / silent-corruption failure modes. (1) discover_files_with_excludes now skips files larger than 10MB (MAX_INDEXABLE_FILE_BYTES) at metadata time, before fs::read, so a stray large generated SQL dump or vendored data file slipping past HARD_EXCLUDE_PATTERNS no longer fills the indexer's heap. Skipped files emit a tracing::warn. (2) Embedder::detect_dim now bails with a clear error when a model's output shape has no static last-dimension, instead of silently falling back to the 384-dim default — a model with a different real dimension would otherwise store wrong-dimension vectors and turn search_knn into noise. (3) truncate_array in the MCP layer now shrinks an oversized first item's content field instead of returning it unchanged: a single 50KB chunk no longer blows past the 32KB token budget.
• Tightened SQLite transaction boundaries on three semantic-write paths so a SQLITE_BUSY or I/O failure on COMMIT can no longer poison the long-lived MCP DB connection. (1) Database::execute_batch now issues an explicit ROLLBACK when COMMIT itself fails, instead of leaving the connection inside an open tx where every subsequent statement errors with "cannot start a transaction within a transaction". (2) repair_fts_sidecar now wraps its DELETE + INSERT…SELECT in BEGIN / COMMIT inside the same execute_batch call, so a crash between the two statements no longer leaves the FTS sidecar empty while the chunk table has data. (3) insert_chunks now wraps each call in a SAVEPOINT so the vec0 + FTS5 inserts stay atomic even when the function is called outside an outer execute_batch. None of these were observed live, but the failure modes are real (per-connection corruption, BM25 returning zero results until next process start, orphaned vec0 rows).
• Fixed codesage index losing all in-flight progress when killed mid-pass. The semantic indexer previously wrapped the entire write pass in one SQLite transaction, so a SIGKILL / OOM / terminal close between embedding and final commit rolled back every file's hash, even files whose embeddings had already been computed. select_semantic_files then reselected those files on the next run, so the same backlog was re-embedded indefinitely. Observed on ~/php-src with Jina v2 base: a 572-file hole persisted across many post-merge hook fires (~10 min/run) until one uninterrupted run was allowed to finish. Writes are now committed in batches of 50 files; an aborted run preserves all files committed up to the last batch boundary.
• Fixed codesage status and codesage doctor to report semantic index freshness for the configured model, including stale or missing semantic-file hashes that structural HEAD drift alone cannot detect.
• Fixed export_context / codesage export callee handling so include_callees returns referenced callee/dependency code instead of being ignored for symbol targets or duplicating the definition chunk for query targets.
• Fixed codesage export --symbol --limit to cap returned symbol definitions and primary chunks, preventing common names from exceeding the requested context budget.
• Fixed scripts/leak-check.sh invalid-regex diagnostics so the underlying grep error is shown instead of an empty detail block.
• Fixed impact_analysis target handling and precision: dotted symbols such as Repository.find now stay symbol targets by default, codesage impact --symbol can force symbol interpretation from the CLI, and qualified-symbol impact/export caller lookup now uses exact qualified references instead of broad same-tail matches.
• Fixed PHP structural references so instance ($object->method()), nullsafe ($object?->method()), and static method names are indexed as call references.
• Fixed scripts/leak-check.sh --range A..B to scan file content at the range endpoint instead of the current HEAD.
• Fixed scripts/leak-check.sh to fail closed on invalid forbidden-pattern regexes, and added CI coverage for shell linting plus maintenance-script regressions.
• Fixed scripts/release.sh to refresh the [Unreleased] changelog compare link when cutting a release.
• Fixed structural lookup and context export edge cases: Rust/C++ :: qualified symbols now resolve through find_symbol, dotted call references such as Go fmt.Println can be found by their bare tail, Python attribute calls are indexed as call references, and export_context binds symbol summaries back to exact qualified definitions instead of the first matching short name.
• Fixed git-history indexing to honor project exclude_patterns from .codesage/config.toml and to decay existing churn/co-change weights even when an incremental run finds HEAD unchanged.
• Fixed codesage init TOML generation for project directory names containing quotes, backslashes, or control characters.