Skip to content

v1.0.8

Choose a tag to compare

@github-actions github-actions released this 30 Sep 17:12
· 32 commits to main since this release
fc7f8b4

[1.0.8] — 2026-10-01

Agent experience — staleness disclosure and MCP tools that teach their use

  • Queries disclose a stale graph — the header now reports how many manifest files changed since the build (# 3 file(s) changed since this build — run astria update before trusting answers), next to the existing # graph built at line. Edits through paths without hooks (Claude Code print-mode sessions — which provably do not run PostToolUse hooks — editors without them, plain typing) previously left agents confidently answering from the past unless they noticed a raw timestamp. The check is stat-only against the file manifest (no re-hashing): milliseconds per query even on large repos, and it clears itself on the next update. Verified live: touch a file → disclosure appears; update → fresh timestamp, silent header.
  • MCP tool descriptions now teach usage — the descriptions agents see when choosing tools: query_graph explains the file:line anchors, the provenance tiers, what "No confident match" means (rephrase toward symbol names — not an error), and that the header discloses staleness; explain documents the -->/<-- real-direction arrows; affected says to run it before changing a shared symbol and how to read RESOLVED vs INFERRED hops. Verified through a live MCP handshake that the served descriptions carry the guidance.

Full ecosystem coverage — MCP for every coding tool, five new platforms, install --all

  • MCP registration everywhere it is supported — joining claude/cursor/gemini/zcode: VS Code (.vscode/mcp.json, native workspace MCP — covers every VS Code-based editor including Copilot inside it), Codex (~/.codex/config.toml, user-global TOML — the installer appends a managed [mcp_servers.astria] table and never touches a hand-written one), Trae, Windsurf, Kiro (.kiro/settings/mcp.json — Kiro's documented workspace-scope config), and OpenCode (.opencode/opencode.json). The Copilot coding agent gets skill + instructions only: it has no committed repo MCP file — repository-level MCP is JSON pasted into the repository Settings on github.com. All JSON flavors are project-scoped, idempotent, and merge-safe like the originals; uninstall removes each, and configs an earlier build wrote at a since-corrected path are migrated away, never duplicated.
  • Verified against the real vendor CLIs and vendor docs, not just schemas: codex mcp list shows astria … enabled (Codex 0.140.0 parsing the appended TOML); gemini mcp list resolves the settings entry; opencode mcp list reports ✓ astria connected — it actually launched the server; claude mcp list reaches the project .mcp.json server (⏸ pending Claude Code's documented one-time project approval). Real-tool testing caught two OpenCode bugs schema-following would have shipped: opencode 1.17+ rejects a plugins key in opencode.json (the pre-existing plugin injector wrote one — plugins now drop into the auto-discovered .opencode/plugins/ — the convention in opencode's current docs, and verified loadable by probing opencode debug config on 1.17.8, which resolves plugins from both the plural and singular directories — with no config key, and installs upgrade away both earlier layouts), and its MCP schema requires servers directly under mcp with {type: "local", command: [...], enabled: true} — not mcp.servers with a stdio shape. It caught one in ours too: VS Code 1.137's own --add-mcp writer uses a bare servers map (no mcpServers key, no type field), so the vscode flavor now matches the vendor's writer exactly, migrating 1.0.8-era entries. User-customized entries are never touched in any flavor. The exact commands every config runs were also proven end-to-end: an MCP stdio handshake against astria mcp (initialize → 10 tools → real query_graph/affected calls).
  • Three install-path corrections from a docs audit of this repo's own agent integration (all caught before release, against vendor documentation): Kiro's workspace MCP config is .kiro/settings/mcp.json per kiro.dev's configuration docs — not a bare root mcp.json, which no Kiro version reads (the dead file a dev build had written is deleted on install/uninstall when it only carries our entry); OpenCode's documented project plugin directory is .opencode/plugins/ (plural) — the singular plugin/ still loads on 1.17.8 but is not the documented convention, so the installer writes the plural directory and migrates both legacy layouts; and the Copilot coding agent reads repository-level MCP only from repository Settings on github.com (confirmed in github/docs: "Configure MCP servers for your repository" — JSON entered in the Settings UI, mcpServers shape) — the .github/copilot-mcp.json a dev build wrote is read by nothing and is cleaned up. The injected ## astria instruction block also caught up with the tool surface: it now names all ten MCP tools (it still said six, omitting god_nodes, list_communities, graph_stats, health), and this repo's tracked AGENTS.md is now itself a managed section (<!-- astria:managed -->) so astria install keeps it in sync instead of treating it as user-owned forever. The repository's own tracked dead copies (root mcp.json, .github/copilot-mcp.json) are dropped in the same change — every working per-tool config stays machine-local behind astria install, with AGENTS.md and .github/copilot-instructions.md (GitHub's documented repo instructions file) the only tracked agent artifacts.
  • Six new platforms: vscode and windsurf (MCP-only — no skill-file mechanism to target), cline (skill → ~/.clinerules/, AGENTS.md), roo (skill → ~/.roo/rules/, AGENTS.md), amp (AGENTS.md — Amp reads it natively), and pi — the Pi coding agent gets an auto-discovered extension (~/.pi/agent/extensions/astria.mjs, every API surface verified against pi 0.87's types) that registers the graph as native pi tools (astria_query, astria_map, astria_explain, astria_path, astria_affected, CLI-backed — pi's own philosophy is registered tools over MCP definitions: a few hundred context tokens vs 10k+), plus automatic graph refresh around tool calls and an /astria guidance command. Freshness is mode-independent by measurement: pi 0.87 print-mode sessions never deliver tool_result events to extensions (an instrumented listener captured zero events — not even for the extension's own tool calls), so every graph tool call itself triggers the throttled refresh, awaited until the update process exists before returning — the session exits the instant a tool returns, and an un-awaited detached child never materializes (both failure shapes measured). Verified end-to-end: the parent exiting immediately after the tool returns does not kill the update — graph_published_at advances. The standard .mcp.json stays registered for pi-mcp-adapter users at zero extra cost — the adapter reads it natively and was observed discovering the astria server. Fully standalone — no adapter required: verified in a pristine pi (--no-extensions -e astria.mjs, every other extension including pi-mcp-adapter disabled) where the extension loads and registers cleanly; pi's own provider serialization consumes plain JSON-schema parameters, which is exactly what the extension registers. All five tools executed through pi's AgentToolResult contract returning real graph data (the MODEL const, RESOLVED-tier blast radii, ranked repo map), and a project without .astria/ gets a build-one-first hint instead of a raw CLI error. The full chain was then verified live with a real pi session (glm-4.7): the model called the native astria_query tool on its own and answered from the graph — jinaai/jina-embeddings-v2-base-code at crates/astria-embed/src/lib.rs:13-15, file:line anchor and all. The platform roster is now sixteen.
  • astria install --all / uninstall --all — one run wires every supported platform; a single-platform install now prints the remaining platforms so multi-tool users discover the rest. Previously the command silently defaulted to claude-only, and a Codex or Trae user who ran astria install got the wrong layout without a hint.
  • Tests: the MCP flavor matrix grew from 4 to 9 parametrized JSON flavors (shape, idempotence, preserve-others, removal) plus dedicated OpenCode-shape/migration, Kiro legacy-path migration, Copilot legacy-file cleanup, Codex TOML round-trip (user-config preservation, never-clobber), and file-stem layout tests; install suite 292 assertions green.

Answer trust and first-rank retrieval — RESOLVED edges, honest directions, Rust docstrings, adaptive semantic ranking

  • RESOLVED edge tier for uniquely-bound calls — a call expression is extracted from source, but its binding (which definition the bare name means) is name inference; the graph previously called the whole edge INFERRED, so affected marked all 12 depth-1 callers of score_nodes as untrusted. Reference resolution now upgrades a call whose name binds to exactly one definition to RESOLVED (strength 0.85): above co-occurrence inference, deliberately below EXTRACTED/DECLARED so --detail high (compiler-grade facts) and health's EXTRACTED-only cycle detection still exclude it. On this repo: 3,013 of 11,828 call edges are RESOLVED; the remaining 8,815 (is_some, join, unbindable names) stay honestly INFERRED. affected shows tiers per hop and saves the legend line for genuinely untrusted hops.
  • explain/neighbors show real edge direction — the explained node was rendered as the source of every connection arrow, inverting caller/callee for incoming edges (the very artifact that made model_cached() look like it called its callers). Connections now render --> (this node calls/imports the neighbor) vs <-- (the neighbor calls/imports this node), on CLI and MCP; --json carries outgoing.
  • Rust /// doc comments and //! module docs are extracted (extraction rules v12) — functions, structs/enums/traits, and file nodes now carry their documentation as docstrings, feeding the doc-evidence ranking, the embedding text, and explain. Before v12, Rust had zero extracted docstrings: the ranking layer's docstring evidence and the open-bench audit's "docstring evidence" fix had nothing to work with in Rust code. 405 code nodes on this repo gained docstrings; only text-changed nodes re-embed (320 on the migration update).
  • Description-shaped queries let strong embeddings rank — when none of a question's identifying (salient) terms has lexical evidence anywhere ("auth flow" against a graph that never uses those words), a strong calibrated cosine now maps into the label-match tier (2.0 at strong_match, capped 2.6 — still under an exact label match's 4.0) instead of capping below every partial token match and waiting for the reserved seed slot. When the graph does answer the question's vocabulary, the tie-breaking cap stands unchanged. The mapping lives on astria_core::calibration::SemanticCalibration (description_seed_score), measured anchors like every other threshold.
  • The embedder loads once per process — the ONNX session (~1s, ~600 MB) is cached behind a lock and reused across queries; the MCP server's steady-state embedded query drops from ~1.4s to ~140ms (measured 917ms first call → 137ms cached in one process), and one instance bounds memory regardless of worker threads.
  • Measured end-to-end (self-corpus golden, 35 questions, same graphs rebuilt at v12): MRR 0.636 → 0.682, hit@1 51.4% → 60.0% (+3 questions answered first), hit@10 unchanged at 91.4%. The previously-missing "what breaks if I change a function blast radius" now surfaces affected.rs (the description-shaped semantic path, rank 15 behind the prose that literally documents the feature — prose-first on feature-intent questions is kept deliberately). Two honest non-wins: "which languages does tree-sitter extraction support" still misses (the docs page and generator script genuinely answer it as well as the golden code files — golden ambiguity, not ranking), and the partial-overlap probe ("untrusted paths blocked…") still seeds on the dedup *_blocked() functions whose labels the question's common words really do match — that shape needs phrase-level intent matching, not another weight.

Embedding model swapped for code-aware retrieval

  • astria-embed now runs jina-embeddings-v2-base-code (fastembed/ONNX, 768 dims, 8k-token context, ~615 MB one-time download) instead of bge-small-en-v1.5. The swap targets the measured binding constraint from the open-bench audit: a natural-language description must rank its true function among thousands of code nodes, and the previous general-prose model's reserved semantic seed hit the RepoQA needle 0/10 times on psf/black. Model choice keeps the local-first property (no API key, offline after download); the one-time download grows from ~90 MB to ~615 MB and all docs now say so.
  • Embedding reads are model-scoped: rebuild_similarity_edges and semantic_scores only score vectors stored under the current model, so a graph that still carries pre-swap vectors never mixes cosines across models (or dimensions); embed_missing_nodes already re-embeds on model mismatch, and model_cached() now probes the current model's cache directory specifically instead of accepting any downloaded model — so a query-time load can never start a surprise download when only a different model is cached.
  • Node text cap raised 1,500 → 4,000 chars (the jina model's 8k-token context takes full docstrings without harmful truncation), and the ignored real-model round-trip test's absolute floor drops 0.6 → 0.45 for the new model's cosine scale.
  • Cosine thresholds now live in one measured calibration table (astria_core::calibration, crates/astria-core/src/calibration.rs) instead of scattered bge-scale constants: noise floor 0.45, strong-match 0.70, reserved-seed slot floor 0.52, similar_to threshold 0.75 — all measured on the Click corpus with the astria-embed calibrate example (cargo run --release -p astria-embed --example calibrate), and astria-query's semantic_seed_score/seed-slot floor now derive from it. Inheriting bge's 0.55 anchors would have dropped jina matches below 0.55 before ranking ever saw them (measured: jina's noise p95 sits at 0.24–0.44 and relevant matches at 0.31–0.69 on psf/black needles, while bge's compressed scale put half of all nodes above 0.55). Model-swap separability also improved structurally: only 0.79% of node pairs clear cosine 0.60 under jina vs 23.1% under bge.
  • The open-bench and LoCoMo runners accept ASTRIA_BIN to measure a working-tree CLI without installing it (repoqa.mjs, run-locomo.mjs; locomo-qa.mjs gained --astria).
  • Validation, measured end-to-end (same 4-repo/40-needle RepoQA sample, current CLI, fresh graphs): structural 40.0/60.0/75.0 file hit@1/5/10 → jina+calibration 40.0/62.5/80.0 (func hit@1 unchanged at 25.0; psf/black needles identical to structural — its descriptions share vocabulary with the code, so token evidence dominates and the semantic layer only adds at the margin). On prose, LoCoMo retrieval over 300 questions is neutral-to-slightly-positive (recall@1 63.0→63.7, recall@5 85.7→85.3 — noise-level). Honest reading: the swap buys a decisively better semantic substrate (separation and selectivity above) and a modest end-to-end gain at the deep ranks (hit@10 +5 points), not a needle-finding breakthrough; most RepoQA needles are not pure zero-vocabulary queries, which is where the reserved semantic seed matters. Exploiting the better cosines in the merged scoring path (heavier semantic weighting for description-style queries) is the identified follow-up, not another model swap.

Judge-layer A/B harness

  • New scripts/bench/quality/judge-ab.mjs productizes the one-off 2026-09-28 modes experiment into a committed harness: build (or reuse, via --mode-dir) plain/llm/llm-jev graphs over one corpus, score every golden set at each budget × detail tier with the run-quality file-rank methodology, and emit a results JSON plus markdown report with graph shape, build tails, and engine/judge provenance. plain runs keylessly; llm needs the engine key; llm-jev additionally requires a Typesafe key and the harness fails before any build spend when one is missing. Keyless re-scoring of the prebuilt 2026-09-28 mode graphs reproduced the experiment's conclusion: on Click, frozen/heldout-v1 sets are perfect on every mode, llm-jev costs a little heldout-v2 recall@5 (80% vs 100%) and doc-intent is mode-neutral — the judge changes graph shape (337 vs 153 communities; 201 fewer edges), not retrieval on these sets. The harness records the judge's measured gate stochasticity (31 vs 17 files gated on identical replay) in every report.

LOCOMO QA decomposed — retrieval vs reader vs ceiling, shared-model Graphify leg

  • scripts/bench/open/locomo-qa.mjs now decomposes QA accuracy into three legs per tool: retrieval (gold evidence sessions in top-1/top-3, keyless), retrieved (the published reader-over-top-3 leg), and a tool-independent ceiling (same reader and assembly over the gold sessions). The summary splits every retrieved-leg failure into retrieval miss vs gold-retrieved-but-lost (and the latter into assembly-dilution vs reader-limit), with per-category accuracy. --tool astria|graphify|both runs the shared-model protocol — both tools' retrieved legs under one reader and judge in a single process, the cross-publishable shape.
  • First decomposed run (n=30 non-adversarial, gpt-4o-mini reader+judge, fresh 1.0.7 graphs): astria retrieved-leg accuracy 33.3% (mean coverage 0.325) against a 56.7% gold-context ceiling — retrieval gold@3 66.7%, gold@1 53.3%. Of 20 failures: 9 were retrieval misses, 11 had gold in the top-3 (6 of those the reader answered from gold alone — context assembly dilution; 5 the reader lost even with gold — reader/judge limit). So roughly a third of the remaining gap is retrieval, a third assembly, a third reader/protocol — and the 12k-char context cap is a stated protocol bound. The same 30 questions scored 23.3% under the pre-1.0.7 pinned CLI, so the prose-ranking fixes carried ~10 points end-to-end.
  • Graphify structural on prose is a null result, now measured: orig_run.py --include-documents feeds detected markdown through graphify's own extract_markdown dispatch (opt-in; code-corpus benchmarks unchanged), yet its label-seeded query still surfaces nothing for prose questions — 0% gold@3 (1/30), 0% QA accuracy under the same reader and judge, 29/30 failures retrieval misses. Graphify's published LoCoMo numbers therefore rest on its LLM layer; its structural substrate cannot rank prose content.
  • The QA jsonl evidence names are matched to ranked paths by basename (evidence entries are bare session names; ranked paths carry the .astria/transcripts/ prefix).

First measured comparison against targeted search

  • The external runner's deterministic baseline (question-derived rg terms, term-occurrence file ranking, source windows around first matches) now has a checked-in measurement (worked/external-baseline/). Across the eight Click/Express/ripgrep golden questions, structural astria retrieval ranked the defining file first for 8/8 at every budget from 250 to 4,000 tokens — 184–234 delivered tokens per response at the 250 budget, zero clipping — while the baseline never ranked it first at any budget (0/8 in the top five at 250–500, 2/8 at 4,000, best ranks 4–12). Relative to this single-pass floor that is ~19× less delivered context for strictly better first-shot ranking. Boundaries carried in the write-up: deterministic baseline, not an expert iterative searcher; n=8 symbol-heavy questions; one observation per condition; graph-build cost excluded from token accounting. The docs site's retrieval validation page gains the September 30 section.

Answer trustworthiness — provenance, no-confident-match, model-swap self-heal, constant nodes

  • affected labels INFERRED hops — blast-radius hits now carry the traversed edge's provenance and render it (spec() (calls INFERRED) via tests/spec.rs, with a legend line on CLI and MCP; --json includes the field). Measured motivation: INFERRED call edges are reconstructed from name references and their direction is not guaranteed — three real call sites call model_cached() while the graph recorded the reverse — so a radius that mixed them silently with source-verified edges overstated certainty exactly where agents are told to trust it.
  • File cycles are EXTRACTED-only — astria health's file-cycle SCC now consumes only edges the source literally contains. Including INFERRED calls edges previously chained 82 unrelated files (astria-pdf, viewer.js, bench scripts) into one "cycle" — 10 of the 26 deducted health points rested on that single artifact.
  • No-confident-match floor — when no node in the graph matches any of the query's salient (highest-IDF, answer-identifying) terms AND no node matched even half of the effective terms, query returns an explicit miss naming the missing vocabulary instead of traversing ("No confident match: none of the question's key terms (payroll, overtime, pay) appear in the graph's labels, docstrings, or ids…"). Two measured cases now refuse: "Where is OAuth authentication handled?" (previously seeded handle_message() through the fuzzy "handled"→"handle" rescue; 445 nodes of unrelated code) and "How does the payroll module calculate overtime pay?" (previously seeded index.module.css on the common word "module" — one full label match out of five terms is real evidence, but it does not identify an answer). Weak-but-on-topic matches are never refused: any node covering half the effective terms ("session handling" → SessionManager) traverses, entry-intent queries are exempt (their terms match nothing by design and the import DAG answers them), a qualifying semantic-only candidate overrides (zero-overlap conceptual queries are embeddings' job), and ASTRIA_QUERY_SEED_FLOOR=off restores always-traverse.
  • Model swaps self-heal on update — the silent embedding refresh now treats foreign-model vectors as "embeddings exist": an ordinary astria update re-embeds them through embed_missing_nodes' model-scoped query (previously the current-model-scoped has_embeddings gate made the jina swap silently skip, stranding the graph with vectors no query could score). When the new model is not yet cached the refresh still stays offline, but the mismatch is printed with the one-time --embed migration command instead of silence (stored_embedding_models powers both paths).
  • Rust constants are nodes (extraction rules v11) — pub and ///-documented const/static items extract as constant nodes whose signature carries the initializer verbatim (pub const MODEL: EmbeddingModel = EmbeddingModel::JinaEmbeddingsV2BaseCode;) with the /// block as docstring; initializer calls attribute to the constant. Value questions ("which embedding model", "what threshold") were structurally unanswerable before — the answer's carrier was not a node. Private undocumented constants stay out. The v11 hash bump re-extracts structurally on the first update; the LLM semantic cache is keyed separately and is not invalidated.
  • Release validation (self-corpus golden, 35 questions, local 1.0.8 build, fresh graph with jina embeddings): MRR 0.636, hit@1 51.4%, hit@5 85.7% (recall@5 81.4%, gate ≥50% passed), hit@10 91.4%, zero failed queries, ~1.4 s per query; token benchmark unchanged (160.5× reduction, ~3,405 tokens per answer). The floor refused zero golden questions — both misses ("tree-sitter language support", "blast radius") delivered full ~4,000-token traversals whose prose matches (docs pages) outranked the golden code files, the same rank-drift shape as before. The two refusal probes above were re-measured after embedding the graph: the payroll/helicopter family now traverses when jina's calibrated cosine clears the reserved-seed floor on its on-topic half (ingestion), which is the semantic override working as designed; with embeddings off they refuse. Live probes: MODEL is the top-1 seed for "which embedding model" (previously unanswerable), the 82-file health "cycle" is gone (one credible 5-file cycle remains), and affected score_nodes marks all 12 depth-1 hops INFERRED — this codebase's function-level call edges are largely inferred, now visibly so.

Entry-intent phrase matching, explicit corpus mode, transcript writer

  • Entry-point intent matches intact phrases — the trigger is now a contiguous, stem-equal token run instead of a substring test, and bare bootstrap is removed from the phrase list. Two false-positive classes die with it: "domain files" contains the substring "main file", and any question touching it re-ranked the import root above every lexical match the question had earned (the boost is a lexical-ceiling override, so a false intent is not a nudge — it wins outright); "bootstrap" names a CSS framework in plenty of repos. "entry points"/"entrypoints" still match via plural stemming; the entry-point golden ("what is the CLI entry point") is unaffected. Regression tests pin both failure shapes.
  • Corpus mode is explicit — the hidden prose_share > 0.95 switch that re-ranked chunks/documents and opened the doc-seed quota is now a named, designed threshold (DOCS_MAJORITY_PROSE_SHARE) behind an explicit CorpusMode: auto-detected by default, pinnable with ASTRIA_CORPUS_MODE=docs|code (unrecognized values warn and fall back to auto). The query header discloses the active mode whenever it is not auto-detected code-majority (# corpus: docs-majority (99% prose), or , pinned via ASTRIA_CORPUS_MODE), and the no-confident-match floor reads the same mode — one decision, stated in the answer, instead of three scattered comparisons against a magic number.
  • astria add --transcript <file|-> — the transcript-sidecar contract gets a built-in writer: a file path is saved under .astria/transcripts/ with its (filesystem-sanitized) name kept, and - reads piped stdin into a timestamped file, so any transcription tool can stream straight in (whisper ... | astria add --transcript -). The graph updates in the same run. Before this, the only writer of transcript sidecars anywhere was the LoCoMo bench preparation script.

Full Changelog: v1.0.7...v1.0.8