Skip to content

julie-extract v2.9.0

Choose a tag to compare

@github-actions github-actions released this 07 Jul 13:04

v2.9.0

This minor release adds workspace reference resolution on top of v2.8.1.
It is an additive artifact-schema feature (SQLite schema 34); it does not
change the extraction contract or any existing table or column.

New Capability: Workspace Reference Resolution

Extraction records where a reference is used (identifiers) and an unresolved
description of what it targets (pending_relationships). v2.9.0 adds a
deterministic, tiered pass that connects the two across the whole workspace:

  • Fills identifiers.target_symbol_id with the resolved definition symbol.
  • Records resolved pending relationships in a new pending_resolutions overlay
    and every attempted identifier outcome in a new identifier_resolutions
    overlay.
  • Runs inside the same writer transaction as the scan on every mutating flow
    (full scan → full pass; incremental update / delete → delta pass), so an
    artifact is never observed with half-applied resolution.
  • Is a derived overlay, never an extraction fact: FK cascades invalidate
    resolutions automatically when a target symbol dies, and a resolver error is
    non-fatal — the scan still commits and the report records resolution_failed.

Resolution is honest by construction: it uses no best-guess selection. An
edge resolves only when exactly one same-language candidate survives the tier
filters; otherwise it stays ambiguous or missing. A wrong edge is worse than
a missing one — it would corrupt downstream trace/impact and hide a live
symbol behind a false dead-code verdict.

Resolution tiers

Tier Signal Confidence Method
1 Same-file scope 0.95 tier1_local
2 Import-guided (TypeScript / JavaScript only today) 0.85 tier2_import
3 Receiver-typed (bounded by type_facts emission) 0.75 / 0.65 inferred tier3_receiver
4 Unique language-global (disabled for member_access / method calls) 0.55 tier4_global

Tiers 1 and 4 are universal (same-language, no per-language gating). Tier 2 is
enabled only where a fixture-tested import contract exists; tier 3 ships with its
measured type_facts-bounded coverage. Both gated tiers advertise their limits
as reference_resolution.tier2_import / .tier3_receiver capability-gap rows.

Tier 2 trusts an aliased import only when its relative module specifier
resolves to a concrete workspace file — an alias whose source module is missing
does not resolve, rather than matching an unrelated same-named symbol. Every full
or force scan also re-checks already-resolved overlays against the current
workspace and demotes any that are no longer a unique target, so a full pass
self-corrects stale rows even when the referencing file was unchanged.

Measured Resolution Rates

Measured on a real-repo dogfood scan (the julie-extractors workspace itself, one
full pass at revision 1). The headline: 38.7% of reference outcomes resolved
25,866 of 66,807. The 66,807 total is identifier_resolutions (62,189) +
pending_resolutions (4,618); the per-language rows below are the report's
by_language breakdown of that same total.

Language Resolved Outcomes Rate Resolved by tier
rust 25,573 64,932 39.4% t1=8,479 t4=17,094
qml 74 796 9.3% t1=72 t4=2
javascript 41 308 13.3% t1=34 t2=2 t4=5
r 4 128 3.1% t1=2 t4=2
elixir 26 88 29.5% t1=19 t4=7
typescript 6 63 9.5% t1=3 t4=3
tsx 7 46 15.2% t1=3 t4=4
python 15 38 39.5% t1=12 t4=3
java 9 32 28.1% t1=9
csharp 6 30 20.0% t1=3 t3=2 t4=1
kotlin 7 29 24.1% t1=6 t4=1
ruby 1 29 3.4% t1=1
gdscript 5 26 19.2% t1=5
jsx 3 25 12.0% t1=3
php 6 25 24.0% t1=6
dart 7 23 30.4% t1=4 t4=3
go 13 23 56.5% t1=6 t4=7
scala 9 21 42.9% t1=7 t4=2
bash 2 17 11.8% t1=2
swift 7 16 43.8% t1=6 t4=1
sql 0 14 0.0%
lua 4 13 30.8% t1=3 t4=1
razor 7 13 53.8% t1=7
zig 9 12 75.0% t1=7 t4=2
css 0 11 0.0%
vue 6 9 66.7% t1=6
cpp 7 8 87.5% t1=7
powershell 3 7 42.9% t1=2 t4=1
regex 0 6 0.0%
vbnet 5 6 83.3% t1=5
c 4 5 80.0% t1=4
html 0 5 0.0%
yaml 0 3 0.0%

Rust dominates the corpus (a large Rust workspace), so its 39.4% rate anchors the
overall 38.7%. The by_language total is dominated by identifier outcomes;
tier 1 (same-file) and tier 4 (unique global) carry almost all resolutions today,
which is expected until tier 2 broadens beyond TS/JS (F4) and tier 3 broadens
with type_facts emission (F2). Data and markup languages (sql, css, regex,
html, yaml) resolve 0% because their reference outcomes are no_context — they
carry no call/receiver context to resolve against.

Performance

Measured on Apple Silicon against a synthetic ~92k-identifier artifact (release
build):

  • Full resolution pass: ~1.2 s at ~92k identifiers — within the < 2 s
    design budget (~1.6× headroom).
  • Single-file delta pass: ~82 ms — within the < 100 ms design budget. The
    delta scopes its identifier-locator and covered-set loads to the files it
    touches (every co-location join is same-file), so the cost tracks the size of
    the change rather than the workspace.

The pass is set-based SQL over the writer transaction; two identical scans
produce byte-identical resolution tables. The by-names delta worklists chunk
their IN (...) binds, so a delta touching an arbitrarily large distinct-name
set resolves rather than hitting SQLite's bound-variable limit. Both budgets are
enforced by a measurement-derived regression gate (test-perf).

Known Limitations

  • The full resolution pass is O(workspace) (~1.2 s at 92k identifiers): a
    full or force scan rebuilds the whole-workspace candidate index and re-resolves
    every reference. Incremental delta passes are scoped and far cheaper; the full
    cost grows with workspace size but stays within the < 2 s budget at the measured
    scale.
  • Tier coverage is deliberately conservative and per-language. Import-guided
    (tier 2) resolution is enabled only for TypeScript/JavaScript; receiver-typed
    (tier 3) resolution is bounded by type_facts emission (broader coverage is
    tracked as F2). Ambiguous or unproven references stay unresolved rather than
    guessing. Per-language coverage is advertised in capabilities.json and
    enforced by the contract fixtures.

Capability Honesty

  • fixtures/extraction/capabilities.json gains a top-level reference_resolution
    block documenting the tier model, confidence values, outcome vocabulary,
    metadata keys, and the per-language tier posture (which languages are proven by
    fixture, which are universal, which are recorded gaps, and which are
    not_applicable). It mirrors — and points to — the machine-enforced honesty
    surfaces: the runtime language_capability_gaps rows and the
    resolution_contract fixtures.
  • Each scan emits reference_resolution.tier2_import gap rows for every language
    outside TypeScript/JavaScript and reference_resolution.tier3_receiver gap rows
    for every language, each with a concrete reason, required closure, and planned
    closure task (F4 and F2 respectively).
  • The strict data-quality gate (node scripts/language-data-quality-report.mjs --strict) now fails on either silent cells or quality-bar debts; the baseline
    stays at 0 / 0.

Contract Changes

  • Rust package versions change from 2.8.1 to 2.9.0.
  • SQLite schema version changes 34 (additive). New tables
    pending_resolutions and identifier_resolutions; the resolver now populates
    identifiers.target_symbol_id; three new reference_resolution_* metadata
    keys; three new indexes. Every v3 table and column is unchanged. See
    docs/contracts/sqlite-schema-v4.md.
  • Artifact integer extract_contract_version remains 3 — the extraction
    contract is unchanged; only the derived resolution overlay is new.
  • JSONL schema version remains 3. The JSONL artifact record gains three
    additive reference_resolution_* fields (mirroring the metadata keys, null
    when resolution never ran) so JSONL consumers can apply the same status-gating
    rule; the identifier record's target_symbol_id is now resolver-populated
    on v4 artifacts. JSON report schema version remains 3; mutating-command
    reports gain an additive languages.reference_resolution section (documented
    in docs/contracts/reports.md).
  • Consumers must gate on the reference_resolution_status metadata key, never
    on the schema version or table probing.
    A v4 artifact can carry resolution
    status absent (not yet backfilled) or failed (resolver error); schema
    version only answers whether the binary can read the artifact. A NULL
    target_symbol_id continues to mean "unknown".
  • A v3 artifact opened for reading under --strict-schema is rejected
    (schema_migration_required); resolution backfills on the write path (any
    scan / update / delete triggers a full pass that creates the overlay tables and
    populates them).
  • Parser dependency versions are unchanged.

Verification

Local prep gates (run before staging a release package):

  • cargo fmt --all -- --check
  • cargo test -p julie-extract-cli (includes the resolution unit, contract, and
    operations suites)
  • cargo test -p julie-extract-cli --test resolution_contract (per-language
    tier + parity fixtures)
  • cargo test -p julie-extract-cli --features test-perf --test resolution_perf
    (full/delta budget gate)
  • cargo test -p julie-extract-artifact (schema + storage contract)
  • node scripts/language-data-quality-report.mjs --strict
    (silent_cells: 0, quality_bar_debts: 0)
  • cargo xtask dogfood repo --root . --out-dir target/dogfood/julie-extractors
    (real-repo resolution rates above; no resolution_failed diagnostic)

Publishing this release (tag / build / upload) requires explicit maintainer
approval and is not performed by this notes draft. Published-asset verification
and the checksum table will be appended when the release is staged.