Skip to content

devia 0.8.0 — context-aware memory delivery, and the evidence that proved it

Latest

Choose a tag to compare

@schneiderjoseph schneiderjoseph released this 13 Sep 15:41
· 7 commits to main since this release

Two commands, and the hardening pass that proved them. The standard moves to 0.2.0 with three
new rules: AGT-012, AGT-013, PRIV-005.

0.7.0 was never published. It prepared these two commands, but no source state for it
survived and 0.8.0 rewrote their surfaces before either reached npm. Its CHANGELOG section is
kept as the record of what the commands were when they were written. npm goes 0.6.0 → 0.8.0.

Nothing changes for a repository that does nothing. Both features are additive; check,
validate, doctor, rules, read and sync behave exactly as before, and neither new
command needs GitHub authentication, network access or a dependency to do its local work.

Context-aware memory delivery — devia context

The smallest sufficient context for one task. The corpus is addressable items, not files: a rule,
a memory section split at its heading, one never/always line, one gap or debt row, one
impact-map duty. Every selection carries its reason, so --explain answers both "why is this
here?" and "why is that not?".

Measured on this repository: a 10,286-token corpus reduced to 598 tokens for one task in strict
mode at a 600 target — 94.2%, with every blocking rule still present.

Token-budgeted context, advisory or strict

Three numbers always travel together, so a target is never mistaken for a floor:

Target              1200 tokens  (advisory)
Mandatory floor     1642 tokens in 37 items
Selected            1642 tokens in 37 items
Status           OVER TARGET
advisory (default) strict
Mandatory items always whole compressed toward their identifier, never dropped
The target may be exceeded, and says so never exceeded
When it cannot fit OVER TARGET, with the floor IMPOSSIBLE — nothing produced, exit 1

Compression is minimal and monotonic: every mandatory item starts at its smallest form and is
bought back toward full text in relevance order, so a larger target always returns more text.
context.budget and context.mode in devia.json; context.maxTokens is still read.
devia check gains CTX-BUDGET (P2), which compares a declared target against the floor so a
target nobody revisits cannot quietly become a permanent overrun.

Impact-map-aware routing

impact-map.yaml is the one routing table the project itself wrote. A declared change type now
routes the domains of the memory files it names, so a project that invents new_consent_record
routes exactly as well as a built-in does. --type declares it explicitly.

Real-world contribution evidence — devia contribute

A devia problem hit inside somebody's repository, turned into an issue or a pull request under
two hard constraints.

Evidence, not opinion (AGT-012). A candidate is eligible because devia re-ran the recorded
invocation inside a minimal fixture and observed the reported behaviour:
observed → reproduced → fixed → issue or pull request. Every run records a digest of the
fixture and of the bin/ + src/ that ran it, and a fix must agree about the first and disagree
about the second — editing the fixture until it passes reports the reproduction changed between the two runs and stays at reproduced.

Sanitization and privacy (PRIV-005). Nothing is uploaded. A payload carries a standalone
fixture, devia's version metadata and the two behaviours. Secrets, credential assignments,
addresses, IP addresses, the home directory, the account name and the repository path are
redacted on the way in and the redactions are reported; an environment file is refused outright
rather than sanitized and copied. The finished payload is re-scanned, and a surviving secret
shape blocks the upload rather than warning about it.

The GitHub workflow is not automatic. submit writes the payload and prints the gh command
for you to run. submit --yes is the only path in devia that can reach the network, and it
refuses unless the candidate is eligible, the payload is clean, an identity is declared in
devia.json, that identity is not the maintainer's, and gh is authenticated as it. devia
stores no token, reads none from the environment, and never commits, branches or pushes in
anyone's checkout. A security defect is routed to the private advisory path and never becomes an
issue or a PR. "contribution": { "enabled": false } turns the whole feature off.

Benchmark

npm run benchmark:context runs 352 combinations: 4 corpus shapes × 11 task types × 4
targets × 2 modes. Measured on this release:

Critical-rule recall     100% in 352/352 runs
Routing accuracy         100% in 352/352 runs
Strict budget compliance 176/176 runs never exceeded the target
Advisory over target      68/176 runs, all because the mandatory floor exceeded it
Mean task-generic share  66.9% of selected tokens
Mean supporting filler    2.3% of selected tokens
Mean selection time      1.7 ms per run

In all 68 advisory overruns, selected equals the mandatory floor to the token — the excess is
entirely mandatory items and nothing optional was added on top. That is an invariant the
benchmark and the test suite both enforce.

Token counts are estimates from src/lib/tokens.mjs, not a tokenizer's output. Cost per
correct decision is not measured
: it needs an agent and a graded task set, which this benchmark
does not have, and the output says so rather than implying otherwise.

Hardening and regression coverage

Closes G11 (a target that is never revisited), D12 (routing by the project's own vocabulary)
and D10 (evidence bound to the experiment that produced it).

The benchmark found defects the percentage alone never would have: strict compression that shrank
mandatory rules to bare identifiers and then spent the freed tokens admitting optional rules at
full text — precisely backwards, now a test; a "noise ratio" that read 0.0% in all 352 runs
because nothing could score above zero; and a corpus rebuilt for every combination, forty times
slower than the thing it measures (79s → 1.5s).

Verified for this release: 135 tests pass · validate clean (141 rules, 551 links, index
current) · devia check P0 clear · memory validate 19 pass · 352 benchmark runs · tarball
installed into a clean directory and exercised through --version, init, validate, check,
context (both modes) and contribute.