Skip to content

Releases: RahulDhiman93/Tokenomy

v0.1.10

Choose a tag to compare

@RahulDhiman93 RahulDhiman93 released this 18 May 07:22
Immutable release. Only release title and notes can be modified.
c7c06f4

Tokenomy v0.1.10 — Reliability + availability

A reliability/availability follow-up to 0.1.9. Two independent code reviews (opencode qwen 3.6+ and codex CLI) plus internal audit drove the work. 30+ named phases shipped across 49+ commits on feat/reliability-0.1.10. 17 rounds of codex review iterated to consensus (9+ consecutive zero-P0/P1 rounds at ship time).

Test count went from 840 → 909 (+69 net). Two RCE-class supply-chain holes closed.

TL;DR

  • Security: self-update no longer runs npm lifecycle scripts; the graph builder no longer imports node_modules/typescript from a malicious repo by default; git binary is verified absolute-path before spawn; OSS-search responses are capped + labelled untrusted_snippet; hook stdin gets a JSON depth limit.
  • Storage integrity: paired-commit + per-snapshot SHA + cross-repo identity check + .corrupt/<iso>/ quarantine. Crash mid-commit no longer leaves silent corruption; tampered or cross-repo snapshots get caught at load.
  • MCP hardening: top-level structured-error catch (no more transport-killing throws), inflight cap, per-tool monotonic deadline, new health tool exposing live integrity / worker / inflight state.
  • Build correctness: TOCTOU guard on source reads, equal-mtime defense via size + inode, sentinel cap (1 MB / 10 K entries), foreign-user .dirty recovery, concurrent-safe NDJSON freshness counters, monotonic clocks everywhere durations matter.
  • Watcher self-heal: fs.watch failures fall back to polling (NFS / FUSE / inotify quota); inotify-quota surface in tokenomy doctor.
  • Performance: process-local snapshot LRU, glob ReDoS guard, budget binary-search truncation, MCP response compaction.
  • Report: multi-repo discovery (no more "1 repo" bug), per-repo roll-up table, integrity status surfaced per repo.
  • CLI ergonomics: NO_COLOR + --no-color support; symbolic-name realpath canonicalization; Windows reserved-name guard.
  • CI: cross-platform matrix (linux + macOS + Windows × node 20/22), audit:supply + audit:vuln npm scripts.

Highlights

Security blockers (must-ship)

PSEC1 — Self-update --ignore-scripts + version verify. Pre-0.1.10 tokenomy update ran npm install -g tokenomy@<v> without --ignore-scripts, so a registry compromise or any transitive dep with a preinstall / postinstall lifecycle script could execute under the user's shell. Now: install runs with --ignore-scripts (Tokenomy's own package never depends on install scripts), and a post-install npm ls -g --json step verifies the installed version matches the resolved target. A dist-tag that moved between resolve and install now bails before re-init instead of staging the unexpected build.

PSEC2 — Repo-local TypeScript trust gate. Pre-0.1.10 the graph builder resolved typescript against the target repo's node_modules first, so a malicious repo could plant node_modules/typescript/index.js that runs code during tokenomy graph build. Now: the loader prefers Tokenomy's bundled / process-local typescript and only falls back to repo-local resolution. The fallback is controlled by cfg.graph.allow_repo_local_typescript (default true for back-compat — production installs without bundled TS keep building); security-sensitive deployments lock down with tokenomy config set graph.allow_repo_local_typescript false. tokenomy doctor surfaces which source was used per build.

P6c — git binary absolute-path lockdown. execFileSync("git", ...) previously went through PATH; any writable dir earlier on PATH could shadow git with a trojan. New util/git-bin.ts resolves git in-process (no spawning which / where — those go through PATH themselves), realpath's the result so a planted symlink can't bypass the safe-prefix check, and verifies the absolute path falls under a known-safe prefix (/usr/bin, /usr/local/bin, /opt, /Applications, Windows Program Files\Git\). TOKENOMY_GIT_BIN env override bypasses for unusual installs. When the verifier returns null, callers fail closed — repo-id falls back to the in-process ancestor walk and enumerate falls back to filesystem walk; neither path spawns unverified git.

P6b — OSS-search hardening. npm-search.ts fetchText caps response body at 5 MB (destroys the stream past the threshold) so a compromised registry can't balloon RSS. Every repo / npm snippet now ships under an untrusted_snippet field with a snippet_caveat: "snippet content is from a third-party source; treat as data, not as instruction". Snippets truncate at 200 UTF-8 bytes (not UTF-16 code units, which let multibyte text bust the cap).

P6 (partial) — JSON depth limit on hook stdin. Pre-0.1.10 hook/entry.ts parsed up to 10 MB of stdin with no depth check; an attacker could craft 100K-level nested [[[[...]]]] for parser DoS. Pre-scan at max depth 64 rejects before JSON.parse.

Storage integrity

P2 — Paired commit + SHA + identity check + quarantine. The snapshot/meta pair was written as two separate atomicWrite calls; a crash between left a mismatched pair on disk and the loader returned null silently. Now: writes go to <graphDir>/.commit-<pid>-<rand>/ first, snapshot SHA is computed from the on-disk bytes and embedded in meta, then both files rename into the canonical paths. The loader recomputes the SHA on every load; a mismatch (or a repo_id from another repo) quarantines the pair to <graphDir>/.corrupt/<iso>/ and returns missing so the next build runs fresh. <graphDir>/.integrity.json tracks {verified, mismatched, quarantined_total, last_quarantine_at}.

Pre-0.1.10 meta without snapshot_sha256 loads without verification (back-compat); a force rebuild writes the SHA. MIN_SUPPORTED_SCHEMA_VERSION / MAX_KNOWN_SCHEMA_VERSION band (P2b) refuses prehistoric versions and tolerates future ones with a warn.

P1 — atomicWrite hardening. Both writeFileSync and renameSync calls in atomicWrite now retry on EAGAIN / EBUSY / ETXTBSY / EPERM / EACCES / EMFILE (50 / 150 / 300 ms backoff). After 3 rename failures, falls back to copyFileSync + best-effort unlinkSync — handles the Windows AV / Search Indexer hold-target-open scenarios. Structured AtomicWriteError carries the underlying errno code so callers branching on e.code === "EACCES" keep working. Target gets fsync after rename for power-loss durability.

P10b — Build-time TOCTOU guard + equal-mtime defense. safeReadStable(absPath) does stat → SHA → read → stat and verifies inode / mtime / size are stable across the read; mid-read mutation retries once and surfaces parse_error reason: "unstable-during-build" if still unstable. Meta now persists file_sizes + file_inos alongside file_mtimes; the cheap stale check requires all three match — touch -r content edits that restored mtime no longer slip through.

P10c — Sentinel cap + drive-letter normalize. .dirty rotates when it crosses 1 MB (parse, dedupe, keep newest 10 K entries). readDirtySentinel recognizes Windows-absolute paths (C:\..., c:/...) regardless of process platform; POSIX readers drop them instead of producing corrupt repo-relative keys.

P10d / P10e — Foreign-user .dirty recovery, NDJSON freshness counters. A .dirty written by another uid (root via CI hook, sudo) can no longer wedge the build into a perpetual stale loop — we rename it aside or truncate. Freshness counters switched from read-modify-write JSON to append-only NDJSON deltas so concurrent MCP queries no longer lose increments.

MCP server hardening

P3 — Top-level structured-error catch. Every dispatch is wrapped; throws (Error, string, anything) become {ok:false, code:"internal", message, request_id}. Pre-0.1.10 a bare throw escaped to the MCP SDK and killed the transport.

P4 — Inflight cap + per-tool deadline. cfg.mcp.max_inflight (default 8) bounds parallel tool calls — overflow returns {code:"busy", retry_after_ms:50, request_id}. cfg.mcp.tool_deadline_ms (default 5000) races each call against a monotonic timer; timeout returns {code:"timeout", elapsed_ms, request_id}. Build-triggering tools (build_or_update_graph plus the cacheable read tools that can sync-block on a missing graph) get the larger cfg.graph.build_timeout_ms deadline so first builds don't surface false timeouts.

P5 — health MCP tool. New tool returns {worker_active, last_build_ms, schema_version, snapshot_integrity_ok, snapshot_sha256, inflight, inflight_max, version, uptime_ms}. snapshot_integrity_ok reflects current state (compares last_quarantine_at vs meta.built_at), not the cumulative mismatched counter — a successful rebuild after a quarantine resets the flag.

Watcher self-heal + polling

P7 — fs.watch failure fall-through. WorkerEntry.mode is "watch" | "poll" | "off". EMFILE / ENOSPC / EBADF errors fall through to a setInterval(500).unref() that stat's .dirty and triggers on any change to (mtime, size, ino) — coarse-mtime filesystems no longer miss edits. EMFILE / ENOSPC additionally writes .last-async-failure.json with the actionable inotify-quota hint. Empty .dirty no longer schedules useless rebuilds (size-gate applied to every reader site).

Multi-repo report fix

P9a — tokenomy report discovery. Pre-0.1.10 the report only counted the cwd's repo, so users with N tracked repos saw a single-repo summary. New cli/report-repos.ts walks both home-mode (~/.tokenomy/graphs/) and in-repo (via projects.json) storage, dedupes by repoId preferring the newer built_at, and renders a "Repos tracked: N" table with shortId · path · nodes · last_build · integrity status. integrity_ok per-repo also routes through the last_quarantine_at vs built_at recency check so recovered repos stop showing QUARANTINE.

Internal correctness

  • Monotonic clocks for build durations, MCP dea...
Read more

v0.1.9

Choose a tag to compare

@RahulDhiman93 RahulDhiman93 released this 16 May 08:09
Immutable release. Only release title and notes can be modified.
bcca3ec

Tokenomy v0.1.9 — Graph staleness, fixed at the core

Users reported the code graph being "constantly marked stale mid-session" — to the point that agents stopped trusting the freshness signal and fell back to broad Read sweeps, defeating the point of the graph MCP. 0.1.9 is a focused, core-level pass on that problem: five architectural fixes plus 29 rounds of Codex review (0 P0 / 1 P1 / ~47 P2-P3 findings, all resolved).

TL;DR

  • stale: true no longer fires for unrelated edits. Read responses now carry a precise stale_in_scope list — the subset of edited files that intersect this query's reachable surface.
  • whole_graph_stale: true distinguishes config-level invalidations (tsconfig/exclude flip) from ordinary drift.
  • An in-process rebuild worker watches .dirty with fs.watch + 150 ms debounce and rebuilds before the next query needs it. Reads observe lag instead of driving rebuilds.
  • A mid-build edit can no longer be silently swallowed — postBuildSuccess snapshots the sentinel's inode + mtime + size at lock acquisition and only clears .dirty when those match.
  • Statusline + tool responses now share the same truth source. No more "badge says fresh, response says stale."
  • tokenomy report and tokenomy analyze gain a Graph freshness block with rebuild count, latency, and scoped-stale hit/miss ratio.

Highlights

Query-scoped staleness

Every cacheable graph tool (find_usages, get_impact_radius, get_minimal_context, get_review_context) now emits four staleness fields:

field semantics
stale_files whole-graph drift list (sentinel + TTL-cached mtime/added-file walk)
stale_in_scope subset of stale_files that intersects this query's reachable surface
whole_graph_stale set when an exclude/tsconfig/.tokenomy.json change invalidates every query
lag_ms wall-clock age of the oldest unconsumed dirty signal (never cached)

stale: true stays conservative (true whenever any drift exists), because the scoped surface is built from the OLD snapshot and a new edit can introduce edges the snapshot can't show. Callers compare:

if (result.stale_in_scope?.length === 0
    && !result.whole_graph_stale
    && result.stale) {
  // drift exists but is likely unrelated to this answer — proceed at low risk
}

In-process rebuild worker

The MCP server (tokenomy graph serve) now spawns a per-repo fs.watch + debounce loop:

  1. Agent calls Edit / Write / MultiEdit / NotebookEdit
  2. PostToolUse hook anchors the edited path against input.cwd and appends to <graphDir>/.dirty
  3. fs.watch fires; worker debounces 150 ms and runs buildGraph
  4. Next MCP read sees stale: true + lag_ms while the rebuild is in flight, OR a freshly rebuilt graph

Reads no longer drive rebuilds — they observe lag. Cross-platform (POSIX absolute, Windows C:\…, ./prefix, backslash separators all collapse to forward-slash repo-relative). Cleanly opt-out for hostile filesystems:

tokenomy config set graph.rebuild_worker.enabled false

The worker also auto-disables when graph.async_rebuild: false (the documented synchronous-rebuild opt-out — users typically set it when they need build failures like repo-too-large to surface on the call that triggered drift).

Sentinel race guard

buildGraph snapshots the dirty sentinel's inode + size + mtime at lock acquisition. postBuildSuccess clears .dirty only when all three match — any mid-build edit leaves the signal for the next cycle. Build results call reflectPostBuildSentinel, so a synchronous rebuild that left .dirty returns stale: true instead of misleading callers into thinking the graph is current.

Statusline alignment

Pre-0.1.9 the badge used a 24-hour built_at heuristic, divorced from the MCP read path. Now it reads the same .dirty sentinel + meta validity that the read path uses, so:

[Tokenomy v0.1.9 · 4.2k saved · graph stale - rebuild]

…and the tool response's stale flag never disagree.

Graph freshness in report + analyze

Graph freshness
  worker:            active
  rebuilds:          14   last: 132ms   avg: 156ms
  last rebuild:      2026-05-16T14:23:45.123Z
  dirty pending:     0 file(s)
  scoped stale:      8 hit / 47 miss  (15% of drift actually relevant)

The hit/miss ratio quantifies the win — "X% of drift was actually relevant to a query." Counters live in <repoRoot>/.tokenomy-graph/.rebuild-stats.json.

Codex review pass (29 rounds, all green)

severity count examples
P0 0
P1 1 worker retry loop bounded to build-in-progress with 3-attempt cap
P2 ~35 sentinel race, cache key collisions, server-mode worker gate, cross-platform path normalization, exclude/tsconfig fingerprint preservation, added-file detection, in-flight build coalescing, runtime config opt-out, storage-location flip, budget-clip protection
P3 ~12 scoped-stale doc precision, hotspot surface bounds, malformed-meta validation

Round-by-round notes are inlined as code comments (codex round N P[1-3]) for future debugging context.

Migration

No schema bump. No breaking changes.

  • stale_in_scope, whole_graph_stale, and lag_ms are additive optional fields on Ok<T>. Existing consumers that read only stale and stale_files continue to work.
  • graph.rebuild_worker: { enabled: true, debounce_ms: 150 } is the new default. Existing .tokenomy.json files inherit it from DEFAULT_CONFIG.
  • .rebuild-stats.json is created on first worker registration or first recordScopedStaleSample call; absent file = zeros.
  • Statusline and read-path use the same staleness check, so the badge may now flip to "stale" sooner than the old 24h heuristic would have. This is expected — it reflects what the agent actually sees.

Install

npm install -g tokenomy@0.1.9
tokenomy update

In every repo where you've registered the graph:

tokenomy init --graph-path "$PWD"

Full changelog

See CHANGELOG.md.

Reporting issues

tokenomy diagnose emits a JSON health report covering every feature + environment — paste it into tokenomy feedback (or the GitHub issue) when something looks wrong.


Co-authored-by: Claude Code (primary) + Codex CLI (29-round review counterpart) — the consensus loop that landed this PR is documented in docs/features/cross-agent.md.

v0.1.8

Choose a tag to compare

@RahulDhiman93 RahulDhiman93 released this 12 May 04:12
Immutable release. Only release title and notes can be modified.
b15bf0a

What's Changed

Full Changelog: v0.1.7...v0.1.8

v0.1.7

Choose a tag to compare

@RahulDhiman93 RahulDhiman93 released this 01 May 05:49
Immutable release. Only release title and notes can be modified.
f28a358

What's Changed

Full Changelog: v0.1.6...v0.1.7

v0.1.6

Choose a tag to compare

@RahulDhiman93 RahulDhiman93 released this 30 Apr 01:51
Immutable release. Only release title and notes can be modified.
29ceaaa

What's Changed

  • release: 0.1.6 — production-scale graph defaults + diagnostic build-failure surfacing by @RahulDhiman93 in #39

Full Changelog: v0.1.5...v0.1.6

v0.1.5

Choose a tag to compare

@RahulDhiman93 RahulDhiman93 released this 29 Apr 16:59
Immutable release. Only release title and notes can be modified.
ffdcaa8

What's Changed

  • feat: 0.1.5 — production hardening, RECON v2, tokenomy diagnose by @RahulDhiman93 in #38

Full Changelog: v0.1.4...v0.1.5

v0.1.4

Choose a tag to compare

@RahulDhiman93 RahulDhiman93 released this 29 Apr 07:50
Immutable release. Only release title and notes can be modified.
378c7c3

What's Changed

Full Changelog: v0.1.3...v0.1.4

v0.1.3

Choose a tag to compare

@RahulDhiman93 RahulDhiman93 released this 29 Apr 07:30
Immutable release. Only release title and notes can be modified.
6edc25d

What's Changed

  • feat: 0.1.3 — live graph freshness, cross-repo isolation, auto update-check by @RahulDhiman93 in #36

Full Changelog: v0.1.2...v0.1.3

v0.1.2

Choose a tag to compare

@RahulDhiman93 RahulDhiman93 released this 27 Apr 00:57
Immutable release. Only release title and notes can be modified.
fc3e41f

What's Changed

  • feat(graph): ship Phase 3 — local code-graph MCP server (0.1.0-alpha.5) by @RahulDhiman93 in #3
  • feat: 10-feature release — schema-aware profiles, dedup, redaction, report, find_usages (0.1.0-alpha.6) by @RahulDhiman93 in #4
  • feat: Phase 2 — tokenomy analyze (Claude Code + Codex transcripts, fancy CLI) (0.1.0-alpha.7) by @RahulDhiman93 in #5
  • feat: Phase 4 — Bash input-bounder + branding polish (0.1.0-alpha.8) by @RahulDhiman93 in #6
  • fix(logo): use mark-primary-512-2x everywhere instead of the wide wor… by @RahulDhiman93 in #7
  • fix(logo): switch doc headers to src/assets/logo.jpg by @RahulDhiman93 in #8
  • fix(init): stage {"type":"module"} package.json alongside the built hook by @RahulDhiman93 in #9
  • fix(init): register MCP server in ~/.claude.json + auto-register with Codex CLI by @RahulDhiman93 in #10
  • fix(pre-dispatch): resolve relative Read file_path against HookInput.cwd by @RahulDhiman93 in #11
  • feat(bash-bound): strip trailing shell comments + refresh README (diagram + real savings) by @RahulDhiman93 in #12
  • chore(release): 0.1.0-alpha.9 by @RahulDhiman93 in #13
  • chore(release): 0.1.0-alpha.10 (re-publish of alpha.9) by @RahulDhiman93 in #14
  • feat(rules,analyze): enumeration trim fixes + probe detector + docs by @RahulDhiman93 in #15
  • chore(release): 0.1.0-alpha.11 by @RahulDhiman93 in #16
  • chore(release): 0.1.0-alpha.12 by @RahulDhiman93 in #17
  • feat(cli): tokenomy update + 0.1.0-alpha.13 release by @RahulDhiman93 in #18
  • feat(graph): exclude-glob support + 0.1.0-alpha.14 release by @RahulDhiman93 in #19
  • fix(graph): CLI usages wiring, find_usages cross-module + 0.1.0-alpha.15 by @RahulDhiman93 in #20
  • fix(mcp): bind query-budget into cache key + bump defaults + 0.1.0-alpha.16 by @RahulDhiman93 in #21
  • feat(graph): tsconfig.paths resolution + 0.1.0-alpha.17 by @RahulDhiman93 in #22
  • feat(nudge): OSS-alternatives-first + Write nudge + 0.1.0-alpha.18 by @RahulDhiman93 in #23
  • fix(nudge): real npm ranking via registry search + 0.1.0-alpha.19 by @RahulDhiman93 in #24
  • fix(nudge): unbreak repo_results on large repos + update restages graph + 0.1.0-alpha.20 by @RahulDhiman93 in #25
  • feat(nudge): rank repo matches by distinct token hits + 0.1.0-alpha.21 by @RahulDhiman93 in #26
  • feat(nudge): UserPromptSubmit prompt-classifier + 0.1.0-alpha.22 by @RahulDhiman93 in #27
  • feat(golem): terse-output-mode plugin + 0.1.1-beta.1 (alpha→beta graduation) by @RahulDhiman93 in #28
  • feat(beta-2): compress + statusline + bench + shell-trace + multi-agent + 0.1.1-beta.2 by @RahulDhiman93 in #29
  • [codex] Add Codex prompt hooks roadmap by @RahulDhiman93 in #30
  • feat(beta-3): 8-feature bundle + codex review fixes → 0.1.1-beta.3 by @RahulDhiman93 in #31
  • feat(raven): cross-agent handoff + review bridge → 0.1.1-beta.4 by @RahulDhiman93 in #32
  • feat: 0.1.1-beta.5 — statusline update marker + Raven in report/analyze by @RahulDhiman93 in #33
  • feat: 0.1.2 — audit fixes, Golem recon mode, Kratos shield, README split by @RahulDhiman93 in #34

Full Changelog: v0.1...v0.1.2

v0.1.1-beta.5

Choose a tag to compare

@RahulDhiman93 RahulDhiman93 released this 24 Apr 16:12
Immutable release. Only release title and notes can be modified.
8d3ae32

What's Changed

  • feat: 0.1.1-beta.5 — statusline update marker + Raven in report/analyze by @RahulDhiman93 in #33

Full Changelog: v0.1.1-beta.4...v0.1.1-beta.5