v0.27.1

pi-shazam v0.27.1

Emergency fix release. v0.27.0 shipped the full 0.27 feature set (V3 ProtoBuf
cache, impact provenance + Mermaid graph, overview module-density ranking, lookup
provenance edges, GitHub Action wrapper, and more) but was pulled because a
packaging defect excluded the runtime protobufjs dependency, breaking the MCP
server and V3 cache on a fresh install. v0.27.1 contains everything in v0.27.0
plus the dependency fix below. If you installed v0.27.0, upgrade to v0.27.1.

What's Changed in v0.27.1

Fixed

• Runtime protobufjs missing from published package (#722): protobufjs was
  declared in devDependencies but imported at runtime by core/proto-schema.ts,
  so the published 0.27.0 tarball excluded it and the MCP server / V3 cache path
  failed with ERR_MODULE_NOT_FOUND on a fresh install. Moved protobufjs to
  dependencies (kept protobufjs-cli in devDependencies) so it ships in the package.

v0.27.0 Feature Highlights (included in v0.27.1)

Added

• V3 ProtoBuf cache format (#628, #646): New compact binary cache that
  encodes edge data as columnar ProtoBuf arrays behind a 4-byte SHA\3
  magic header. On a 1000-symbol project the on-disk cache drops from
  ~800KB (V2 JSON) to ~30% smaller in practice; the V2 JSON format stays
  readable for backward compatibility. Schema lives in core/graph.proto
  (source of truth) with a JSON runtime mirror in core/proto-schema.ts
  (protobufjs, no generated static module in dist/). saveGraphCache
  now writes V3 by default; loadGraphCache detects the magic header and
  falls back to V2 for older caches, preserving the fileMtimes map so
  mtime-based invalidation is unchanged.

• Impact enrichment: provenance breakdown + Mermaid call graph (#631 B):
  shazam_impact JSON now carries a provenanceCounts field per affected
  symbol (resolved vs. heuristic at a glance) plus a compact R:N N:M H:K
  badge on the affected-file markdown line; call-chain edges carry an
  explicit provenance. Every CallChainEntry also gains a self-contained
  Mermaid flowchart TD block (mermaid field) with provenance as edge
  labels and solid/dashed arrows, capped at 30 nodes.

• Overview module-density ranking (#631 B): shazam_overview surfaces a
  topByDensity list on the JSON envelope and a "Module Density (Top 10 by
  symbols-per-file)" markdown section to flag "god module" candidates.

• Lookup provenance edges + provenance-weight sort (#631 B, #643):
  shazam_lookup JSON mode now exposes incomingEdges / outgoingEdges
  (up to 20 each, with provenance) and a precomputed provenanceCounts
  summary per entry (#643). Entries are sorted by provenance weight
  (resolved > name_match > heuristic > unresolved, PageRank tiebreak) so the
  most-trustworthy matches come first; markdown output keeps the original
  PageRank ordering.

• Changes structural (line-count) working-tree view (#631 B):
  shazam_changes adds a structuralChanges field with added/removed/
  modified line counts from git diff --numstat (staged + unstaged, binary
  files skipped, capped at 20 files), plus a compact (+N -M lines across K files) markdown badge.

• GitHub Action wrapper for shazam_verify (#638, #652): Composite
  action at .github/actions/shazam-verify/ runs shazam_verify on every
  PR and posts a risk-scored review as a PR comment (plus step summary).
  Adds tools/verify-comment.ts (formatVerifyComment), an action.yml
  manifest, orchestrator/entrypoint scripts, and tests/github-action-*.test.ts
  (25 tests). Adds the yaml devDependency.

• ProtoBuf schema mirror consistency test (tests/cache-proto-schema.test.ts):
  Parses core/graph.proto text and asserts the field set, IDs,
  types, and repeated flags match the JSON mirror in
  core/proto-schema.ts. Catches drift between the two schema
  sources before it can corrupt a cache file.

• V3.0 / V3.1 legacy deserializers (deserializeGraphV3V0,
  deserializeGraphV3V1, +WithMetadata variants) plus
  encodeGraphPayloadV31 / decodeGraphPayloadV31 for tests.
  Public surface for the in-place converter; kept so the load
  path can decode legacy wire formats.

• Real-world size sanity check in tests/benchmark-v3.test.ts:
  Documented the V3.1 ratio on a self-scan of pi-shazam (3499
  symbols, 1264 symbol-level edges, 7236 file-level rows): V3.1
  is 96% of V2 on a real project (vs. 53% on the edge-heavy
  synthetic benchmark), because real projects spend most bytes
  on the JSON metadata + non-deduped file-level rows.

Changed

• Cyclomatic complexity via tree-sitter AST (#642): Replaced the regex
  sweep that counted if|else|for|while|case|catch + &&/||/:? across
  whole source slices (which inflated scores by counting keywords inside
  comments/strings) with a tree-sitter complexity query matching real
  branching AST nodes only, applying the else-if rule (plain else adds 0,
  else if adds 1) on a baseline score of 1. Queries added for
  TS/TSX/JS/Python/Go/Rust; the original regex is kept as a fallback for
  languages without a compiled query, preserving coverage.

• Tool defaults tightened for LLM agents + per-edge provenance (#629,
  #633, #634, #635, #636; #640): Bundled four related quality PRs. Unified
  Affected-Tests detection (core/test-patterns.ts is now the single source
  of truth; symbol-mode JSON carries affectedTests). Display polish +
  path-traversal UX (classifyFilePath distinguishes traversal vs. missing;
  did-you-mean for lookup/impact/format). Per-edge provenance on Edge
  with LSP-driven promotion (upgradeEdgesToResolved / upgradeEdgesForHotspots,
  zero extra LSP latency). Always-on defaults: ### Hotspots always in
  overview text and JSON.hotspots.{byPageRank,byComplexity}; verify uses a
  single-line-per-diagnostic format with a summary header (drops suggestedFixes
  text + .shazam auto-export); inferImpactMode() picks symbol vs. files
  mode from input shape, removing the strict mutual-exclusion error.

• Verify flags migrated to .pi-shazam/config.json; lookup mode=state
  removed (#630): New core/config.ts loads .pi-shazam/config.json
  (missing/empty = silent defaults, malformed = warn). shazam_verify
maxFiles now resolves from config.verify.maxFiles (default 100); dead
  noCascade / noSecrets options removed. shazam_lookup mode=state was
  first deprecated (warn) then fully removed — callers now get a clear error
  pointing at --name / --direction.

• Default scan now excludes tests/ (#632): scanProject skips the
  tests/ directory by default (override with includeTests).

• Typed result builders across all tools (#631 A): Introduced pure,
  typed builders — buildVerifyResult, buildFormatResult,
  buildOverviewResult, buildImpactResult + buildCallChainResult,
  buildChangesResult (+ renderChangesMarkdown), typed XxxResult for all
  three lookup modes, and a kind discriminator with executeRenameSymbolJson.
  Tool internals now build a typed model before rendering JSON/markdown,
  reducing envelope drift and enabling direct result assertions in tests.

• V3.1 cache: string table for symbol ID dedup (#647): The on-disk
  graph cache (core/cache.ts V3 format) now stores each unique
  symbol ID exactly once in a top-level string_table; every edge's
  source / target is referenced by int32 index. On a 1000-symbol
  graph with ~3 edges per node this brings the V3 cache to ~53% of
  the V2 size (down from ~68% in V3.0), close to the plan's 50%
  target. The on-disk magic header bumped from SHA\3 to SHA\4;
  old V3.0 caches fall through to the V2 JSON path on load (which
  fails to parse) and the scanner rebuilds them on the next scan.
  Schema changes are additive and live in core/graph.proto (source
  of truth) + core/proto-schema.ts (JSON runtime mirror).

• V3.2 cache: kind int enum + silent in-place upgrade (#647 follow-ups
  D + E): Edge kind is encoded as int32 (1 byte varint) instead
  of string (5-7 bytes per row) — on a real-world self-scan this
  shaves another ~30-50 KB on top of V3.1. The on-disk magic header
  bumped from SHA\4 to SHA\5. loadGraphCache now auto-upgrades
  legacy V3.0 (SHA\3, no string table) and V3.1 (SHA\4, kind as
  string) caches to V3.2 in place on first read, so users on v0.27.0
  (PR-G) and the just-shipped V3.1 release of this branch pay no
  re-scan cost on upgrade. The V3.1-specific ProtoBuf schema
  (kind: string) lives in core/proto-schema.ts as a separate
  Root so the in-place decoder can interpret the legacy wire
  format; the V3.2 writer always produces the canonical int32 form.

Fixed

• Batch code-review fixes (code-review findings #687-#701): Resolves
  15 code-review-labelled findings across the tool surface. Highlights:

  • Rust import resolution (#687): Non-prefixed multi-segment use
    paths (use utils::helper::doThing) are now resolved from the crate
    root (Rust 2018+ semantics) instead of being dropped as external crates,
    so shazam_impact / shazam_verify no longer miss local Rust edges.
  • Symlink path-traversal in auto-format (#688, P0): hooks/shazam-guide.ts
autoFormatFile now validates via validatePathInProject (realpathSync
    symlink-escape check) instead of the string-only isPathInRoot, and
    imports it from tools/_factory.ts (resolving the hook→tools layer
    violation flagged in #697).
  • Silent audit-log discards (#689): Two empty catch blocks in
    index.ts shazam-doctor parsing now log via _logWarn instead of
    swallowing malformed internal.log / shazam-calls.log lines.
  • Silent cache-write failures (#690): saveGraphCache catch blocks in
    core/scanner.ts now emit _logWarn (incremental + full paths) so
    ENOSPC/EACCES/EROFS cache drops are observable, not silent.
  • MAX_FILES truncation has no output signal (#693): RepoGraph gains a
    truncated flag set when collectSourceFiles hits the cap; overview
    output now warns the agent the graph is incomplete.
  • Edge provenance lost in diff identity (#695): edgeIdentity /
    edgeIdentityFromRow now include provenance, so shazam_changes
    reports provenance-only edge upgrades (heuristic → resolved).
  • Concurrent getGraph duplicate scan (#691): mcp/entry.ts getGraph()
    gains a re-entrancy guard so overlapping MCP calls share one scan
    (defensive against the _scanning re-entry throw).
  • Impact compact mode drops affected tests (#692): shazam_impact --compact
    now reports an "Affected Tests (must re-run): N" count line.
  • LSP false PASS on large projects (#694): initialize timeout raised
    8s → 30s and diagnostic poll 1s → 10s so shazam_verify no longer
    reports PASS before the language server finishes indexing.
  • MCP depth not coerced from string (#696): dispatchImpact now
    coerces a string depth to a finite number (falls back to 3 on
    NaN / non-numeric), preventing undefined traversal behavior.
  • Wrong (mcp) label in verify PR comment (#698): formatVerifyComment
    now emits (verify) for 1-9 incoming-caller critical paths.
  • Over-broad unhandledRejection suppression (#699): vitest.setup.ts
    now suppresses ERR_STREAM_DESTROYED only when opted in by LSP teardown
    tests, so non-LSP tests that throw it correctly fail instead of passing.
  • GBK config files fail silently (#700): loadConfig reads via
    readFileAdaptive (UTF-8 → GBK → GB2312) instead of hardcoded
    UTF-8, so Windows GBK .pi-shazam/config.json parses correctly.
  • LspEnrichContext interface missing mtime (#701): trackOpenedFile
    gains an optional mtime? param matching LspManager, preventing a
    cache-invalidation regression if callers are refactored to the interface.

• MCP graph released on idle; memory / native-heap leak fixes (#626, #627):
  mcp/entry.ts getGraph() now honours a TTL (default 10 min,
  PI_SHAZAM_GRAPH_TTL_MS; 0 disables) and nulls the cached RepoGraph on
  idle so the next access rebuilds from the persistent disk cache instead of
  holding ~500MB-1GB forever. dispatchVerify no longer calls resetCache()
  (avoids two graphs coexisting during verify). lsp/manager.ts gains
  closeOpenedFiles() (called after diagnostics) so the LSP child releases
  per-document AST. core/scanner.ts resetCache() now preserves the
  TreeSitterAdapter singleton, stopping 100-300MB native-heap inflation per
  verify cycle in long-lived MCP mode.

• LSP per-file AST cache invalidated on mtime change (#641): When a source
  file was edited between verify calls, the LSP server's per-document AST
  returned diagnostics against stale content (requiring a manual touch/
  restart). LspManager now records each opened file's mtime and, before each
  didOpen, sends didClose for any file whose mtime changed so the server
  drops the old AST.

• shazam_changes keeps ## Change Summary header in compact output:
  The compact path introduced in #640 dropped the header, breaking
  tests/mcp-pi-parity.test.ts#238 which expects both Pi and MCP to emit it.
  The header is now kept (the shortcut is the absence of the other section
  headers).

• benchmark-v3 timing assertion no longer red CI under load (#650):
  tests/benchmark-v3.test.ts (the V3 cache encode/decode speed
  assertions) is flaky only when run inside the full npm test
  suite under parallel load — it passes in isolation. It is now
  excluded from the broad npm test / ci runs via
  vitest run --exclude tests/benchmark-v3.test.ts, and is instead
  executed by the dedicated benchmark CI job in isolation alongside
  tests/benchmark.test.ts. This removes environment-dependent CI
  noise from unrelated PRs without loosening the performance
  regression thresholds.

• shazam_changes compact/full output parity test parameterized (#644):
  Split the single fragile assertion into explicit clean/dirty tree variants
  so the compact no-op (#634) and full output are each verified for the exact
  sections they emit.

• Batch code-review fixes (Groups A-D, #656/#666/#658/#665/#664,
  #667/#660/#663/#661/#659, #662, #668): Addresses the code-review
  labelled findings across the tool surface. Highlights:

  • Impact direction naming (#656): computeFileDirection returns a
    stable "upstream caller" / "downstream callee" / "both" label (tie ->
    "both") so callers no longer infer direction from raw counts.
  • Impact rename-gate guard (#666): recordCallChain now resolves the
    symbol through graph.nameIndex first; a non-existent symbol can no
    longer satisfy the rename safety gate (reopens #569).
  • Format apply result (#658): buildFormatResult runs the formatters
    when not in dry-run and exposes formatResults / recommendedCommands
    on FormatResult (both text and JSON paths).
  • Verify description clean-up (#665): dropped stale "graph diffs" and
    "cache bypassed" claims from the shazam_verify description.
  • Docstring cache stat logging (#664): _extractDocstring logs
    non-ENOENT statSync failures via _logWarn instead of swallowing them;
    ENOENT still falls back to mtime=0 silently.
  • Resolve-import Rust default + path normalization (#667, #660, #663):
    .rs imports now return null by default; moduleMatchesFile,
    isNonSourceFile, and validatePathInProjectCore normalize backslash
    paths so Windows graph keys match forward-slash comparisons.
  • Project-root default (#661): path-utils / _factory default the
    project root to getEffectiveRoot() instead of process.cwd().
  • Tree-sitter types exported (#659): Tree / SyntaxNode interfaces
    are exported from core/treesitter.ts for shared AST consumers.
  • Overview JSON exposes text-only sections (#662): buildOverviewResult
    now surfaces dataStructures, entryPoints, httpRoutes,
    complexityHotspots, suggestedReadingOrder, parserWarnings, and
    moduleStructure (filter mode omits them, matching the text view);
    module-structure / density grouping normalizes Windows backslash paths.
  • MCP HOME_ONLY accepts Windows projects (#668): validateProjectRoot
    now treats a project under USERPROFILE (Windows) as in-home, not just
    the POSIX HOME.

• Windows path normalization for Git-Bash and WSL styles (#673, #675):
  core/path-utils.ts normalizePathInput() now collapses Git-Bash
  /c/foo and WSL /mnt/c/foo paths to C:\foo at ingress, so the MCP
  server and tool paths resolve correctly on Windows regardless of the
  invoking shell. All user-supplied paths must route through it.

• Platform-aware timeouts for Windows (#677): LSP initialization and
  other blocking operations scale their timeout by 1.5x on win32, and
  vitest.config.ts raises the win32 testTimeout to 180s, eliminating the
  Windows-specific timeouts that were failing the local full suite.

• MCP entry no longer kills the test worker on a bad root (#676, #679):
  mcp/entry.ts moved the process.exit(1) for an invalid PROJECT_ROOT
  out of module load and into main() (behind the isMainModule guard).
  Importing the module under vitest no longer aborts the worker, which had
  cascade-failed 12 tests that share the file; getGraph() falls back to
  cwd when imported under test. The real-server fail-closed behavior
  (exit on bad root) is preserved.

• Hook injection content and context delivery fixed (#674): Corrected
  the content and context delivered by the before-agent-start hook so the
  injected project snapshot matches what the tools consume.

• Local full-suite flaky hardening (#678, #680, #681): Windows test
  stability improvements — symlink tests probe privilege at module load and
  it.skipIf(!canCreateSymlinks()) (no admin/Developer-Mode required);
  verify-worktree beforeAll hookTimeout raised to 30s with retry+backoff
  on every git op; benchmark-v3 speed assertions use vitest retry: 2.
  No production code touched; CI (windows/macos/ubuntu) remains authoritative.

Security

• Escape backslashes before quotes in Mermaid label sanitization (#2):
  CodeQL js/incomplete-sanitization (high) at tools/impact.ts. The
  previous sanitizer escaped quotes but not backslashes, so a name like
  x\"y produced a malformed Mermaid diagram and a potential XSS sink when
  embedded in HTML. Now escapes backslashes first, then quotes
  (mermaidSafeName exported for isolated unit testing).

Dependencies

• Bump prettier 3.9.1 → 3.9.4 (#625).
• Bump runtime, lsp, and @types/node dependency groups (#622, #623, #624).

Chore

• Review rules: memory-leak / output-parity / MCP long-lived checks (#637):
  Extended docs/review-rules.md with P0 patterns (resource leaks incl.
  tree-sitter native handle recreation and caches without TTL; text/JSON
  output data drift; MCP stale-graph leak) and P1 patterns (test pollution
  from tests/; Affected Tests on both impact modes; output placeholder
  strings; summary-counter math; path-traversal vs file-missing distinction).

Upgrade

pi install npm:pi-shazam@latest

Or for MCP clients:

{ "mcpServers": { "pi-shazam": { "command": "npx", "args": ["-y", "-p", "pi-shazam@latest", "pi-shazam-mcp"] } } }

Full Changelog: v0.26.0...v0.27.1