Skip to content

[Bug] codegraph_explore blast radius does not traverse Test_*.bas / Test_*.cls files — ⚠️ no covering tests is misleading #78

Description

@ardelperal

codegraph_explore blast radius does not traverse Test_*.bas / Test_*.cls files — ⚠️ no covering tests found is misleading

Summary

codegraph_explore's blast-radius surface reports ⚠️ no covering tests found for a symbol whenever the symbol has no callers in the same file as its declaration. When the only callers / users of a symbol live in sibling Test_*.bas / Test_*.cls files (the standard convention in every supported language that has a test-file convention), the warning fires even when the symbol is fully covered. Every AI agent auditing code coverage with this tool produces systematic false-positive gap reports.

In a recent audit of ardelperal/VBA_TOOLKIT_BENCH, 3 out of 8 reported coverage gaps were false positives of this kind:

  • EnumPublicabilidadVeredicto enum (PublicabilidadEdicion.bas:415) — actually covered by Test_PublicabilidadRiesgo_JustificacionPC.bas and Test_RiesgoMaterializacionContingencia.bas
  • tPublicabilidadRiesgoDatos UDT (PublicabilidadEdicion.bas:413) — covered by Test_PublicabilidadRiesgo_JustificacionPC.bas:67, 169
  • tbCambiosParaPublicacion table reference — covered by Test_InformePublicacionHelper.bas (1501-line test file with 11 atoms)

The blast radius is the only surface AI agents use to navigate test coverage; making it aware of test-file references is what turns the tool from "useful for unit tests only" into "useful for full audit."

Scope (what to touch)

File Change
src/core/services/blast-radius.ts (or equivalent — locate the file that emits the per-symbol N callers in <file> / ⚠️ no covering tests found lines) Make the blast-radius callee-finder traverse Test_*.bas, Test_*.cls, Test_*.form.txt, Test_*.report.txt files in addition to the same file.
src/mcp/tool-contracts.ts (or equivalent) Document in the codegraph_explore tool description that blast radius includes cross-file test references; expose a blastRadiusTraverseTests: boolean (default true).
src/adapters/config/indexing-config.ts (or equivalent) Add the new boolean to the indexing config schema; document the default.
test/core/services/blast-radius.test.ts (or equivalent) Add the test cases below.

Non-goals (do NOT touch)

  • Do NOT change codegraph_explore semantics for non-blast-radius outputs (the symbol body, the source-code verbatim block) — those already cross files correctly.
  • Do NOT change the indexed file set — the indexer already includes Test_*.bas and Test_*.cls in the project tree. The change is in how the blast radius queries the indexed facts, not in indexing itself.
  • Do NOT silently flip the default if the operator has indexed a non-test project where Test_*.bas files happen to exist as production code with that prefix — let the default true ride, but the opt-out knob is mandatory.

Repro

Setup: a multi-module project where the indexer has indexed all files including Test_*.bas files (the standard for ardelperal/VBA_TOOLKIT_BENCH and most non-trivial downstream forks).

# 1. Pick an enum declared in module A:
#    Public Enum X
#        Foo = 1
#        Bar = 2
#    End Enum

# 2. The same enum is used in Test_HelperForA.bas and Test_Other.bas —
#    NOT referenced elsewhere in module A.

# 3. Run via MCP:
codegraph_explore({ query: "X" })

# 4. Output's blast-radius section reads:
#    **Blast radius — what depends on these (update/verify before editing)**
#    - `X` (src/modules/A.bas:42)
#       — 1 caller in `src/modules/A.bas`; ⚠️ no covering tests found
#
#    # Test_HelperForA.bas and Test_Other.bas are NEVER shown.

# Expected: the blast radius reports the cross-file callers including
#    the test files; the warning `⚠️ no covering tests found` is suppressed
#    when the Test_*.bas references exist.

# Actual: the blast radius scopes to in-file usage only; the warning fires
#    even when the symbol has cross-file test coverage.

Root cause

src/core/services/blast-radius.ts (or the file that owns the per-symbol "what depends on these" section of codegraph_explore):

Current behavior: the blast-radius resolver walks edges in the symbol-index graph but filters "callers" by file scope = the same file as the symbol declaration. This was likely a deliberate performance/clarity decision early in the project, but it misses the convention that agents and humans treating blast radius as a coverage signal expect: any reference path counts as a dependency, including reads from a Test_*.bas file.

The graph itself is correct (the edges exist in the indexed fact store); the resolver is just scoping the projection wrong.

Suggested implementation

In src/core/services/blast-radius.ts (or equivalent), make the scope configurable per-call and have it include test files by default:

export interface BlastRadiusOptions {
  traverseTestFiles?: boolean;  // default true
  includeFiles?: string[];      // optional opt-out: only these files
  excludeFiles?: string[];      // optional opt-out: never these files
  maxCallersPerSymbol?: number; // existing knob, keep
}

export async function buildBlastRadius(
  symbolId: SymbolId,
  index: ProjectIndex,
  options: BlastRadiusOptions = { traverseTestFiles: true, maxCallersPerSymbol: 10 }
): Promise<BlastRadiusResult> {

  const allCallers = await index.callersOf(symbolId);

  const isTestFile = (p: string) =>
    /(^|\/)Test_[^/]+\.(bas|cls|form\.txt|report\.txt)$/i.test(p);

  const visibleCallers = allCallers.filter(caller => {
    if (options.excludeFiles?.some(re => re.test(caller.file))) return false;
    if (options.includeFiles && !options.includeFiles.some(re => re.test(caller.file))) return false;
    if (!options.traverseTestFiles && isTestFile(caller.file)) return false;
    return true;
  });

  const coveringTests = visibleCallers.filter(c => isTestFile(c.file));
  const otherCallers  = visibleCallers.filter(c => !isTestFile(c.file));

  return {
    symbolId,
    coveringTestCount: coveringTests.length,
    coveringTests: coveringTests.map(c => ({ file: c.file, line: c.line })),
    otherCallerCount: otherCallers.length,
    otherCallers: otherCallers.slice(0, options.maxCallersPerSymbol ?? 10),
    warning: otherCallers.length === 0 && coveringTests.length === 0
      ? '⚠️ no references found anywhere'
      : (coveringTests.length === 0
          ? '⚠️ no covering tests found'
          : null),
  };
}

In the tool description string for codegraph_explore (in src/mcp/tool-contracts.ts or equivalent), document the new opt-out knob explicitly.

In src/adapters/config/indexing-config.ts (or equivalent), add:

explore?: {
  blastRadiusTraverseTests?: boolean; // default true
};

Acceptance criteria

  1. Red → green test in test/core/services/blast-radius.test.ts:

    • With a fixture project containing 1 Test_*.bas file that uses the target symbol:
      • Before fix: blast radius reports ⚠️ no covering tests found.
      • After fix: blast radius reports the test references, suppresses the warning, and the response includes a coveringTests: [{ file, line }] array.
    • With traverseTestFiles: false (passed in the call), behavior reverts to current (warning shown if no in-file callers).
    • With a fixture project that has zero Test_*.bas references, the warning fires as today.
  2. Operator repro on ardelperal/VBA_TOOLKIT_BENCH:

    • Run codegraph_explore({ query: "EnumPublicabilidadVeredicto" }) and codegraph_explore({ query: "tPublicabilidadRiesgoDatos" }).
    • Verify that Test_PublicabilidadRiesgo_JustificacionPC.bas and Test_RiesgoMaterializacionContingencia.bas appear in the blast radius.
    • Verify that the ⚠️ no covering tests found warning is gone.
  3. CI: pnpm test is green. The existing tests for in-file-only blast radius behavior stay green.

  4. Documentation note added to CHANGELOG.md under the upcoming minor release, noting:

    • "blast radius now traverses Test_*.bas / Test_*.cls / Test_*.form.txt / Test_*.report.txt by default; opt out via the per-call traverseTestFiles: false knob or the project-level explore.blastRadiusTraverseTests: false config."
    • "the ⚠️ no covering tests found warning now considers cross-file test references as coverage."

Notes

A related issue was opened on the downstream fork ardelperal/codegraph-vba#28 with a VBA-specific framing for the same root cause. The fix lives in the upstream tool (this repo) and the fork will pick it up on next rebase.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workingstatus:approvedApproved for implementation

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions