Skip to content

v0.2.5838 — precise lexical codedb_callers results

Latest
Latest

Choose a tag to compare

View all tags
@justrach justrach released this 05 Aug 03:27
v0.2.5838
f58517f

Summary

This release makes codedb_callers distinguish executable lexical code from text that merely looks like a call. Results now exclude whole-word symbol mentions that occur only inside comments, literals, or JavaScript/TypeScript regular expressions, while retaining real references elsewhere on the same source line.

The change fixes #682 and includes the initial trailing-comment correction from #683 plus the broader lexical hardening and regression coverage from #684.

Why this was needed

codedb_callers starts from ranked text-search candidates and then filters those lines. The previous filtering logic could reject full-line comments and quoted-string-only matches, but it still treated a symbol-shaped token in a trailing comment as a call site:

init(); // then renderX() draws the frame

A simple “cut at the first comment marker” fix was not sufficient. Comment-looking bytes can occur inside live syntax, and valid code can follow them on the same line:

url := `https://example`; renderX()
const url = "https://example"; renderX();
#[RenderMetadata]
renderX();

Conversely, skipping marker-containing spans only while looking for a comment boundary could re-admit names found solely inside those spans. Correct filtering therefore required one scanner to decide both where comments begin and whether a whole-word match occurs in lexical code.

What changed

Unified lexical caller matching

Candidate lines now pass through an allocation-free, language-aware scanner before they are returned by codedb_callers. The scanner advances over non-code spans and accepts a result only when the requested name appears as a whole word in lexical code.

The following no longer count as call sites when the symbol exists only in the indicated span:

  • full-line or trailing line comments;
  • single-quoted and double-quoted strings;
  • Go/JavaScript-style backtick text where backticks are literals;
  • Rust raw strings, including hash-delimited forms;
  • complete inline block comments;
  • JavaScript and TypeScript regular-expression literals;
  • JavaScript and TypeScript template text outside ${...} expressions.

This is deliberately a single matching pass rather than a comment-prefix pre-pass followed by a separate string matcher. Using one lexical view prevents the two stages from disagreeing about which bytes are code.

Real code after marker-like text stays visible

The scanner does not truncate a line merely because it encounters //, #, /*, or URL text. It first establishes whether the marker is inside a string, raw literal, backtick literal, regular expression, or completed block comment. A real reference later on that line remains searchable.

Covered examples include:

  • calls after strings containing https://;
  • calls after Go or JavaScript backtick literals containing //;
  • calls after Rust raw strings containing comment-like text;
  • calls after complete inline block comments;
  • PHP code following valid PHP 8 attribute syntax.

Language-specific behavior

Language/syntax Behavior in caller filtering
PHP Recognizes both // and # line comments; keeps #[Attribute] as code rather than treating it as a comment.
Swift Recognizes // comments and configured nested block comments.
Shell Recognizes # comments after whitespace or unescaped operators such as ;; preserves live parameter syntax including ${#var} and $#.
Rust Skips raw string literals, including hash delimiters; supports configured nested block comments.
JavaScript / TypeScript Skips template text and regular-expression literals while recursively scanning code inside ${...} expressions.
R Preserves callable backtick identifiers such as `renderX`(1) instead of treating every backtick span as a string.
OCaml and other configured languages Honors language-appropriate block-comment delimiters and nesting behavior.
HTML-style comment languages Skips complete configured HTML-style comment spans without hiding later code on the same line.

JavaScript and TypeScript regex/template correctness

Template literals need mixed treatment: plain template text is not code, but ${...} is. The scanner therefore ignores the text portion and scans each expression for the requested name.

Regular-expression literals inside those expressions are skipped, including regexes containing braces that must not be mistaken for the end of ${...}. Regex recognition covers expression-leading punctuation and keywords including:

  • return
  • throw
  • yield
  • await

This prevents text such as return /renderX/ from becoming a caller while retaining a real renderX() expression in the same template expression.

Correctness and safety properties

  • Allocation-free hot path: caller-line filtering does not allocate per candidate line.
  • Bounded recursion: nested template-expression scanning is capped at 16 levels, preventing unbounded stack growth and hostile-input rescanning.
  • Conservative malformed-input handling: unterminated or incomplete lexical spans are handled without scanning beyond the isolated line.
  • Whole-word semantics preserved: the scanner changes where a match is eligible, not the existing identifier-boundary rule.
  • Definition filtering preserved: lines corresponding to known symbol definitions continue to be excluded before results are returned.
  • Output contract unchanged: the MCP tool name, arguments, result formatting, and max_results behavior are unchanged.

Verification

The release candidate passed all repository release gates:

  • seven focused issue-#682 regression groups;
  • the complete zig build test suite;
  • MCP end-to-end verification: 20/20 assertions across the root-handshake regression, normal explicit-root mode, and no-roots-client mode;
  • zig build bench locally;
  • GitHub’s paired benchmark regression gate, including the repository’s 10% regression threshold;
  • Zig formatting/lint checks and git diff --check.

Regression coverage proves both sides of the change: comment/literal/regex-only mentions are removed, and real references are not lost when comment-like bytes occur earlier on the same line.

Intentional scope and limitations

codedb_callers remains a fast lexical heuristic over ranked, isolated source lines; it is not a full parser or semantic call graph. A whole-word symbol reference in lexical code can still be reported even when the expression is not a function invocation.

Because ranked search supplies individual lines, the scanner intentionally does not carry comment, string, or template state forward from earlier lines. A multiline construct opened on a previous line is therefore outside this fix’s scope. Template-expression nesting beyond 16 levels is treated conservatively rather than recursed without bound.

These constraints preserve the tool’s cross-language coverage, allocation profile, and benchmark-critical search path while fixing the concrete false positives without introducing per-language parsers into caller ranking.

Upgrade impact

  • No configuration changes are required.
  • No MCP schema or tool-call changes are required.
  • Existing clients receive more precise codedb_callers results after replacing the binary.
  • Consumers that depended on comment-, literal-, or regex-only mentions being returned should treat their removal as the intended bug fix.

Signed and notarized assets

Both macOS binaries are signed with the project’s Developer ID Application certificate using the hardened runtime and a secure timestamp. Each architecture-specific archive was submitted to Apple through the codedb-notary keychain profile and returned Accepted before publication.

Notarization evidence:

  • macOS arm64 — submission 4a3e4dba-c43a-40e9-9d3b-360fc4379661, Accepted
  • macOS x86_64 — submission 9d932146-c4ee-413f-8656-5a699ea4aef2, Accepted

The .notary.zip files are the exact root-level archives submitted to Apple. The bare codedb-darwin-* binaries are byte-identical to the binaries inside those accepted archives. The downloaded standalone binaries also pass strict codesign verification and report codedb 0.2.5838 on both architectures.

Build provenance and artifact integrity

5ba0542e5f2bb1d50165a33f822485944ba3f9c099474230b57e713904fb9847  codedb-darwin-arm64
8b25c95b553199f20e9ad0edeab6e5c21ecc03b06354482684d08c1dda1e121a  codedb-darwin-arm64.notary.zip
af63fc885aec91b68b1b5525dcd9a5c5bc4fb28cb55193c12a37486749f4a1a8  codedb-darwin-x86_64.notary.zip
b69e27dcfc6a9b79a6743ef07d1e2e3a31cd711d6a6ba31c7893a28f994875e9  codedb-linux-arm64
c7cff9a6f3834ef2e0836c28c237c2ab3dd271c694f368701d22cc503cf243d1  codedb-windows-x86_64.exe
dbf2e4f4c061c910a1ff2228e99317015d86a94e76b4b0f26b6f2f19f81f3d81  codedb-linux-x86_64
e892bd5839cb6a015b4eced7151b30b81241864c34c05156f38e091ce18947ca  codedb-darwin-x86_64

Links

Assets 10
Loading