Skip to content

fix(vba-preprocess): reassemble multi-line _ continuations so wrapped statements aren't dropped #81

Description

@ardelperal

Context

The VBA extraction pipeline preprocesses source before the regex sweeps run. Order (src/extraction/vba-extractor.ts:125-127):

const joined = joinLineContinuations(this.source);
const preprocessed = preprocessConditionalCompilation(joined);
const uncommented = stripVbaComments(preprocessed);

Problem

joinLineContinuations (src/extraction/vba-preprocess.ts:54-64) does not actually join continuation lines. Its regex

return src.replace(/ _(?:(\r?\n)|$)/g, (_m, nl) => ' ' + (nl ?? ''));

replaces _ with a single space but preserves the newline (deliberately, to keep line-count parity so downstream startLine = i + 1 math stays correct — see the doc comment at :56-60). The consequence: a VBA statement split across multiple physical lines with the _ continuation is left as separate physical lines, and the regex sweeps only ever see its first fragment.

This is acknowledged as a known gap: src/extraction/vba-extractor.ts:2807"Cross-physical-line concat via _ continuation is OUT OF SCOPE."

Impact

Any symbol or call whose defining tokens spill past the first physical line is under-parsed or missed:

Public Declare PtrSafe Function GetTickCount _
    Lib "kernel32" () As Long          ' Lib/alias metadata on line 2 is not seen

modUtils.RegistrarEvento _
    strUsuario, strAccion              ' the call args on line 2 are not seen

Public Sub Procesar( _
    ByVal a As Long, _
    ByVal b As Long)                   ' params on continuation lines are not seen

Machine-generated Dysflow exports rarely wrap, so this is low-frequency there — but hand-written VBA that wraps long signatures/calls/SQL loses graph coverage silently, which defeats impact analysis on exactly the kind of code humans write.

Secondary: the function name joinLineContinuations is misleading — it strips, it does not join.

Proposed Approach

Implement real logical-line reassembly while preserving accurate line numbers:

  1. Build a logical source where a line ending in _ is concatenated with the following physical line(s) into one logical line.
  2. Maintain a line map: logical-line index → the physical startLine of its first fragment, so every node/edge startLine still points at the original physical line the statement began on.
  3. Feed the sweeps the logical lines but resolve startLine through the map.
  4. Keep string-literal safety: a _ that is genuinely the last non-space chars of a physical line is a continuation in VBA regardless of string state (documented at :48-52), so the existing string-agnostic detection is acceptable — but add a test asserting a string that ends ..._" (quote after underscore) is NOT touched.
  5. Rename joinLineContinuations → something honest, or make it actually join. If renamed, update vba-extractor.ts and __tests__/extraction-vba-preprocess.test.ts.

Acceptance Criteria

  • A multi-line Declare reassembles: Lib/alias/PtrSafe metadata is captured from continuation lines.
  • A multi-line cross-module call (modUtils.Foo _ \n arg1, arg2) resolves to the same edge as its single-line form.
  • A multi-line Sub/Function signature captures all parameters.
  • Every node emitted from a reassembled statement still reports the physical startLine of its first fragment (regression-guard the line numbers explicitly).
  • Existing extraction-vba-preprocess.test.ts stays green (or is updated only for the rename).
  • New tests in __tests__/extraction-vba-preprocess.test.ts cover: continuation in a Declare, a call, a Sub signature, and the string-ending-in-_" non-case.

Testing

npm run build
npx vitest run __tests__/extraction-vba-preprocess.test.ts __tests__/extraction-vba.test.ts
npm test

Out of Scope

  • Concatenating string literals split across lines for SQL-table extraction beyond what reassembly naturally enables (can be a follow-up).

Metadata

Metadata

Assignees

No one assigned

    Labels

    area:vbaVBA/Access-specific work (parent codegraph product)bugSomething isn't workingstatus:approvedApproved for implementation

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions