Skip to content

ast-bro v4.2.0

Latest

Choose a tag to compare

@aeroxy aeroxy released this 08 Aug 07:31
· 25 commits to main since this release

1. High-Level Summary (TL;DR)

  • Impact: High - Substantially improves Go module dependency resolution by properly supporting multi-module workspaces, nested modules, and root-level package imports.
  • Key Changes:
    • Multi-Module Support: Removed the limitation of a single root go.mod. The indexer now discovers and parses every go.mod in the repository during the file walk.
    • 🧠 Smarter Resolution: Re-implemented Go import resolution to evaluate multiple module candidates, ranking them by enclosing directory depth, prefix length, and path closeness.
    • 🧹 Struct Cleanup: Removed alias_prefix and go_module from the crate-internal resolution contexts, shifting Go module tracking directly into the SuffixIndex.
    • E2E Testing & CI: Added a comprehensive suite of tests for complex Go module layouts and introduced a GitHub Actions workflow (.github/workflows/tests.yml) to run tests across macOS and Ubuntu.

2. Visual Overview (Code & Logic Map)

The following diagram illustrates how the new Go module resolution handles discovery and candidate ranking.

graph TD
    subgraph "Index Building (src/deps/resolver/build.rs)"
        walk["Walk Directory Tree"]
        parse["parse_go_module(path)"]
        store["Store in SuffixIndex.go_modules"]
        
        walk -->|"find go.mod"| parse
        parse --> store
    end

    subgraph "Dependency Resolution (src/deps/resolver/resolve.rs)"
        import["Import 'mymod/pkg/foo'"]
        candidates["Filter by matching prefix"]
        rank["Rank Candidates: Enclosing module > Prefix length > Closeness"]
        pick["find_dir_file(idx, key)"]
        
        import --> candidates
        candidates -->|"Evaluate context"| rank
        rank --> pick
    end
    
    store -. "Used for lookup" .-> candidates

3. Detailed Change Analysis

🏗️ Go Module Indexing (src/deps/resolver/build.rs, src/deps/manifest.rs)

  • What Changed: Instead of only looking at the repository root for a go.mod file, the build_suffix_index() function now checks for go.mod files during its full repository walk. When found, it extracts the module prefix and stores the tuple (module_path, module_directory) in a new vector SuffixIndex.go_modules. The list is pre-sorted longest-module-path first.

🧠 Import Resolution (src/deps/resolver/resolve.rs)

  • What Changed: The Go import resolution logic (resolve()) was completely rewritten. When evaluating an import like example.com/multi/tools/helper, it evaluates all known modules that match the prefix. It ranks candidates based on three strict criteria:
    1. Deepest Enclosing Directory: The module directory that actually contains the importer file wins first.
    2. Longest Module Path: If the importer is outside, nested modules win over their enclosing modules.
    3. Closeness: If there are equal prefixes (e.g., duplicated modules or vendored copies), it picks the one closest to the importer.
  • It also adds support for resolving the module's root package (e.g., import "example.com/rootpkg" where the code lives directly next to go.mod).

📦 Struct & Context Refactoring

  • What Changed: With Go modules now handled dynamically via SuffixIndex, the global resolution contexts no longer need to carry a single Go alias.
Struct / Context Removed Field Added Field Reason
ProjectAliases go_module None Go modules are dynamically collected during the index walk.
ResolveCtx alias_prefix None Prefix stripping is handled directly inside resolve.rs.
SuffixIndex None go_modules Stores all discovered Go module mappings (prefix, dir).

🧪 Testing & CI (tests/deps_e2e.rs, .github/workflows/tests.yml)

  • What Changed: Added a robust set of E2E tests validating edge cases: nested modules winning over enclosing ones, duplicate module resolution (e.g., vendored copies), and sibling module conflicts. Added a new GitHub Actions workflow to ensure tests run automatically on main pushes and PRs.

4. Impact & Risk Assessment

  • No breaking changes. ResolveCtx and ProjectAliases did drop their Go-specific fields, but neither type is reachable from outside the crate: src/lib.rs declares every module privately (mod deps;), so the pub on deps::resolver and deps::manifest grants crate-internal visibility only. The library's entire public surface is LineRange and run(). Confirmed by compiling an integration test — which links the lib exactly as a downstream consumer would — against both types:

    error[E0603]: module `deps` is private
      --> src/lib.rs:10:1
    

    No downstream code can have depended on these fields, so nothing external needs to adapt. This is a crate-internal refactor.

  • ⚠️ Behavioural change for Go repositories. This is the real compatibility surface. A repository whose go.mod sits below the root previously had every Go import bucketed as external; it now resolves. That silently changes the output of deps, reverse-deps, graph, cycles, impact and callers for an entire language. The new answers are the correct ones, but anyone diffing output across versions should expect movement.

  • 📊 Performance: unchanged in the common case, materially worse in a pathological one. Measured on release builds, three runs each:

    Shape Before After
    1 module, 1500 files, 4500 imports 0.19 s 0.19 s
    11 prefix-nested modules, every candidate missing 0.09 s 0.58 s

    The cost is not the sorting — candidate lists are 0–2 entries for a typical import, and most imports (stdlib, third-party) match no module at all. It is find_dir_file, which linearly scans every entry of idx.by_file on each call; this change multiplies the number of calls by the count of prefix-matching modules. The scan itself predates this work. Single-module repositories — the overwhelming majority — are unaffected.

  • 📌 Known follow-ups, both pre-existing and neither introduced here:

    • Manifest files (go.mod, tsconfig.json, Cargo.toml, composer.json) are resolution inputs but are not tracked by compute_delta, because is_indexable() is extension-based. Editing one leaves a stale graph until --rebuild. Reproducible on the previous release.
    • find_dir_file's O(all-files) scan wants a directory → representative-file map built during the walk, which would retire both the scan and the multiplier above.