Skip to content

feat(config): restore path aliases with TS 6.x-compatible subpath imports#672

Merged
carlos-alm merged 6 commits intomainfrom
feat/dynamic-sql-edge-cases
Mar 29, 2026
Merged

feat(config): restore path aliases with TS 6.x-compatible subpath imports#672
carlos-alm merged 6 commits intomainfrom
feat/dynamic-sql-edge-cases

Conversation

@carlos-alm
Copy link
Copy Markdown
Contributor

@carlos-alm carlos-alm commented Mar 29, 2026

Summary

  • Replace deprecated baseUrl/paths with Node.js subpath imports (package.json imports field) + customConditions in tsconfig.json
  • Uses @codegraph/source condition for compile-time resolution to src/, default condition for runtime resolution to dist/
  • Migrates 3 deeply nested files as proof-of-concept; remaining 300+ imports can be migrated incrementally

Details

How it works:

Code:       import { logger } from '#infrastructure/logger.js';
TypeScript: resolves → src/infrastructure/logger.ts  (via @codegraph/source condition)
Node.js:    resolves → dist/infrastructure/logger.js  (via default condition)

Changes:

  • package.jsonimports field with conditional mappings for all 11 src/ subdirectories + #types
  • tsconfig.jsoncustomConditions: ["@codegraph/source"]
  • vitest.config.tsresolve.conditions: ['@codegraph/source'] for Vite resolver
  • scripts/verify-imports.ts — resolves #-prefixed specifiers through the imports map
  • 3 proof-of-concept files migrated (build-structure.ts, build-edges.ts, collect-files.ts)

Test plan

  • npx tsc --noEmit — zero errors
  • npm run build# specifiers preserved in compiled JS
  • node dist/cli.js --help — runtime resolves via default condition
  • npm run verify-imports — 281 files pass
  • npm test — 2128 pass (1 pre-existing failure unrelated to this PR)
  • biome check — clean on migrated files

Closes #668

Add queryAll/queryGet methods to NativeDatabase for parameterized SELECT
execution via rusqlite, returning rows as serde_json::Value objects.
This enables NodeQuery and arbitrary SQL to dispatch through the native
engine without porting the SQL builder to Rust.

- NativeDatabase: queryAll, queryGet, validateSchemaVersion methods
- NodeQuery: all()/get() accept optional nativeDb for native dispatch
- connection.ts: closeDbPair/closeDbPairDeferred lifecycle helpers
- finalize.ts/pipeline.ts: use unified close helpers
- detect-changes.ts, build-structure.ts: 3 starter straggler migrations
- Comprehensive db.prepare() audit (194 calls, tiered migration plan)
- 42 new tests (parity, query execution, version check; skip-guarded)
@claude
Copy link
Copy Markdown

claude bot commented Mar 29, 2026

Claude finished @carlos-alm's task —— View job


Reviewing PR: Generic Query Execution on NativeDatabase

Conducting comprehensive 5-phase review of this performance improvement PR.

Todo List

  • Phase 1: Root cause analysis - understand the problem being solved
  • Phase 2: Configuration change detection - check for high-risk changes
  • Phase 3: Test integrity - validate test quality and coverage
  • Phase 4: Code quality - review implementation details
  • Phase 5: Backlog compliance - assess against foundation principles
  • Final assessment with rating and recommendation

Root Cause Analysis

Problem Being Solved: This PR addresses performance bottlenecks in the dual-engine architecture by adding generic query execution capabilities to the native Rust backend. Currently, JavaScript builds SQL queries but executes them through the slower better-sqlite3 path even when NativeDatabase is available.

Why This Approach: The PR enables the existing NodeQuery builder to dispatch SELECT queries to Rust when nativeDb is available, maintaining the same SQL generation logic while leveraging faster native execution. This is a logical progression after Phase 6.15 (bulk writes) to now optimize the read path.

Risk Assessment: Low risk - purely additive functionality with comprehensive test coverage and fallback to existing behavior when native DB is unavailable.

Configuration Change Detection

No high-risk configuration changes detected. The PR:

  • Does not modify any linting rules or build configuration
  • Does not change test timeouts or coverage thresholds
  • Does not disable TypeScript strict mode flags
  • Only adds new functionality without relaxing existing safeguards

Test Integrity

Strong test coverage with proper safeguards:

  • 42 new tests across 3 comprehensive test files
  • Skip-guarded for native binary availability - tests only run when the native addon is present
  • Engine parity validation - query-builder-native-parity.test.ts uses combinatorial testing to verify identical results between JS and Rust execution
  • Parameter type coverage - tests string, number, null parameters with proper edge cases
  • Error handling - validates SQL errors and closed database exceptions
  • Normalization logic - handles potential BigInt differences between engines (line 23-31 in parity tests)

The test quality is excellent with proper isolation, comprehensive edge case coverage, and rigorous parity validation.

Code Quality

Rust Implementation (native_db.rs):

  • Secure parameter handling - json_to_rusqlite_params properly validates and converts parameter types (lines 796-821)
  • SQL injection protection - Parameters are properly bound via rusqlite's prepared statements
  • Error propagation - Comprehensive error handling with descriptive messages
  • Memory safety - Proper column name collection and row-to-JSON conversion
  • ⚠️ Minor concern: The validateSchemaVersion method prints directly to stderr (line 636-639) rather than using structured logging

TypeScript Integration (query-builder.ts):

  • Backward compatible - Optional nativeDb parameter preserves existing API
  • Type safety - Proper generic type handling for return types
  • Fallback logic - Clean conditional dispatch: nativeDb ? native_path : js_path
  • Parameter casting - Appropriate type assertion for parameter arrays

Connection Management (connection.ts):

  • Lifecycle helpers - closeDbPair/closeDbPairDeferred properly sequence native → better-sqlite3 close operations
  • Performance optimization - Deferred WAL checkpoints for incremental builds

Critical Architecture Compliance:

  • Dual-engine parity: The comprehensive parity tests ensure both engines produce identical results, which is a core requirement per CLAUDE.md
  • Schema migration sync: Migration DDL is properly mirrored between TypeScript and Rust (noted in comments)

Backlog Compliance

Zero-dep: ✅ - No new runtime dependencies added, builds on existing rusqlite foundation

Foundation-aligned: ✅ - Fully compliant with all principles:

  • Maintains local-only operation (P1)
  • Improves performance without external dependencies (P4)
  • Preserves embeddable-first API (P5)
  • Does not add agent-like behavior (P8)

Problem-fit: 4/5 - Directly addresses AI token waste and improves query performance for large codebases. While not solving the core "lost AI" navigation problem directly, faster queries enable more responsive AI assistance and reduce round-trip latency.

Breaking: No - Purely additive with backward-compatible optional parameters

Tier: 1 (zero-dep ✓ + foundation-aligned ✓)

Critical Concerns

Minor Issues:

  1. Stderr logging in Rust: validateSchemaVersion uses eprintln! instead of structured logging
  2. Migration coverage gap: The audit document shows 194 db.prepare() calls still need migration, but this is acknowledged as future work

Strengths:

  • Excellent test coverage with rigorous parity validation
  • Clean architectural design with proper separation of concerns
  • Comprehensive documentation and migration strategy
  • Strong error handling and parameter validation

Final Recommendation

  • Rating: ⭐⭐⭐⭐☆ (4/5)
  • Action: APPROVE WITH CONDITIONS
  • Reasoning: This is a well-engineered performance improvement that follows the project's architectural principles. The comprehensive test coverage, especially the engine parity validation, addresses the critical requirement that both engines produce identical results. The migration audit provides a clear roadmap for future work. The minor logging concern doesn't impact functionality and can be addressed in a follow-up.

Conditions for approval:

  1. Consider addressing the stderr logging in validateSchemaVersion to use structured logging in a follow-up PR
  2. Ensure CI validates that all new tests pass with the native binary

@greptile-apps
Copy link
Copy Markdown
Contributor

greptile-apps bot commented Mar 29, 2026

Greptile Summary

This PR does two things under one commit: (1) migrates path aliases from tsconfig.json baseUrl/paths to Node.js subpath imports (package.json imports field + customConditions) for TS 5.2+ / TS 6.x compatibility, and (2) delivers Phase 6.16 — generic parameterized query execution on NativeDatabase (query_all, query_get, validate_schema_version in Rust/NAPI), connection lifecycle helpers (closeDbPair/closeDbPairDeferred), and NodeQuery native dispatch. Three deeply-nested pipeline files are migrated as a path-alias proof-of-concept; the remaining ~300 imports are tracked for incremental follow-up.

Key changes:

  • package.jsonimports field with @codegraph/source / default conditional mappings for all 11 src/ subdirectories + #types
  • tsconfig.json / vitest.config.tscustomConditions: [\"@codegraph/source\"] for compile-time and test-time resolution
  • crates/codegraph-core/src/native_db.rsquery_all, query_get, validate_schema_version NAPI methods; json_to_rusqlite_params / row_to_json helpers
  • src/db/query-builder.tsNodeQuery.all() / .get() now dispatch through rusqlite when nativeDb is available
  • src/db/connection.tscloseDbPair / closeDbPairDeferred ensure NativeDatabase closes before the WAL-checkpointing better-sqlite3 handle
  • src/domain/graph/builder/stages/detect-changes.tshasTable semantics correctly restored (prior regression fix) + full nativeDb dispatch
  • 3 new unit test files with combinatorial parity coverage across all filter/JOIN/ORDER BY combinations
  • docs/migration/db-prepare-audit.md — tiered migration plan for remaining 194 db.prepare() calls

Note: The PR title and description focus entirely on path aliases, but the Phase 6.16 NativeDatabase work is the larger portion of the diff. Future PRs may benefit from separating infrastructure changes from configuration changes to simplify review scope.

Confidence Score: 5/5

Safe to merge — no P0/P1 issues found; all remaining findings are P2 style suggestions.

The prior regression in detect-changes.ts (hasTable semantics) is correctly fixed. The Rust query methods are well-tested with a combinatorial parity suite. Connection lifecycle ordering is correctly enforced. The subpath imports setup is validated by npx tsc --noEmit, the full test suite (2128 pass), and verify-imports. All open findings are P2: a minor null-safety note on a COUNT(*) cast that can never fail in practice, a silent cell-error mapping in Rust, and a future-proofing note about the #types exact pattern.

No files require special attention. crates/codegraph-core/src/native_db.rs and src/domain/graph/builder/stages/build-structure.ts have the minor P2 observations noted above.

Important Files Changed

Filename Overview
crates/codegraph-core/src/native_db.rs Adds query_all, query_get, and validate_schema_version NAPI methods plus json_to_rusqlite_params / row_to_json helpers. Logic is sound; only minor concern is silent null on cell read errors in row_to_json.
package.json Adds imports field with conditional subpath mappings for all 11 src/ subdirectories and #types. Patterns are well-formed; #types is intentionally exact-match only.
src/db/query-builder.ts Adds optional nativeDb parameter to all() and get(), dispatching to rusqlite when available. validateNativeParams guard is correct at runtime; the null check inside a (string
src/db/connection.ts Adds LockedDatabasePair, closeDbPair, and closeDbPairDeferred — correctly closes NativeDatabase first (fast), then better-sqlite3 (WAL checkpoint). Error from nativeDb.close() is swallowed intentionally.
src/domain/graph/builder/stages/detect-changes.ts Correctly threads nativeDb through getChangedFiles for both the file_hashes existence probe and the full hash-map read. The hasTable = true semantics fix from the prior review is properly restored.
src/domain/graph/builder/stages/build-structure.ts Migrates imports to #-prefixed aliases and adds nativeDb dispatch for the file-count query. The queryGet result is cast without null guard (minor pre-existing pattern).
src/domain/graph/builder/pipeline.ts Replaces manual dual-close error path with closeDbPair; correctly preserves the ctx.earlyExit and ctx.db guards.
scripts/verify-imports.ts Adds resolveSubpathImport to handle #-prefixed specifiers via the package.json imports map. Handles exact and single-wildcard patterns correctly; .ts fallback for .js-resolved paths works well.
tsconfig.json Adds customConditions: ["@codegraph/source"] — the correct TS 5.2+ mechanism for resolving package.json conditional exports/imports at compile time.
vitest.config.ts Adds resolve.conditions: ['@codegraph/source']. Vite treats this as additive to its default conditions, so standard node/import/default resolution is unaffected.
tests/unit/query-builder-native-parity.test.ts Comprehensive combinatorial parity test covering all filter combinations, JOINs, ORDER BY, LIMIT, and queryGet across both engines. normalizeRows handles BigInt differences correctly.
src/types.ts Adds queryAll, queryGet, and validateSchemaVersion to the NativeDatabase interface, matching the new Rust NAPI exports exactly.

Sequence Diagram

sequenceDiagram
    participant Caller as Pipeline Stage
    participant NQ as NodeQuery
    participant Native as NativeDatabase (Rust/NAPI)
    participant SQLite as better-sqlite3

    Caller->>NQ: .all(db, nativeDb?) / .get(db, nativeDb?)
    NQ->>NQ: build() → { sql, params }

    alt nativeDb available
        NQ->>NQ: validateNativeParams(params)
        NQ->>Native: queryAll(sql, params) / queryGet(sql, params)
        Native->>Native: json_to_rusqlite_params()
        Native->>Native: stmt.query_map() / query_rows.next()
        Native->>Native: row_to_json() per row
        Native-->>NQ: Vec<serde_json::Value> / Option<Value>
        NQ-->>Caller: TRow[] / TRow | undefined
    else JS fallback
        NQ->>SQLite: db.prepare(sql).all(...params)
        SQLite-->>NQ: TRow[]
        NQ-->>Caller: TRow[]
    end

    note over Caller,SQLite: closeDbPair() ensures NativeDatabase.close() before better-sqlite3.close() (WAL checkpoint ordering)
Loading

Reviews (2): Last reviewed commit: "Merge remote-tracking branch 'origin/fea..." | Re-trigger Greptile

Comment on lines 63 to 70
let hasTable = false;
try {
db.prepare('SELECT 1 FROM file_hashes LIMIT 1').get();
hasTable = true;
const probe = nativeDb
? nativeDb.queryGet('SELECT 1 FROM file_hashes LIMIT 1', [])
: db.prepare('SELECT 1 FROM file_hashes LIMIT 1').get();
if (probe) hasTable = true;
} catch {
/* table doesn't exist */
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 hasTable probe semantics changed for JS fallback path

The old JS-only code set hasTable = true if the prepare/get call didn't throw — regardless of whether any rows were returned:

// Old:
db.prepare('SELECT 1 FROM file_hashes LIMIT 1').get();
hasTable = true;

The new code adds an existence check (if (probe)), so an empty file_hashes table now sets hasTable = false on both the native and the JS fallback paths:

const probe = nativeDb
  ? nativeDb.queryGet(...)
  : db.prepare('SELECT 1 FROM file_hashes LIMIT 1').get();
if (probe) hasTable = true;   // returns undefined for empty table on JS path

In practice this makes the two paths behave consistently and triggers isFullBuild: true for an empty hash table, which is arguably more correct. However the JS fallback path now has different semantics than the JS-only code it replaced, which could be surprising to anyone relying on the old "table exists = incremental" contract. A brief comment noting the intentional behaviour change would prevent future confusion.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed -- this was actually a regression, not an intentional behavior change. The old JS-only code set hasTable = true unconditionally when the query succeeded (table exists), regardless of row count. The new code added if (probe) which broke the empty-table case, causing isFullBuild: true instead of incremental.

Restored the original semantics: the try/catch now runs the probe query for its side-effect (throw if table missing), and sets hasTable = true unconditionally on success. Both the nativeDb and JS fallback paths now correctly treat an existing-but-empty file_hashes table as incremental (not full build). This was the CI failure root cause.

…orts

Replace deprecated baseUrl/paths with Node.js subpath imports (package.json
imports field) + customConditions in tsconfig.json. Uses @codegraph/source
condition for compile-time resolution to src/, default condition for runtime
resolution to dist/.

- Add imports field with conditional mappings for all 11 src/ subdirectories + #types
- Add customConditions: ["@codegraph/source"] to tsconfig.json
- Add resolve.conditions to vitest.config.ts for Vite resolver
- Update verify-imports.ts to resolve #-prefixed specifiers
- Migrate 3 deeply nested files as proof-of-concept

Closes #668
@carlos-alm carlos-alm changed the title perf(db): generic query execution on NativeDatabase (6.16) feat(config): restore path aliases with TS 6.x-compatible subpath imports Mar 29, 2026
#672)

The nativeDb migration changed the file_hashes probe from unconditionally
setting hasTable=true on successful query to only setting it when the
query returns a row. This caused an empty file_hashes table (exists but
no rows) to be treated as a full build instead of incremental, breaking
the detect-changes test.
- Document row_to_json BLOB/error-to-null contract in Rust doc comment
- Clarify queryAll/queryGet doc comments re: SELECT-only intent
- Add validateNativeParams runtime check before dispatching to nativeDb
@carlos-alm
Copy link
Copy Markdown
Contributor Author

Addressed all Greptile findings:

  1. BLOB/cell-error in row_to_json: Added doc comment to row_to_json explicitly documenting the contract -- BLOB columns map to null because the codegraph schema has no BLOB columns, and cell-level read errors map to null to avoid partial-row failures.

  2. Unsafe params cast in query-builder.ts: Added validateNativeParams() runtime check that validates every param is string, number, or null before dispatching to nativeDb. Throws a DbError with the offending param index and type on violation.

  3. queryAll/queryGet accept arbitrary SQL: Updated doc comments on both Rust methods to clarify they are designed for SELECT statements and note that DML/DDL is not an intended use. Current callers all pass SELECT-only SQL from NodeQuery.build().

  4. hasTable probe semantics: This was actually a regression, not an intentional change. Fixed by restoring the original "table exists = incremental" semantics. The try/catch now sets hasTable = true on successful query execution regardless of row count. This was the CI failure root cause.

@carlos-alm
Copy link
Copy Markdown
Contributor Author

Addressed Claude review conditions:

  1. stderr logging in validateSchemaVersion: Acknowledged -- the eprintln! usage is a known limitation of the Rust native layer which doesn't have access to the structured JS logger. This is consistent with other native methods. A follow-up to pipe native warnings through a callback would be broader scope than this PR.

  2. CI validation of new tests: The CI fix has been pushed -- the hasTable probe regression is resolved. Tests should now pass with the native binary on all platforms.

Additionally addressed the Greptile findings with doc comments, runtime param validation, and the hasTable fix. See the comment above for details.

@carlos-alm
Copy link
Copy Markdown
Contributor Author

@greptileai

@carlos-alm
Copy link
Copy Markdown
Contributor Author

@claude

@claude
Copy link
Copy Markdown

claude bot commented Mar 29, 2026

Claude finished @carlos-alm's task —— View job


Acknowledging Review Feedback Implementation

Thank you for the thorough updates addressing both the Claude and Greptile review feedback!

Todo List

  • Review user's implementation of feedback
  • Acknowledge the fixes made

Summary of Changes Addressed

Greptile Findings Addressed:

  1. BLOB/cell-error handling: Added proper documentation to row_to_json explaining the contract for BLOB columns and error handling
  2. Unsafe params cast: Implemented validateNativeParams() with runtime validation for type safety
  3. SQL method documentation: Updated doc comments to clarify SELECT-only intent while noting current implementation accepts arbitrary SQL
  4. hasTable probe regression: Fixed the semantic change that was causing CI failures by restoring original "table exists = incremental" logic

Claude Review Conditions:

  1. Stderr logging: Acknowledged as consistent with existing native layer patterns - reasonable to defer structured logging improvements to future work
  2. CI validation: The hasTable fix should resolve the test failures with native binary

Key Improvements Made

The fix to the hasTable probe semantics was particularly important - restoring the original behavior where an existing-but-empty file_hashes table correctly triggers incremental builds rather than full rebuilds. This was indeed a regression rather than an intentional behavior change.

The addition of validateNativeParams() is excellent defensive programming that will catch type safety issues early rather than allowing them to surface as runtime errors in the Rust layer.

Great work addressing all the review feedback comprehensively while maintaining the PR's core functionality and performance improvements!


@carlos-alm carlos-alm merged commit 5aac6b0 into main Mar 29, 2026
24 checks passed
@carlos-alm carlos-alm deleted the feat/dynamic-sql-edge-cases branch March 29, 2026 07:49
@github-actions github-actions bot locked and limited conversation to collaborators Mar 29, 2026
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

follow-up: restore path aliases with TS 6.x-compatible configuration

1 participant