Skip to content

0.2.1 - #4

Merged
Timwood0x10 merged 31 commits into
mainfrom
dev
Jul 19, 2026
Merged

0.2.1#4
Timwood0x10 merged 31 commits into
mainfrom
dev

Conversation

@Timwood0x10

Copy link
Copy Markdown
Owner

v0.2.1 (2026-07-17)

Open-source release. Closes the gap between the Resolver Pipeline and the query/verify surfaces (call-graph resolve_strategy propagation, module-tree JSON validity, capability verifier LIKE-direction, module-hierarchy materialisation), plus FFI boundary detection, paginated graph export, LadybugDB embedded storage, one-click bootstrap, and a full code-review / portability / documentation pass.

🚀 New Features

  • FFI Boundary Detection (codescope_ffi_boundaries): Automatically detects cross-language FFI boundaries in the codebase — identifies extern "C" blocks, #[no_mangle] symbols, JNI declarations, and C ABI function exports. Helps developers audit unsafe interop surface.
  • Paginated Graph Export (codescope_export_graph): Full graph export with cursor-based pagination. Supports configurable page size, filter by edge type, and streaming output for large codebases. Integrates with MCP tooling for seamless client-side consumption.
  • One-Click Bootstrap (codescope_bootstrap): Zero-configuration project setup — auto-detects project language, runs indexing, and verifies the graph is ready. Single command from clone to queryable graph.
  • LadybugDB Embedded Storage: Optional LadybugDB backend for graph storage — provides faster local graph queries vs SQLite, with automatic fallback.
  • LadybugDB incremental sync: Added lbug_sync_state table to track incremental sync progress (last synced node id, edge rowid, and full-sync flag) so re-syncs only process new graph data.
  • ISSUE_TEMPLATE and CONTRIBUTING guidelines: Added GitHub issue templates and CONTRIBUTING.md to guide open-source contributors.

🐛 Bug Fixes

  • resolve_strategy not propagated to graph_edges: Visitor-level resolve_strategy (p1_intra / external / unresolved) was correctly written to semantic_records but never reached graph_edges through the Resolver Pipeline. find_callees / find_callers / engine_get_callees / engine_get_callers therefore always returned an empty resolve_strategy, surfacing third-party symbols (dropout, backward_hook, means, stds, LSTMLayer) as in-project callees. Fixed by closing the full chain semantic_records → reference → _resolved_edges → graph_edges (schema migration in store_schema.cpp, staging in pipeline.cpp, output restored in query_engine.cpp + store_query.cpp). Verified on bun (8 languages): 100% of edge_type=1 (call) edges carry a non-empty strategy. See docs/bugs/bug_resolve_strategy.{zh,en}.md for the full fix chain.
  • get_module_tree invalid JSON (leading comma in children arrays): GraphStore::getModuleTreeJson (store_project.cpp) used a single shared first flag across the whole recursion. After the first root was emitted, every children array started with a leading comma ([{...},{...}]) — invalid JSON that crashed client json.loads. Fixed by threading first as a bool & parameter so each sibling list owns its own flag. Language-agnostic (any project with ≥2 module-tree levels reproduced).
  • verify_claim(capability_exists) always Contradicted: capability_verifier.cpp had the LIKE match direction reversed in both capabilityDeclared and entitiesWithCallersLOWER(?) LIKE LOWER(name)||'%' (subject LIKE name) instead of LOWER(name) LIKE LOWER(?)||'%' (name LIKE subject). Since the README-derived subject is the longer form and the stored capability/node name is the short form, the reversed direction matched almost nothing — even perfect name matches returned Contradicted. Fixed to align with the correct name LIKE pattern direction already used by architecture_verifier.cpp and contract_verifier.cpp.
  • macOS install instructions missing LadybugDB: README.md, QUICK_START.md, and bootstrap.sh did not list LadybugDB as a dependency, but server/build.rs unconditionally links liblbug. Added brew install ladybug (macOS) and curl -fsSL https://install.ladybugdb.com | sh (Linux) to all install paths.
  • build.rs Linux library path portability: The LadybugDB link search path was hardcoded to /opt/homebrew/lib (macOS-only). Now resolves the correct path per platform.
  • C++ FFI exception safety: All extern "C" boundary functions are now wrapped in try/catch so a C++ exception never crosses the FFI boundary into Rust (which would abort the process).
  • CI now runs C++ tests: GitHub Actions workflow updated to compile and execute the C++ test suite on every push.
  • Documentation consistency: Corrected tool count (37, not 19 or 32), replaced stale 11-table list with the actual 40-table schema, expanded environment variables table from 3 to 11 entries, removed graph_query from the "does not exist" list (it is implemented), standardized token savings to 98.9%, and removed the stale codebase-memory-mcp benchmark table.
  • MemberExpr call edges: C++ a->foo() and b.foo() no longer generate false positive edges to every function named foo in the project. Resolution now checks the qualifier type before matching.
  • Query timeout: Long-running fuzzy searches no longer block the server. Configurable max_query_time_ms (default 5000ms).
  • Graph export OOM: Paginated export prevents memory exhaustion on large graphs (100k+ nodes) by streaming results in pages of configurable size.
  • fix [Bug] #3

🔧 Improvements

  • modules table now populated: GraphStore::insertModule (store_project.cpp) existed but was never called — modules stayed empty, so explain_module / get_module_tree degraded to reading only module_edge (dependency edges) and could not render module hierarchy (parent_id / name / path / language). Added populateModulesHierarchy (async_knowledge.cpp) called after buildKnowledgeGraphSync COMMIT: collapses entity.module_path directories into one modules row per distinct path, with parent_id resolved by next-shorter prefix and file_count / majority language per directory. Idempotent via insertModule's existence check. Verified on bun: 253 modules rows, 21 roots, nested tree JSON valid.
  • Query Limits & Error Handling: Added configurable query timeouts and result caps. Graceful error recovery for malformed queries — returns partial results instead of failing.
  • Graph Building Logic: Optimized buildGraph to handle orphaned nodes and broken references without crashing. Better error messages for cycle detection and constraint violations.
  • MemberExpr False Positives Eliminated: Fixed a bug where C++ MemberExpr (e.g., obj.method()) was incorrectly resolved as a direct call edge to unrelated functions. Now correctly distinguishes qualified member access from free function calls, improving call graph accuracy by ~15% on C++ codebases.
  • test_bun parameterised: engine/tests/test_bun.cpp previously hardcoded /Users/scc/code/researcher/bun. Restored argv[1] parameterisation with the hardcoded path retained as default (backward compatible).
  • Dead code buildCallEdgesSQL fully removed: buildGraph() casts build_calls to (void) (store_graph.cpp:320), so buildCallEdgesSQL (store_intern.cpp) was never called — but the 676-line function body was still maintained, inviting future maintainers to edit dead code. Removed the function body and the stale docstring in store.h; left a comment block pointing to the Resolver Pipeline and docs/bugs/bug_resolve_strategy.zh.md Bug 1 for rationale.
  • containment edges (edge_type=3) now write resolve_strategy: store_graph.cpp containment-edge INSERT now JOINs semantic_records psr and writes psr.resolve_strategy. Ineffectual for the strategy itself (parent is a declaration node; strategy semantics only apply to CallExpr kind=9) but keeps the column populated for schema consistency.

🔒 Code Review Fixes

  • LadybugDB stale data on re-index: buildGraph now calls resetLadybugSyncState before sync to force a full sync when the SQLite graph was rebuilt, preventing stale nodes/edges from accumulating in LadybugDB.
  • build.rs / CMakeLists.txt LadybugDB synchronization: build.rs now reads the CMake cache (CMakeCache.txt) to determine whether CMake found liblbug, ensuring the Rust link step stays in sync with the HAS_LADYBUG compile definition. Eliminates the mismatch risk when LadybugDB is installed under a custom prefix.
  • CSV temp file collision risk: Incremental sync CSV filenames now include project_id to prevent concurrent-project collisions.
  • CSV cleanup consistency: Node and edge CSV error handling now uniformly retain the CSV for debugging on COPY FROM failure.
  • FFI contract clarity: Added exemption comment to engine_free_string documenting why free() is exempt from the try/catch wrapper requirement.
  • Redundant try/catch removed: Simplified engine_find_connected_components by merging the redundant inner/outer try/catch into a single wrapper.
  • ffi::init() return value checked: tools/mod.rs now verifies the engine re-init return code after worker subprocess, preventing silent permanent failure.
  • Rust server panic safety: Replaced 4 serde_json::to_value().expect() calls in server.rs with proper -32603 error responses — server no longer crashes on serialization failure.
  • Removed dead tokio dependency: The server is fully synchronous; tokio was unused and added compile time/binary size.
  • engine_version() FFI: Added version function + --version CLI flag for runtime version inspection.
  • .clang-format rewrite: Replaced 807-line Linux kernel config with 94-line project-specific config (c++17, removed GPL header, removed 600 irrelevant ForEachMacros).
  • C-style casts eliminated: 12 (const char *) casts replaced with reinterpret_cast across engine_ffi.cpp and query_analysis.cpp.
  • Configurable synchronous mode: PRAGMA synchronous now defaults to OFF but can be overridden via CODESCOPE_SYNCHRONOUS=NORMAL|FULL|OFF env var.
  • Chinese comments translated: All CJK comments in engine/src/ and engine/include/ translated to English.
  • Global singleton thread-safety documented: Added prominent contract block in engine_internal.h documenting the sequential-dispatch model and future migration path.

📚 Documentation

  • docs/bugs/bug_resolve_strategy.{zh,en}.md: bilingual bug-fix process records for the resolve_strategy propagation defect — root cause, fix actions per file, verification data across bun / Transformer_Explorer / Neural_Network_Math_Explorer.
  • docs/dev_plans/ffi_detection_plan.md: development plan (next-next step, not shipped in 0.2.1) for turning CodeScope from an FFI boundary locator into an FFI boundary correctness checker. Scope narrowed to in-project source (excludes third-party / stdlib callees) with accuracy re-estimate Phase 1 95-98% / Phase 2 80-90% / Phase 3 60-75%. Includes independent ffi_* storage schema (ffi_boundary / ffi_findings / ffi_scan_summary) isolated from the main analysis tables, and the decision rule reusing existing resolve_strategy + BuiltinRegistry::isKnownExternal for in-project vs third-party discrimination.

🧹 Chores

  • Removed accidentally committed binary version file: A stray binary artifact was removed from the repository.
  • Committed Cargo.lock: Required for reproducible builds of the binary crate. Was previously gitignored.
  • Gitignored runtime artifacts: runtimelog/, llvm_ir/output/, and *.lbug files are now properly ignored.
  • Open-source community files: Added CONTRIBUTING.md, SECURITY.md, CODE_OF_CONDUCT.md, .github/CODEOWNERS.
  • GitHub Actions SHA-pinned: All workflow actions pinned to commit SHAs for supply-chain security.
  • Non-destructive release pipeline: build.yml no longer force-pushes tags or deletes existing releases. Added semver monotonicity validation.
  • CI timeout reduced: 120min → 45min to fail fast on hangs.
  • Test suite expanded: TEST_EXES expanded from 28 to 37 (added test_fp_*, test_graph_semantic, test_semantic_unit, test_type_extraction, etc.). Manual debug tools moved to engine/manual/.
  • Known-failing tests documented: test_enhance_e2e, test_fp_rust, test_fp_java, test_{js,ts,tsx}_visitor excluded from TEST_EXES with documented reasons.

This commit adds several key improvements to the codebase:
1.  Added builtin symbol registry to categorize external/unresolved calls
2.  Added call resolve_strategy field to track call resolution provenance
3.  Extended query APIs with optional file_filter to disambiguate homonyms
4.  Updated all language visitors to set proper call strategies
5.  Added database migrations for new schema fields
6.  Updated all FFI and test code to support new APIs
7.  Added comprehensive tests for resolve strategy and homonym filtering
@Timwood0x10
Timwood0x10 marked this pull request as draft July 18, 2026 03:04
@Timwood0x10
Timwood0x10 marked this pull request as ready for review July 19, 2026 11:44
@Timwood0x10
Timwood0x10 merged commit 33292ad into main Jul 19, 2026
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]

1 participant