Evolution: Symbol Index Multi-Language Extension (#03) by laynepenney · Pull Request #172 · synapt-dev/codi

laynepenney · 2026-01-26T05:16:24Z

Symbol Index Multi-Language Extension

Based on thorough review, reoriented proposal from full LSP integration to extending Codi's existing Symbol Index for multi-language support.

Key Changes from Original Proposal:

✅ Recognized existing symbol index capabilities
✅ Added comprehensive Symbol Index vs LSP comparison
✅ Proposed extending existing infrastructure vs replacing it
✅ Added LSP as optional completion-only feature (hybrid approach)

Existing Symbol Index (Already Working):

goto_definition - Fast navigation
find_references - Find all usages
find_symbol - Symbol search
get_dependency_graph - Dependency analysis
get_inheritance - Type hierarchy
get_call_graph - Function calls
SQLite-based (<10ms queries)
TypeScript & Kotlin support

New Proposal - Extend Symbol Index:

Python symbol extraction (def, class, import)
Rust symbol extraction (fn, struct, impl, trait)
Go symbol extraction (func, struct, interface)
Optional: LSP for code completion (completion only)
Optional: Real-time diagnostics

Why This Approach:

Symbol index is faster (<10ms vs 50-200ms IPC)
Builds on proven infrastructure
Lower resource usage (50MB vs 200-500MB)
Easier to maintain (self-contained)
LSP only for completion (main feature gap)

Implementation Plan:

Phase 1: Python support (1 week)
Phase 2: Rust & Go support (1 week)
Phase 3: Code completion via LSP (optional, 2 weeks)

Total Timeline: 2-4 weeks

See evolution/#03-symbol-index-extension.md for detailed comparison tables and implementation details.

Files Changed:

evolution/#03-symbol-index-extension.md - New reoriented proposal
evolution/README.md - Updated proposal status
evolution/#03-lsp-integration.md - Removed (replaced)

Wingman: Codi codi@layne.pro

**Critical Fixes:** - Delete legacy WebSearchTool (replaced by EnhancedWebSearchTool) - Delete legacy web-search.test.ts test file **Code Quality Improvements:** - Extract magic numbers to named constants (RELEVANCE_SCORES, RATE_LIMITS) - Remove unused 'sort' property from template configuration - Update relevance scoring to use constants for maintainability - Update rate limiting to use constants for consistency **Test Coverage Enhancements:** - Add test: should enforce rate limiting for engines - Add test: should reset rate limits after time period - Verify rate limit behavior with 5 requests/minute per engine - Verify automatic reset after 60-second timeout period **Changes Summary:** - Deleted: src/tools/web-search.ts (legacy implementation) - Deleted: tests/web-search.test.ts (legacy tests) - Modified: src/tools/enhanced-web-search.ts (constants, removed unused props) - Modified: tests/enhanced-web-search.test.ts (+2 new tests) **Test Results:** 8/8 tests passing (including 2 new rate limiting tests) Wingman: Codi <codi@layne.pro>

Document web search tool migration from WebSearchTool to EnhancedWebSearchTool. Wingman: Codi <codi@layne.pro>

Mark implementation as complete with updated status, goals, and success criteria. Reflect current state in documentation references. Wingman: Codi <codi@layne.pro>

Added comprehensive implementation summary covering: - Phase 1 & 2 deliverables - Code quality enhancements - Documentation updates - Beyond-scope improvements - Phase 3 readiness assessment Updated document version to 1.3 and date to 2026-01-26. Wingman: Codi <codi@layne.pro>

Create detailed evolution document for Language Server Protocol integration: - Symbol navigation (, ) - Code completion and IntelliSense - Semantic code understanding - Multi-language server support - AI-enhanced code analysis Proposed implementation in 3 phases over 4 weeks. Updated evolution README with current proposal status. Wingman: Codi <codi@layne.pro>

Based on thorough review, reoriented proposal from full LSP integration to extending the existing symbol index for multi-language support. Key changes: - Removed full LSP integration proposal - Added comprehensive Symbol Index vs LSP comparison table - Proposed extending existing symbol index for Python, Rust, Go - Added LSP as optional completion-only feature (hybrid approach) - Recognized existing symbol index capabilities (TS, Kotlin support) - Updated README with new proposal name and details - Reduced timeline from 4 weeks to 2-4 weeks The symbol index already provides: - goto_definition, find_references, find_symbol - get_dependency_graph, get_inheritance, get_call_graph - SQLite-based fast navigation - Background indexer - TypeScript and Kotlin support New approach focuses on adding Python, Rust, Go extractors + optional completion. Wingman: Codi <codi@layne.pro>

laynepenney · 2026-01-26T05:26:53Z

Self-Review Result ✅

Based on thorough review of the original LSP integration proposal, I've made the following corrections:

Critical Issues Addressed

1. Missing Existing Capabilities Context ✅

Added comprehensive section documenting existing Symbol Index infrastructure
Listed all 8+ tools already implemented (goto_definition, find_references, etc.)
Documented SQLite-based fast navigation architecture

2. No Clear Value Proposition ✅

Created detailed Symbol Index vs LSP comparison table (15 criteria)
Added performance benchmarks and resource usage comparisons
Provided clear justification for the new approach

3. Incomplete Alternatives Analysis ✅

Added 4 alternative options with detailed pros/cons
Recommended extending symbol index over full LSP integration
Added hybrid approach as viable option

4. Added Recommendation Sections ✅

Decision matrix showing symbol index wins for navigation (12/15 criteria)
Clear recommendation: Extend symbol index for new languages
LSP limited to completion-only (optional Phase 3)

Document Improvements

New Sections:

Current State: Detailed existing capabilities
Symbol Index vs LSP Comparison: 15-criteria comparison table
LSP Justification (Completion Only): Clear scope boundaries
New Language Extractors: Python, Rust, Go pattern libraries
Updated Implementation Plan: 3 phases, 2-4 weeks

Files Changed:

✅ Created #03-symbol-index-extension.md (reoriented proposal)
✅ Removed #03-lsp-integration.md (replaced)
✅ Updated README.md with correct proposal name
✅ Fixed #1-interactive-workflow-system.md → #01-interactive-workflow-system.md

Summary

The proposal now:

Builds on proven, fast infrastructure
Avoids unnecessary complexity (full LSP IPC)
Provides clear value proposition (multi-language support)
Addresses all review recommendations
Maintains alignment with Codi's architecture

Verdict: ✅ Ready for review and discussion

Added comprehensive implementation details: **Import Resolution Strategies:** - Python: import/from import, resolve to .py and __init__.py - Rust: use statements, :: to / notation conversion - Go: import "package", absolute path resolution **File Extension Resolution:** - Added LANGUAGE_EXTENSIONS constant with per-language patterns - Updated tryResolveWithExtensions for language awareness **Language Detection:** - Added getLanguageByExtension() function - Maps .py, .rs, .go to appropriate languages **Symbol Kinds by Language:** - Python: module, class, function, method, variable, constant, exception - Rust: function, method, struct, trait, impl, enum, const, macro, module - Go: function, method, struct, interface, type, const, variable **Doc Comment Patterns:** - Python: """...""", '''...''' - Rust: ///, /** */ - Go: //, /* */ **Updated Implementation Plan:** - Phase 1: Python Support (1.5 weeks) - Phase 2: Rust Support (1 week) - Phase 3: Go Support (1 week) - Phase 4: Infrastructure (0.5 weeks) - Phase 5: LSP Completion (2 weeks - Optional) **Updated Timeline:** 4-6 weeks (from 2-4 weeks) **Phase-based Success Criteria:** - Phase 1-3: Language extraction, import resolution, doc extraction - Phase 4: Language detection, filtering, stats - Phase 5 (Optional): LSP integration Wingman: Codi <codi@layne.pro>

laynepenney · 2026-01-26T05:37:56Z

Second Review: Implementation Details Added ✅

Based on thorough code review of existing symbol-index implementation, I've enhanced the proposal with detailed implementation specifications:

🔴 Critical Gaps Addressed

1. Import Resolution ✅

Added Python, Rust, Go import resolution strategies
Python: → module.py, init.py
Rust: → foo/bar.rs, mod.rs
Go: → absolute path resolution

2. Extension Resolution ✅

Added LANGUAGE_EXTENSIONS constant with language-aware patterns
Updated tryResolveWithExtensions() design

3. Language Detection ✅

Added getLanguageByExtension() function
Maps .py, .rs, .go to appropriate languages

4. Symbol Kinds ✅

Defined per-language symbol kinds:
- Python: module, class, function, method, variable, constant, exception
- Rust: function, method, struct, trait, impl, enum, const, macro, module
- Go: function, method, struct, interface, type, const, variable

5. Doc Comment Patterns ✅

Python: """...""", '''...'''
Rust: ///, /** */
Go: //, /* */

📊 Implementation Plan Updates

Timeline: 4-6 weeks (increased from 2-4 weeks)

Phase 1: Python Support (1.5 weeks)
Phase 2: Rust Support (1 week)
Phase 3: Go Support (1 week)
Phase 4: Infrastructure & Language Detection (0.5 weeks)
Phase 5: LSP Completion (2 weeks - Optional)

✅ Verification

Reviewed actual codebase:

- Lines 138-145 handle Kotlin, returns undefined for others
- Lines 162-166 only supports .ts/.kt extensions
- Schema already language-agnostic (no changes needed)
- TS/Kotlin patterns already extracted

Verdict: Implementation details now complete. Proposal ready for review.

laynepenney added 6 commits January 25, 2026 22:58

docs: add v0.17.0 changelog entry with breaking change notice

f5221bb

Document web search tool migration from WebSearchTool to EnhancedWebSearchTool. Wingman: Codi <codi@layne.pro>

docs: update evolution document to reflect Phase 2 completion

e6adf50

Mark implementation as complete with updated status, goals, and success criteria. Reflect current state in documentation references. Wingman: Codi <codi@layne.pro>

laynepenney changed the title ~~Evolution: LSP Integration Proposal (#03)~~ Evolution: Symbol Index Multi-Language Extension (#03) Jan 26, 2026

laynepenney merged commit e7ae542 into main Jan 26, 2026
3 checks passed

laynepenney deleted the feat/web-search-phase3 branch January 26, 2026 12:45

laynepenney mentioned this pull request Jan 26, 2026

chore: bump version to 0.17.0 #186

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Evolution: Symbol Index Multi-Language Extension (#03)#172

Evolution: Symbol Index Multi-Language Extension (#03)#172
laynepenney merged 7 commits intomainfrom
feat/web-search-phase3

laynepenney commented Jan 26, 2026 •

edited

Loading

Uh oh!

laynepenney commented Jan 26, 2026

Uh oh!

laynepenney commented Jan 26, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

laynepenney commented Jan 26, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Symbol Index Multi-Language Extension

Uh oh!

laynepenney commented Jan 26, 2026

Self-Review Result ✅

Critical Issues Addressed

Document Improvements

Summary

Uh oh!

laynepenney commented Jan 26, 2026

Second Review: Implementation Details Added ✅

🔴 Critical Gaps Addressed

📊 Implementation Plan Updates

✅ Verification

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

laynepenney commented Jan 26, 2026 •

edited

Loading