[Feature] Phase 1: Native AST Codebase Analyzer via Tree-Sitter

[purvanshjoshi]

[Feature] Phase 1: Native AST Codebase Analyzer via Tree-Sitter

Objective

To reduce our dependency on external indexers (like GitNexus) and provide our agents with deep, structural understanding of any repository, we need to build a native AST (Abstract Syntax Tree) parsing engine. This engine will map classes, functions, and imports with exact file paths and line numbers.

Requirements

1. Dependency Addition: Add tree-sitter and the primary language grammars (tree-sitter-python, tree-sitter-javascript, tree-sitter-typescript) to the requirements.txt file.
2. Core Engine: Create a new backend module backend/ast_indexer.py.
3. CodebaseMapper Class:
   • Develop a TreeSitterParser wrapper.
   • Develop a CodebaseMapper class that traverses a local repository.
   • For each file, parse the AST and extract:
     • Class definitions (name, start line, end line)
     • Function/Method definitions (name, start line, end line)
     • Imports/Dependencies
4. Output Generation: The mapper should export a structured architecture_map.json representation of the repository.

Acceptance Criteria

• tree-sitter libraries are successfully integrated into the backend environment.
• Running the CodebaseMapper on a test repository correctly identifies all functions and their line numbers without crashing.