Using mature code indexing

[kcramakrishna]

Consider Mature codeindexing to override coding-cli inbuilt features.
Compare inbuilt code-indexing of opencode, aider etc with more mature tools like Joern etc and integrate these into our harness. Maybe make this configurable. I think that the harness results will be significantly better with better code indexing. This integration will be a big factor in allowing us to use cheaper (or local) models for coding. i.e. accurate context will be crucial in getting excellent results from cheaper (or local) models too.

This will help greatly with impact analysis discussed here: #11

[kcramakrishna]

Some research throws this up. I am skeptical about dropping Joern but we can go ahead with below for now and run experiments with other tools later.

The Winning "Twin-Engine" Setup

By dropping the legacy Joern setup, you are left with two highly optimized, complementary MCP servers that perfectly balance your agent's cognitive needs.

1. The Structural Engine: `codebase-memory-mcp

Role: High-accuracy structural mapping.
Best For: Graph-native structural queries, tracking variables, following function flows, and analyzing blast radius.
Why keep it: It provides your planner (GLM 5.2) with the fastest, lowest-token way to understand exactly how your application is wired together.

2. The Semantic Engine: `code-memory

Role: Offline local semantic/vector search.
Best For: Finding high-level features, reading human intent, and scanning your TypeScript documentation.
Why keep it: Structural graph tools (like codebase-memory-mcp and codebadger) typically strip away and ignore inline documentation like JSDoc during parsing. code-memory acts as your semantic layer, explicitly using local embeddings to natively search your world-class JSDoc and TypeDoc. It saves up to 50% or more of your token usage by retrieving targeted code slices instead of full files.

** 3. Consider 'Graphify' as alternatitive to 'code-memory'**
https://www.youtube.com/watch?v=5kAJ7CA7ZB4
We need to compare 'code-memory' and 'Graphify' . We also need to edit AGENTS.md to instruct agents to use the appropriate tool for the specific use-case

[Pranav2612000]

Hey @kcramakrishna ! Great project! I was looking for solutions to the specification drift problem ( the code has evolved, but the docs still talk about stale decisions ), and landed here. Found this thread while browsing open issues to look for things I can contribute to. I'm currently part of the team working on https://github.com/infino-ai/code-context which is local first, and solves this exact problem ( Obviously not as powerful as Joern, but could be a good fit for "2. The Semantic Engine" ). Would you be interested in trying this out if I open up a PR with the integration?

[kcramakrishna]

Hi Pranav, Regret the late reply. Love to see you working on this problem. agentic-tdd is my hobby project which I work in my free time and weekends. Can you compare your tool to some of the existing standard ones - See this: https://www.youtube.com/watch?v=5kAJ7CA7ZB4
A detailed comparison will be useful to help me decide.

[kcramakrishna]

Our harness can use a lightweight 'inbuilt' , 'real-time' code indexer to index between passes.

Considered ast-grep/napi and web-tree-sitter. We will start with napi and later go on to look at web-tree-sitter.

Napi supports all major languages. Maybe let user pick his tool during install time.

https://share.gemini.google/sxz04NKxaLYx

[nathanschmid08]

I think making code indexing a configurable layer of the harness makes a lot of sense, especially if the goal is to support cheaper or local models.

One thing I'd consider is separating indexing from retrieval/context construction. A mature indexer could provide things like:

• symbol and definition lookup
• call/caller relationships
• inheritance and implementation relationships
• import/dependency graphs
• references across the repository
• potentially data/control-flow information where available

The harness could then use that information to construct the context for a particular task instead of relying primarily on the model's own repository search.

This could also make the impactanalysis use case from #11 much more deterministic. For example, before modifying a function, the harness could query the index for callers, implementations, tests and dependent modules and explicitly provide that context to the model.

I'd probably avoid coupling the harness directly to one indexing implementation, though. A small adapter/interface around the index would make it possible to experiment with different backends (OpenCode/Aider-style indexing, Joern, language-specific indexes, etc.) and choose one depending on the project.

The interesting part would be benchmarking this: same task + same model, with and without indexed context. If the indexed version consistently improves results for smaller models, that would be a pretty strong argument for making this a firstclass harness component.

[kcramakrishna]

Thanks. There are 2 levels at which we attacked this problem:

1. Targeted context: We are building custom context for each agent. Each agent logs the files, symbols etc. to a temporary 'statefile'. A 'context builder' builds the necessary context for the 'next' agent. We pass only the necessary files, symbols and hunks of code to each agent. Eg: 'Agent 3' - logs all the files, methods, classes etc (including line ranges) as shown below.
   -->
   "3": {
   "status": "completed",
   "filesTouched": [
   "rc/tictactoe/gui.py",
   "artifacts/Example-state-prompt-4.json"
   ],
   "attempts": 1,
   "commitHash": "44d1fce1891fbca10e1b3222622783020ec1fac4",
   "targetSymbols": {},
   "startedAt": "2026-08-10T09:47:39.874Z",
   "completedAt": "2026-08-10T09:47:39.874Z",
   "fileChanges": {
   "artifacts/Example-state-prompt-4.json": {
   "commitHash": "44d1fce1891fbca10e1b3222622783020ec1fac4",
   "kind": "edited-file",
   "hunks": [
   {
   "range": {
   "start": 1,
   "end": 135
   },
   "kind": "added",
   "addedLines": 135,
   "removedLines": 0,
   "symbols": [],
   "anchor": "{\n "schemaVersion": "0.2.0",\n "context": {\n "featureName": "Prompt-4",\n "testCmd": ["
   }
   ]
   },

2. Formal code indexing: Sometimes agents need to explore code beyond the context we are providing. For that we use powerful, industry standard code-indexers. The agents prioritise 'codebase-memory-mcp' --> 'graphifyy' --> ' opencode native/inbuilt indexer'. 'codebase-memory-mcp' is doing an amazing job and has solved the problem.

The Agents are instructed to first focus on the files/hunks that we are providing in context AND THEN use ''codebase-memory-mcp'' for any exploratory work. This ensures that LLM does not halucinate.

I expect to make a Beta release this week. Will be providing architecture documentation too. I will reply here once I make the release.