Audit and Revamp Code & Snippet Agents + Judge Logic

[ishaanxgupta]

The current implementation of the CodeAgent, SnippetAgent, and their associated logic in the JudgeAgent needs a comprehensive audit and revamp. Several architectural gaps and technical debts have been identified that hinder reliability and scalability.

Lets have discussion on this issue first, whoever has the best solution will be assigned the issue, we can have multiple contributors on this as well.

Bounty: $20 API Credits

[LewiX-369]

Happy to tackle this issue.

/opire try

[Ramanand-Shirbhate]

Hey @ishaanxgupta, I'd love to help tackle this revamp.

I see you want to discuss solutions first before assigning. Based on the mention of architectural gaps and technical debt in the CodeAgent, SnippetAgent, and JudgeAgent, my initial audit strategy would focus on these three pillars:

Reliability (Judge Logic): Auditing the JudgeAgent's parsing and validation logic to ensure it gracefully handles unexpected LLM outputs, hallucinations, or malformed snippets without breaking the pipeline.

Scalability (Context & State): Reviewing how the CodeAgent and SnippetAgent manage state and context windows. Tightly coupled state often creates bottlenecks, so I'd look for ways to ensure they operate more modularly.

Technical Debt: Identifying tightly coupled dependencies between the agents and decoupling them into isolated, easily testable structures.

Could you point me to the specific directory or files where these agents currently live? I'll do a quick review of the current implementation and draft a concrete technical proposal for us to discuss here.

[ved015]

Happy to tackle this issue.

/opire try

Thank you for taking interest pls share your approach here

[vovchuklena-source]

Hi, I'd like to apply to work on this issue.

Quick read of the repo — I see this as medium, ~4h to a clean PR.

Approach: The task involves auditing and revamping the CodeAgent, SnippetAgent, and their associated logic in JudgeAgent. The main focus will be on improving error handling, simplifying code structure, and ensuring consistency with the provided system prompts.

Files I'd touch:

• src/agents/code.py
• src/agents/snippet.py
• src/prompts/code.py

I can open a draft PR within 24–48h once assigned. Let me know if you'd like me to take it.

/opire try

[LewiX-369]

Hi @ved015 and @ishaanxgupta,

Thanks for getting back to me! Here is my detailed technical proposal for the audit and revamp:

1. Reliable Structured Output (Addressing Parsing Brittleness)

Currently, CodeAgent and SnippetAgent rely on fragile string parsing (splitting by ```json) which leads to pipeline breaks when the LLM hallucinates markdown or extra text.
Approach: I will migrate both agents to use LangChain's .with_structured_output(Schema, include_raw=True). This enforces strict Pydantic schema validation at the model level. I'll implement a _call_model_structured wrapper in BaseAgent to handle this cleanly while preserving access to the raw AIMessage so that our metrics tracking (_track_llm_call) remains fully intact.

2. Deterministic Judging & Deduplication (Addressing Scalability)

Currently, JudgeAgent passes all incoming items to the LLM to decide on ADD, UPDATE, DELETE, or NOOP. For highly structured domains like code and snippet, this is an expensive and unnecessary bottleneck.
Approach:

• I will introduce _deterministic_code and _deterministic_snippet methods in JudgeAgent.
• Code: Deduplicate deterministically by matching target_symbol/target_file + annotation_type. If they match and content is identical, it's a NOOP. If content differs, it's an UPDATE.
• Snippet: Exact string match on the natural language description or code for NOOP/UPDATE.
• This bypasses the LLM entirely for these domains, saving tokens, latency, and eliminating hallucinations during the judging phase.

3. Better Context for Fallback Judging (Addressing Context State)

For cases where the semantic LLM judge must be used, passing raw dictionaries is suboptimal.
Approach: I will add _code_item_to_string and _snippet_item_to_string to format the context cleanly (e.g., [bug_report] auth_service.py: Token expires too early). I'll also update src/prompts/judge.py to explicitly teach the judge the domain-specific rules for code and snippet.

I have already audited the codebase (src/agents/code.py, src/agents/snippet.py, src/agents/judge.py, src/prompts/judge.py) and validated this approach locally. I can open a PR shortly if you approve this direction!