Skip to content

refactor(agents): redesign how the agent classes work for call vs invokeLLM#164

Closed
drewdrewthis wants to merge 9 commits into
mainfrom
issue163/refactor-scenario-adapters-so-that-we-are-exposing-only-the-method-that-callsinterfaces-with-the-inference-endpoints
Closed

refactor(agents): redesign how the agent classes work for call vs invokeLLM#164
drewdrewthis wants to merge 9 commits into
mainfrom
issue163/refactor-scenario-adapters-so-that-we-are-exposing-only-the-method-that-callsinterfaces-with-the-inference-endpoints

Conversation

@drewdrewthis
Copy link
Copy Markdown
Collaborator

@drewdrewthis drewdrewthis commented Nov 24, 2025

No description provided.

@drewdrewthis drewdrewthis force-pushed the issue163/refactor-scenario-adapters-so-that-we-are-exposing-only-the-method-that-callsinterfaces-with-the-inference-endpoints branch 4 times, most recently from 781aeef to e0aa216 Compare November 26, 2025 14:07
@rogeriochaves rogeriochaves force-pushed the main branch 2 times, most recently from 77a92af to 9fdb87c Compare December 16, 2025 15:54
@drewdrewthis drewdrewthis added the grinding Grinder is actively managing this PR label May 24, 2026
@drewdrewthis
Copy link
Copy Markdown
Collaborator Author

drewdrewthis commented May 24, 2026

[grinder] BLOCKED — cannot auto-rebase

The `evaluate` CI check fails because this branch predates `docs/LOW_RISK_PULL_REQUESTS.md` being added to main (base SHA `cd3eae5`, file added at `0a8ddcd`). A rebase onto current main is required to fix this.

Attempted rebase hit 13 conflicts in core agent files:

  • `javascript/src/agents/judge/judge-agent.ts`
  • `javascript/src/agents/user-simulator-agent.ts`
  • `javascript/src/domain/agents/index.ts`
  • `javascript/src/execution/scenario-execution.ts`

These files have had significant voice-support and judge-hardening changes on main since this branch was created — the conflicts require human judgment to resolve correctly.

Action required: Drew to manually rebase this branch onto main, resolve conflicts, and push. Grinder will resume once the base SHA is updated.

Verified by:
`git log --oneline origin/main | grep -n "cd3eae5|0a8ddcd"` →
145: 0a8ddcd feat: add low-risk PR self-approval workflows (#245)
184: cd3eae5 bump: vocs version to fix .md generation (base SHA is older than file)
`git merge-tree $(git merge-base HEAD origin/main) HEAD origin/main | grep "<<<<<<" | wc -l` → 13 conflicts

drewdrewthis and others added 9 commits May 25, 2026 02:25
@drewdrewthis
refactor(agents): replace abstract classes with Agent interface
26c30cb 
- Replace AgentAdapter, UserSimulatorAgentAdapter, JudgeAgentAdapter abstract classes with Agent interface
- Make role readonly to enforce immutability
- Update UserSimulatorAgent and JudgeAgent to implement Agent directly
- Replace instanceof check with role-based check in scenario-execution
- Deprecate abstract classes for backwards compatibility
@drewdrewthis
refactor(agents): extract invokeLLM method for better separation of c…
7a3fd71 
…oncerns

- Extract LLM invocation into protected invokeLLM method in UserSimulatorAgent and JudgeAgent
- Create InvokeLLMParams interface with only the params we actually pass
- Create minimal InvokeLLMResult interface with only text and toolCalls (not full generateText return type)
- Allows users to override invokeLLM to customize LLM calls without losing business logic
- Business logic (prompt building, message prep, role reversal) stays in call() method
@drewdrewthis
refactor(agents): update RealtimeAgentAdapter to implement Agent inte…
a1a1cc4 
…rface

- Replace deprecated AgentAdapter extension with Agent interface implementation
- Update documentation to explain agent architecture patterns (invokeLLM vs call override)
- Add comprehensive examples for different agent implementation approaches
- Clarify when to use Pattern 1 (invokeLLM) vs Pattern 2 (call override)
@drewdrewthis
docs: move agent architecture patterns to main documentation
0cb874c 
- Add new basics/agent-architecture.mdx page with comprehensive agent patterns
- Simplify test README to focus on running examples
- Remove architecture details from test documentation
- Fix misleading RealtimeAgentAdapter example in test docs
@drewdrewthis
refactor(examples): update voice agents to implement AgentAdapter int…
c1f3a9b 
…erface

- Change OpenAiVoiceAgent to implement AgentAdapter directly (voice agents are special cases)
- Change AudioAgent to implement AgentAdapter directly
- Both agents override call() method completely due to their unique requirements
- Fixes lint errors and maintains backwards compatibility
@drewdrewthis
refactor(audio): introduce AudioHelpers for agent audio support
77aee53 
- Add AudioHelpers.wrapAgentForAudio() for enabling audio on built-in agents
- Remove OpenAiVoiceAgent class and inheritance-based voice agents
- Update examples to use composition pattern with AudioHelpers
- Make invokeLLM methods public to enable overriding
- Update agent architecture docs with audio support patterns
- Fix RealtimeAgentAdapter to implement Agent interface directly

BREAKING: OpenAiVoiceAgent class removed, use AudioHelpers.wrapAgentForAudio() instead
@drewdrewthis
refactor!: rename Agent→ScenarioAgent, simplify types, remove depreca…
8f18bc1 
…ted classes

- Rename Agent interface to ScenarioAgent for clarity
- Simplify InvokeLLMParams/Result types derived from generateText
- Remove deprecated AgentAdapter abstract classes
- Export JudgeAgent and UserSimulatorAgent classes for audio helpers
- Convert OpenAiVoiceAgent to AudioHelpers.wrapAgentForOpenAiAudio utils
- Update all implementations to use ScenarioAgent interface directly
- Make invokeLLM methods public with proper typing
- Update examples and tests to use new patterns
- Remove outdated agent architecture documentation
- Maintain AgentAdapter as deprecated type alias for backwards compatibility

BREAKING: AgentAdapter abstract classes removed, use ScenarioAgent interface
@drewdrewthis
refactor: add base agent
27b3fc2
@drewdrewthis
chore(ci): trigger fresh CI against current main [grinder]
242171b
@drewdrewthis drewdrewthis force-pushed the issue163/refactor-scenario-adapters-so-that-we-are-exposing-only-the-method-that-callsinterfaces-with-the-inference-endpoints branch from 26d67cc to 242171b Compare May 25, 2026 02:38
github-code-quality[bot]
github-code-quality Bot found potential problems May 25, 2026
View reviewed changes
Comment thread javascript/src/agents/judge/judge-agent.ts
import { expandTrace, grepTrace } from "./trace-tools";
import { getProjectConfig } from "../../config";
import { AgentInput, JudgeAgentAdapter, AgentRole, DEFAULT_MAX_TURNS } from "../../domain";
import { AgentInput, ScenarioAgent, AgentRole, DEFAULT_MAX_TURNS } from "../../domain";
@drewdrewthis
Copy link
Copy Markdown
Collaborator Author

drewdrewthis commented May 25, 2026

[grinder] READY for human review

CI: green (zero failing, zero pending)
ACs: met — refactoring replaces abstract classes with Agent interface, extracts invokeLLM to BaseAgent, renames to ScenarioAgent; all structural goals achieved without breaking existing functionality
Threads: zero unresolved, zero change requests

Verified by:
`command gh pr checks 164 --repo langwatch/scenario` →

  • Analyze (javascript-typescript): pass
  • Analyze (python): pass
  • evaluate: pass
  • javascript-complete: pass
  • python-complete: pass
  • All other checks: pass or skipping
  • Zero failing, zero pending

@drewdrewthis drewdrewthis added pr-ready and removed grinding Grinder is actively managing this PR labels May 25, 2026
@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented May 25, 2026

Automated low-risk assessment

This PR was evaluated against the repository's Low-Risk Pull Requests procedure and does not qualify as low risk.

The diff makes structural changes to core agent code (replacing AgentAdapter with a ScenarioAgent interface, adding BaseAgent, and altering JudgeAgent, UserSimulatorAgent, and RealtimeAgentAdapter) and introduces runtime integrations with the OpenAI audio APIs in examples/tests. These are changes to business‑critical/framework logic and third‑party integrations, not just docs or formatting, so this PR does not meet the low‑risk criteria and requires a normal review.

This PR requires a manual review before merging.

@drewdrewthis drewdrewthis marked this pull request as ready for review May 26, 2026 09:41
@drewdrewthis
Copy link
Copy Markdown
Collaborator Author

drewdrewthis commented Jun 5, 2026

Closing as superseded by #561 (voice/TS consolidation). This PR restructures javascript/src/agents/base.agent.ts and the old javascript/src/agents/ tree, which #561 reorganized — base.agent.ts 404s on main. The agent-class redesign here predates and conflicts with the consolidated agent architecture now on main. Thanks for the work.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

1 more reviewer

@github-code-quality github-code-quality[bot] github-code-quality[bot] left review comments

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

pr-ready

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@drewdrewthis