feat: Update LangChain runners to implement Runner protocol returning RunnerResult#150
Merged
jsonbailey merged 7 commits intomainfrom May 1, 2026
Merged
Conversation
This was referenced Apr 28, 2026
f1845b4 to
94f09ee
Compare
37bad31 to
b708885
Compare
94f09ee to
8463109
Compare
b708885 to
c553fbd
Compare
8463109 to
eacddee
Compare
c553fbd to
5df809b
Compare
eacddee to
842e4e6
Compare
5df809b to
c6e35a4
Compare
842e4e6 to
4c95357
Compare
c6e35a4 to
08dfcb7
Compare
4c95357 to
3f6882c
Compare
08dfcb7 to
ca37e74
Compare
3f6882c to
efeea93
Compare
ca37e74 to
9c0003c
Compare
efeea93 to
330acf1
Compare
9c0003c to
1aa1069
Compare
330acf1 to
cd983aa
Compare
1aa1069 to
f811cf8
Compare
cd983aa to
184be64
Compare
f811cf8 to
e56ea8f
Compare
184be64 to
4138d3c
Compare
e56ea8f to
4e0b78d
Compare
4138d3c to
7df7854
Compare
4e0b78d to
c1c2c8c
Compare
a770b1f to
069c0ee
Compare
dc86b07 to
2ba8406
Compare
069c0ee to
1c0255f
Compare
2ba8406 to
9254ccc
Compare
1c0255f to
0355872
Compare
9254ccc to
d113d46
Compare
0355872 to
796bda5
Compare
d113d46 to
2878bda
Compare
796bda5 to
a2db8cb
Compare
a2db8cb to
cbc377d
Compare
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes and found 1 potential issue.
There are 2 total unresolved issues (including 1 from previous review).
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, have a team admin enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit cbc377d. Configure here.
503c7b7 to
06e324f
Compare
cbc377d to
f53f119
Compare
06e324f to
b6d362a
Compare
34598f5 to
36cd239
Compare
keelerm84
approved these changes
May 1, 2026
…unner] The factory's downstream consumers (ManagedModel, ManagedAgent) now take Runner; aligning the factory's return types lets us drop the type: ignore comments at the ManagedModel/ManagedAgent call sites. Provider package PRs will update their concrete implementations to match. Judge still takes ModelRunner, so its call site picks up the type: ignore[arg-type] in its place — that's resolved later in the cleanup PR when Judge migrates to Runner.
- Judge now accepts Runner instead of ModelRunner - evaluate() calls runner.run(output_type=...) instead of invoke_structured_model - response.parsed replaces StructuredResponse.data; None guard added - evaluate_messages() accepts RunnerResult instead of ModelResponse - Tests updated to use RunnerResult and mock_runner.run Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
… RunnerResult - LangChainModelRunner.run() implements the unified Runner protocol; returns RunnerResult with content, metrics (LDAIMetrics), raw, and parsed fields. Structured output is supported via the output_type parameter. - LangChainAgentRunner.run() updated to return RunnerResult; populates tool_calls in LDAIMetrics from observed tool_calls in message responses. - Legacy invoke_model() and invoke_structured_model() retained as deprecated adapters that delegate to run() and wrap results into ModelResponse / StructuredResponse for backward compatibility. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…rit Runner - LangChainModelRunner: replaces invoke_model/invoke_structured_model with run(input, output_type=None); returns RunnerResult - LangChainAgentRunner: replaces AgentResult with RunnerResult; run() signature gains optional output_type parameter - Tests updated to call run() and assert result.content / result.parsed Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…tion in LangChainAgentRunner, add _coerce_input comment
…ME to new run() API
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
7aae3e2 to
1e61d42
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.

Summary
Stacking
Stacked on top of `jb/aic-2388/openai-runner-protocol` (PR #149).
Test plan
🤖 Generated with Claude Code
Note
Medium Risk
This is an API-shape change for LangChain runners (moving to
Runner.run()+RunnerResultand removing old response types), which could break downstream integrations and metric parsing if any callers still expect the legacy methods/fields.Overview
Updates the LangChain integration to the unified
Runnerprotocol:LangChainModelRunnerandLangChainAgentRunnernow expose a singlerun(input, output_type=...)that returnsRunnerResult(content,raw, optionalparsed) instead of the legacyinvoke_model/invoke_structured_modelresponses.Adds structured-output support to
LangChainModelRunner.run()viaoutput_typeand normalizes inputs (string vsList[LDMessage]), and enhances agent runs to capture tool call names intoLDAIMetrics.tool_calls.Docs and tests are updated to use
LDAIClient.create_model/ManagedModel.run()and the new result shapes; LangGraph agent-graph running no longer returns evaluation results onAgentGraphResult(they’re only flushed/tracked internally).Reviewed by Cursor Bugbot for commit 1e61d42. Bugbot is set up for automated code reviews on this repo. Configure here.