feat!: Add invokeStructuredModel method to support new Judge online evals #970
Conversation
|
This PR is reliant on #969 being merged first. |
|
@launchdarkly/browser size report |
|
@launchdarkly/js-sdk-common size report |
|
@launchdarkly/js-client-sdk size report |
|
@launchdarkly/js-client-sdk-common size report |
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
| expect(mockLogger.error).toHaveBeenCalledWith( | ||
| 'LangChain structured model invocation failed:', | ||
| error, | ||
| ); |
There was a problem hiding this comment.
Bug: Mismatched Logging Levels for Model Invocation Errors
The invokeModel and invokeStructuredModel methods in LangChainProvider.ts log caught errors using logger.warn(), but their corresponding tests expect logger.error(). This mismatch causes test failures and suggests an incorrect severity classification for these error conditions.
Additional Locations (3)
| metrics, | ||
| }; | ||
| } catch (error) { | ||
| this.logger?.warn('LangChain structured model invocation failed:', error); |
There was a problem hiding this comment.
Bug: Mismatch: log level for model invocation failures should be error
The invokeModel and invokeStructuredModel methods log model invocation failures at the warn level, but their corresponding tests expect error level logging. This mismatch causes test failures and suggests error is the more appropriate log level for these critical failures.
Note
Adds invokeStructuredModel with structured output support and wraps invokeModel with error handling; updates tests and aligns SDK peer/dev versions.
src/LangChainProvider.ts)invokeStructuredModel(messages, responseStructure)returningStructuredResponsewithdata,rawResponse, and basicmetrics.invokeModelin try/catch; on failure, return empty assistant message withsuccess=falseand log warning.__tests__/LangChainProvider.test.ts)invokeModelerror path.invokeStructuredModelsuccess and failure cases.package.json)@launchdarkly/server-sdk-aito^0.12.0in dev and peer dependencies.Written by Cursor Bugbot for commit 24d7520. This will update automatically on new commits. Configure here.