-
Notifications
You must be signed in to change notification settings - Fork 31
fix: Include the AI Judge Config key with tracked metrics #986
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
|
@launchdarkly/browser size report |
|
@launchdarkly/js-sdk-common size report |
|
@launchdarkly/js-client-sdk-common size report |
|
@launchdarkly/js-client-sdk size report |
🤖 I have created a release *beep* *boop* --- <details><summary>server-sdk-ai: 0.14.1</summary> ## [0.14.1](server-sdk-ai-v0.14.0...server-sdk-ai-v0.14.1) (2025-11-13) ### Bug Fixes * Include the AI Judge Config key with tracked metrics ([#986](#986)) ([213fc79](213fc79)) </details> <details><summary>server-sdk-ai-langchain: 0.3.1</summary> ## [0.3.1](server-sdk-ai-langchain-v0.3.0...server-sdk-ai-langchain-v0.3.1) (2025-11-13) ### Dependencies * The following workspace dependencies were updated * devDependencies * @launchdarkly/server-sdk-ai bumped from ^0.14.0 to ^0.14.1 * peerDependencies * @launchdarkly/server-sdk-ai bumped from ^0.14.0 to ^0.14.1 </details> <details><summary>server-sdk-ai-openai: 0.3.1</summary> ## [0.3.1](server-sdk-ai-openai-v0.3.0...server-sdk-ai-openai-v0.3.1) (2025-11-13) ### Dependencies * The following workspace dependencies were updated * devDependencies * @launchdarkly/server-sdk-ai bumped from ^0.14.0 to ^0.14.1 * peerDependencies * @launchdarkly/server-sdk-ai bumped from ^0.14.0 to ^0.14.1 </details> <details><summary>server-sdk-ai-vercel: 0.3.1</summary> ## [0.3.1](server-sdk-ai-vercel-v0.3.0...server-sdk-ai-vercel-v0.3.1) (2025-11-13) ### Dependencies * The following workspace dependencies were updated * devDependencies * @launchdarkly/server-sdk-ai bumped from ^0.14.0 to ^0.14.1 * peerDependencies * @launchdarkly/server-sdk-ai bumped from ^0.14.0 to ^0.14.1 </details> <details><summary>shopify-oxygen-sdk: 0.1.0</summary> ## 0.1.0 (2025-11-13) ### Features * adding shopify oxygen server sdk ([#991](#991)) ([e70e484](e70e484)) </details> --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please). <!-- CURSOR_SUMMARY --> --- > [!NOTE] > Publishes server-ai 0.14.1 (metrics bug fix), bumps AI provider packages to 0.3.1, updates examples, and adds the new shopify-oxygen package at 0.1.0. > > - **Release versions**: > - `packages/sdk/server-ai` → `0.14.1` with bug fix: includes AI Judge Config key in tracked metrics. > - `packages/ai-providers/server-ai-{langchain,openai,vercel}` → `0.3.1` aligning dev/peer deps to `@launchdarkly/server-sdk-ai@^0.14.1`. > - **New package**: > - `packages/sdk/shopify-oxygen` introduced at `0.1.0`. > - **Examples**: > - Bump `@launchdarkly/server-sdk-ai` to `0.14.1` across example `package.json`s (`bedrock`, `openai`, `tracked-chat`, `vercel-ai`). > - **Manifest**: > - `.release-please-manifest.json` updated to reflect the above versions. > > <sup>Written by [Cursor Bugbot](https://cursor.com/dashboard?tab=bugbot) for commit d1f2610. This will update automatically on new commits. Configure [here](https://cursor.com/dashboard?tab=bugbot).</sup> <!-- /CURSOR_SUMMARY --> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Tracked internally as SDK-1554
Note
Propagates AI config
keythroughout configs and evaluation flow, addsjudgeConfigKeytoJudgeResponse, and tracks judge metrics with the config key.trackJudgeResponse(response: JudgeResponse)toLDAIConfigTracker/LDAIConfigTrackerImplto track eval scores withjudgeConfigKey.TrackedChatnow usestracker.trackJudgeResponseinstead oftrackEvalScoresafter successful judge evaluations.Judge.evaluate/evaluateMessagesnow includejudgeConfigKeyin the returnedJudgeResponse(success and error paths).keyonLDAIConfigand include it in all produced configs.LDAIConfigUtilsto acceptkeyinfromFlagValue,to*Config,_toBaseConfig, andcreateDisabledConfig, embeddingkeyin configs.LDAIClientImplpasseskeyintoLDAIConfigUtilsand returns disabled configs viacreateDisabledConfig(key, mode)on mode mismatch.JudgeResponsewith optionaljudgeConfigKey.keyon returned configs andjudgeConfigKeyin judge results; minor expectation formatting adjustments.Written by Cursor Bugbot for commit ef18b3e. This will update automatically on new commits. Configure here.