Skip to content

Conversation

@jsonbailey
Copy link
Contributor

@jsonbailey jsonbailey commented Nov 6, 2025

Tracked internally as SDK-1554


Note

Propagates AI config key throughout configs and evaluation flow, adds judgeConfigKey to JudgeResponse, and tracks judge metrics with the config key.

  • Tracking & Metrics:
    • Add trackJudgeResponse(response: JudgeResponse) to LDAIConfigTracker/LDAIConfigTrackerImpl to track eval scores with judgeConfigKey.
    • TrackedChat now uses tracker.trackJudgeResponse instead of trackEvalScores after successful judge evaluations.
  • Judge:
    • Judge.evaluate/evaluateMessages now include judgeConfigKey in the returned JudgeResponse (success and error paths).
  • Config Propagation:
    • Introduce required key on LDAIConfig and include it in all produced configs.
    • Update LDAIConfigUtils to accept key in fromFlagValue, to*Config, _toBaseConfig, and createDisabledConfig, embedding key in configs.
    • LDAIClientImpl passes key into LDAIConfigUtils and returns disabled configs via createDisabledConfig(key, mode) on mode mismatch.
  • Types:
    • Extend JudgeResponse with optional judgeConfigKey.
  • Tests:
    • Update tests to expect key on returned configs and judgeConfigKey in judge results; minor expectation formatting adjustments.

Written by Cursor Bugbot for commit ef18b3e. This will update automatically on new commits. Configure here.

@jsonbailey jsonbailey requested a review from a team as a code owner November 6, 2025 22:34
@jsonbailey jsonbailey marked this pull request as draft November 6, 2025 22:36
@github-actions
Copy link
Contributor

github-actions bot commented Nov 6, 2025

@launchdarkly/browser size report
This is the brotli compressed size of the ESM build.
Compressed size: 169118 bytes
Compressed size limit: 200000
Uncompressed size: 789399 bytes

@github-actions
Copy link
Contributor

github-actions bot commented Nov 6, 2025

@launchdarkly/js-sdk-common size report
This is the brotli compressed size of the ESM build.
Compressed size: 24988 bytes
Compressed size limit: 26000
Uncompressed size: 122411 bytes

@github-actions
Copy link
Contributor

github-actions bot commented Nov 6, 2025

@launchdarkly/js-client-sdk-common size report
This is the brotli compressed size of the ESM build.
Compressed size: 17636 bytes
Compressed size limit: 20000
Uncompressed size: 90259 bytes

@github-actions
Copy link
Contributor

github-actions bot commented Nov 6, 2025

@launchdarkly/js-client-sdk size report
This is the brotli compressed size of the ESM build.
Compressed size: 21721 bytes
Compressed size limit: 25000
Uncompressed size: 74698 bytes

@jsonbailey jsonbailey marked this pull request as ready for review November 6, 2025 23:25
@jsonbailey jsonbailey merged commit 213fc79 into main Nov 7, 2025
33 checks passed
@jsonbailey jsonbailey deleted the jb/include-judge-key-with-scores branch November 7, 2025 23:04
@github-actions github-actions bot mentioned this pull request Nov 7, 2025
joker23 pushed a commit that referenced this pull request Nov 13, 2025
🤖 I have created a release *beep* *boop*
---


<details><summary>server-sdk-ai: 0.14.1</summary>

##
[0.14.1](server-sdk-ai-v0.14.0...server-sdk-ai-v0.14.1)
(2025-11-13)


### Bug Fixes

* Include the AI Judge Config key with tracked metrics
([#986](#986))
([213fc79](213fc79))
</details>

<details><summary>server-sdk-ai-langchain: 0.3.1</summary>

##
[0.3.1](server-sdk-ai-langchain-v0.3.0...server-sdk-ai-langchain-v0.3.1)
(2025-11-13)


### Dependencies

* The following workspace dependencies were updated
  * devDependencies
    * @launchdarkly/server-sdk-ai bumped from ^0.14.0 to ^0.14.1
  * peerDependencies
    * @launchdarkly/server-sdk-ai bumped from ^0.14.0 to ^0.14.1
</details>

<details><summary>server-sdk-ai-openai: 0.3.1</summary>

##
[0.3.1](server-sdk-ai-openai-v0.3.0...server-sdk-ai-openai-v0.3.1)
(2025-11-13)


### Dependencies

* The following workspace dependencies were updated
  * devDependencies
    * @launchdarkly/server-sdk-ai bumped from ^0.14.0 to ^0.14.1
  * peerDependencies
    * @launchdarkly/server-sdk-ai bumped from ^0.14.0 to ^0.14.1
</details>

<details><summary>server-sdk-ai-vercel: 0.3.1</summary>

##
[0.3.1](server-sdk-ai-vercel-v0.3.0...server-sdk-ai-vercel-v0.3.1)
(2025-11-13)


### Dependencies

* The following workspace dependencies were updated
  * devDependencies
    * @launchdarkly/server-sdk-ai bumped from ^0.14.0 to ^0.14.1
  * peerDependencies
    * @launchdarkly/server-sdk-ai bumped from ^0.14.0 to ^0.14.1
</details>

<details><summary>shopify-oxygen-sdk: 0.1.0</summary>

## 0.1.0 (2025-11-13)


### Features

* adding shopify oxygen server sdk
([#991](#991))
([e70e484](e70e484))
</details>

---
This PR was generated with [Release
Please](https://github.com/googleapis/release-please). See
[documentation](https://github.com/googleapis/release-please#release-please).

<!-- CURSOR_SUMMARY -->
---

> [!NOTE]
> Publishes server-ai 0.14.1 (metrics bug fix), bumps AI provider
packages to 0.3.1, updates examples, and adds the new shopify-oxygen
package at 0.1.0.
> 
> - **Release versions**:
> - `packages/sdk/server-ai` → `0.14.1` with bug fix: includes AI Judge
Config key in tracked metrics.
> - `packages/ai-providers/server-ai-{langchain,openai,vercel}` →
`0.3.1` aligning dev/peer deps to `@launchdarkly/server-sdk-ai@^0.14.1`.
> - **New package**:
>   - `packages/sdk/shopify-oxygen` introduced at `0.1.0`.
> - **Examples**:
> - Bump `@launchdarkly/server-sdk-ai` to `0.14.1` across example
`package.json`s (`bedrock`, `openai`, `tracked-chat`, `vercel-ai`).
> - **Manifest**:
> - `.release-please-manifest.json` updated to reflect the above
versions.
> 
> <sup>Written by [Cursor
Bugbot](https://cursor.com/dashboard?tab=bugbot) for commit
d1f2610. This will update automatically
on new commits. Configure
[here](https://cursor.com/dashboard?tab=bugbot).</sup>
<!-- /CURSOR_SUMMARY -->

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants