Skip to content

feat: implement additional config optimization api fields#196

Merged
andrewklatzke merged 3 commits into
aklatzke/AIC-2263/sdk-dx-improvementsfrom
aklatzke/AIC-2599/sdk-field-additions
Jun 18, 2026
Merged

feat: implement additional config optimization api fields#196
andrewklatzke merged 3 commits into
aklatzke/AIC-2263/sdk-dx-improvementsfrom
aklatzke/AIC-2599/sdk-field-additions

Conversation

@andrewklatzke

@andrewklatzke andrewklatzke commented Jun 1, 2026

Copy link
Copy Markdown
Contributor

Requirements

  • I have added test coverage for new or changed functionality
  • I have followed the repository's pull request submission guidelines
  • I have validated my changes against all supported platform versions

Describe the solution you've provided

We added first class support for some fields on the UI --

  • Latency optimization
  • Token optimization
  • Auto commit toggle

This PR pulls them into the SDK. token/latency optimization are using their previous paths for now in this PR. Rather than the regex approach we just use the flags from the API now. The latency/cost optimization paths will be updated in a subsequent PR.

Describe alternatives you've considered

The initial implementation of these two code paths for optimizations were kind of hacky to begin with (just using a dictionary to look up words that might mean they want to do it). This was the intended solution.


Note

Medium Risk
Changes when latency/cost gates and judge templates apply (explicit flags vs inferred text) and alters config-judge loading and auto-commit gating, which can shift optimization outcomes for existing runs.

Overview
Wires LaunchDarkly agent optimization API fields into the Python SDK: latencyOptimization, tokenOptimization, and autoCommit on remote configs, plus optional variation_key on OptimizationOptions / GroundTruthOptimizationOptions to start from a specific AI config variation (REST fetch; requires API key and project_key).

Latency and token behavior no longer infer goals from acceptance-statement keyword regexes. Gates, judge prompt augmentations, variation prompts, and model-pricing warnings now key off latency_optimization and token_optimization booleans (from options or API). When unset/false, those paths stay off.

Config judges resolve via raw flag variation() and local {{key}} interpolation (including message_history / response_to_evaluate) instead of LDAIClient.judge_config. System-only judge templates get an auto-built user turn.

optimize_from_config maps the new API fields into built options; auto-commit runs only when both the fetched config’s autoCommit and caller options allow it. Tests drop regex helpers and cover the new flags, judge path, and variation_key validation.

Reviewed by Cursor Bugbot for commit 509240f. Bugbot is set up for automated code reviews on this repo. Configure here.

@andrewklatzke
andrewklatzke requested a review from a team as a code owner June 1, 2026 22:56
Comment thread packages/optimization/src/ldai_optimizer/client.py
@andrewklatzke andrewklatzke changed the title implement additional config optimization api fields feat: implement additional config optimization api fields Jun 1, 2026
@andrewklatzke

Copy link
Copy Markdown
Contributor Author

Now additionally includes the content from here (fix): #197

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, have a team admin enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit 509240f. Configure here.

Comment thread packages/optimization/src/ldai_optimizer/client.py
@andrewklatzke
andrewklatzke merged commit 4ed30d8 into aklatzke/AIC-2263/sdk-dx-improvements Jun 18, 2026
5 checks passed
@andrewklatzke
andrewklatzke deleted the aklatzke/AIC-2599/sdk-field-additions branch June 18, 2026 17:29
@github-actions github-actions Bot mentioned this pull request Jul 17, 2026
andrewklatzke added a commit that referenced this pull request Jul 17, 2026
🤖 I have created a release *beep* *boop*
---


<details><summary>launchdarkly-server-sdk-ai: 1.2.0</summary>

##
[1.2.0](launchdarkly-server-sdk-ai-1.1.0...launchdarkly-server-sdk-ai-1.2.0)
(2026-07-17)


### Features

* implements template/raw methods for fetching configs
([1c06d35](1c06d35))
* implements template/raw methods for fetching configs
([#204](#204))
([5106866](5106866))
</details>

<details><summary>ldai_optimizer: 0.2.0</summary>

##
[0.2.0](ldai_optimizer-0.1.0...ldai_optimizer-0.2.0)
(2026-07-17)


### ⚠ BREAKING CHANGES

* Bump minimum LangChain version to 1.0.0

### Features

* ability to specify variation key on config or from_options for
optimization package
([c5e39eb](c5e39eb))
* ability to specify variation key on config or from_options for
optimization package
([#162](#162))
([f0c4612](f0c4612))
* add auto-commit option
([4cb8859](4cb8859))
* add optimization for duration
([5d76276](5d76276))
* Add optimization package stub
([#109](#109))
([ebd5166](ebd5166))
* add shared dataclass for calls so they can be handled by same handler
([31c8385](31c8385))
* add token limit handling
([e8c6692](e8c6692))
* adds ability to optimize for cost
([94de596](94de596))
* adds ability to optimize for cost
([#172](#172))
([3b4baa3](3b4baa3))
* adds ability to use inverted judges
([0ed243e](0ed243e))
* adds ability to use inverted judges
([#168](#168))
([23baeb4](23baeb4))
* Adds optimization package stub
([58b7731](58b7731))
* Adds optimization package stub
([cc85a05](cc85a05))
* adds reporting for cost and latency optimization failures
([365fa94](365fa94))
* adds reporting for cost and latency optimization failures
([#180](#180))
([d267832](d267832))
* all logs -&gt; debug
([2fd55e2](2fd55e2))
* Bump minimum LangChain version to 1.0.0
([dc592c5](dc592c5))
* Drop support for python 3.9
([#114](#114))
([dc592c5](dc592c5))
* dx improvements for optimization package
([7074cfa](7074cfa))
* ground truth optimization path
([44c8c59](44c8c59))
* implement ability to use completions or agents for judge calls
([ea596a7](ea596a7))
* implement additional config optimization api fields
([#196](#196))
([4ed30d8](4ed30d8))
* implement latency & token tracking for optimizations
([288336e](288336e))
* Implement optimization code paths and functionality for initial
release
([#140](#140))
([5204c47](5204c47))
* implementation of agent optimization + tests
([1712e4f](1712e4f))
* implements LD API client, optimize_from_config path
([2fecd54](2fecd54))
* implements optimize method in SDK, code moved
([9859d08](9859d08))
* partially implement optimize_from_config
([d3e1f96](d3e1f96))
* prevent overfitting via prompt changes and post-processing
([8f9f1e2](8f9f1e2))
* use judge_passed for all calcs
([a8f14de](a8f14de))


### Bug Fixes

* address cursor feedback
([f2f0894](f2f0894))
* adjust iteration logic so validation doesn't consume them
([3042984](3042984))
* better handling of params and custom params for optimization
([572a2aa](572a2aa))
* better handling of params and custom params for optimization
([#163](#163))
([92f51fa](92f51fa))
* consistency with other makefiles
([b9a5601](b9a5601))
* don't only evaluate final input in GT results
([53f455f](53f455f))
* don't only evaluate final input in GT results
([9bedf9e](9bedf9e))
* ensure cost data is persisted
([4eb0bb0](4eb0bb0))
* fix for single-iteration optimizations
([96eadb7](96eadb7))
* fix for single-iteration optimizations
([#209](#209))
([ba5aa63](ba5aa63))
* lint
([aee6aa7](aee6aa7))
* lint + missed variable rename
([f8e5509](f8e5509))
* lints + structured output tool rename
([8481690](8481690))
* move failure forward
([3c65a6e](3c65a6e))
* only strip known provider prefixes
([66bc1f0](66bc1f0))
* **optimization:** address Bugbot comments on PR
[#140](#140)
([9e57d86](9e57d86))
* **optimization:** fix CI test failures and apply Bugbot-reported fixes
([ffad0cb](ffad0cb))
* **optimization:** match model config by key as well as id for pricing
lookup
([a8415be](a8415be))
* **optimization:** remove dead persistence counter code
([c76cb16](c76cb16))
* **optimization:** retry LLM calls on transient provider errors
(429/503/529)
([a131e4b](a131e4b))
* **optimization:** use run-chosen context for config judges; guard
non-dict JSON
([64e12f2](64e12f2))
* pull model configs if available in options path
([4fc1ecf](4fc1ecf))
* remove unnecessary token path
([dc82818](dc82818))
* sort imports
([c032aaf](c032aaf))
* success path + add test, cursor feedback
([8f3468f](8f3468f))
</details>

---
This PR was generated with [Release
Please](https://github.com/googleapis/release-please). See
[documentation](https://github.com/googleapis/release-please#release-please).

<!-- CURSOR_SUMMARY -->
---

> [!NOTE]
> **Low Risk**
> Mechanical version, changelog, and dependency-floor updates from
Release Please; no runtime code changes in this PR.
> 
> **Overview**
> **Release Please** version bump only—no application logic changes in
this diff.
> 
> **`launchdarkly-server-sdk-ai` 1.2.0** — `pyproject.toml`,
`ldai.__version__`, changelog, and provenance docs move from **1.1.0 →
1.2.0**. The new changelog section documents **template/raw config fetch
APIs** (shipped in prior commits, e.g. `completion_config_template`).
> 
> **`launchdarkly-ai-optimizer` 0.2.0** — same pattern (**0.1.0 →
0.2.0**) with an expanded **CHANGELOG** (including **LangChain ≥ 1.0**
as a breaking note for optimizer consumers).
> 
> **Provider packages** — `launchdarkly-server-sdk-ai-langchain` and
`launchdarkly-server-sdk-ai-openai` now require
**`launchdarkly-server-sdk-ai>=1.2.0`** instead of **≥1.1.0**.
> 
> **`.release-please-manifest.json`** is updated to match the new
package versions.
> 
> <sup>Reviewed by [Cursor Bugbot](https://cursor.com/bugbot) for commit
e86bad4. Bugbot is set up for automated
code reviews on this repo. Configure
[here](https://www.cursor.com/dashboard/bugbot).</sup>
<!-- /CURSOR_SUMMARY -->
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants