fix: Preserve custom models during LLM preference reconciliation#11655
Conversation
When reconciling LLM preferences, fall back to custom LLM info if the model is not found in the built-in models list. This ensures custom models saved on execution profiles are not cleared during preference updates.
|
Thank you for your pull request and welcome to our community. We require contributors to sign our Contributor License Agreement, and we don't seem to have the users @KelvinJRosado on file. In order for us to review and merge your code, each contributor must visit https://cla.warp.dev to read and agree to our CLA. Once you have done so, please comment |
github-actions
Bot
added
the
external-contributor
|
I'm starting a first review of this pull request. You can view the conversation on Warp. I reviewed this pull request and requested human review from: Comment Powered by Oz |
|
@cla-bot check |
|
The cla-bot has been summoned, and re-checked this pull request! |
![oz-for-oss[bot]](https://avatars.githubusercontent.com/in/3450139?s=60&v=4){kind=small}
There was a problem hiding this comment.
Overview
This PR updates LLM preference reconciliation so custom BYOK model IDs are considered usable alongside server-provided model choices for base, coding, and CLI-agent profile settings. It also adds a regression test that configures a custom endpoint model, saves it on the default execution profile, runs model-choice reconciliation, and verifies the custom selections are preserved.
Concerns
- No blocking correctness, security, or spec-alignment concerns found in the changed lines.
Verdict
Found: 0 critical, 0 important, 0 suggestions
Approve
Comment /oz-review on this pull request to retrigger a review (up to 3 times on the same pull request).
Powered by Oz
abhishekp106
requested review from
danielpeng2
and removed request for
abhishekp106
|
I swapped the reviewer for someone who has more context. |
danielpeng2
left a comment
danielpeng2
left a comment
There was a problem hiding this comment.
This looks great, thanks for fixing!
4 participants
Description
With the current logic, only the built-in models are checked when preferences are updated/validated (such as on app startup). As a result, setting a custom model as the default will only persist until this preference update runs. This PR fixes this by extending that model check to also include custom models that the user defined.
Linked Issue
Closes #8562 and #11564
Testing
I tested manually by adding a custom BYOK model and verifying that I could use it for a new agent session. Then I quit the app and re-opened to verify that my default model was still set to my custom BYOK model.
./script/runScreenshots / Videos
Updated Profile With Custom Default Model
State After Quitting and Re-Opening