Skip to content

models: add extra_payload to create_custom / update_custom#171

Merged
m-peko merged 4 commits into
mainfrom
dstepanov/custom-model-extra-payload
May 20, 2026
Merged

models: add extra_payload to create_custom / update_custom#171
m-peko merged 4 commits into
mainfrom
dstepanov/custom-model-extra-payload

Conversation

@stepdi
Copy link
Copy Markdown
Contributor

@stepdi stepdi commented May 19, 2026

Summary

Mirrors the new atlas-app backend field (LayerLens/atlas-app#1932) so SDK callers can attach arbitrary JSON to every outgoing chat-completions request body for their Custom Models.

`extra_payload` is deep-merged into the request body; customer values win on conflict with our hardcoded defaults. Practical uses:

  • Override `temperature` (we send `0` for reproducible evals) for providers that reject it.
  • Add provider-specific fields like `top_p`, `presence_penalty`, OpenRouter routing hints (`provider: { order: [...] }`).
  • Send `max_completion_tokens` for OpenAI's gpt-5 / o1 / o3 endpoints, which reject `max_tokens`.

Reserved keys: `messages`, `model`, `stream` — backend returns 400.

Changes

  • `models.create_custom` (sync + async) — new `extra_payload: Optional[Dict[str, Any]]` keyword argument.
  • `models.update_custom` (sync + async) — same. `{}` clears the existing payload.
  • Reference docs (`docs/api-reference/models-benchmarks.md`), README example (`docs/README.md`), examples (`docs/examples/models-and-benchmarks.md`), samples (`samples/core/custom_model.py`, `samples/claude-code/skills/benchmark.md`), CHANGELOG.
  • Example `api_url` values updated to the full `/v1/chat/completions` form to match the backend hint (the system uses the URL verbatim — no path is appended).

Backwards-compatible: existing callers keep working — the field is optional, and the backend additions are additive.

Test plan

  • `client.models.create_custom(..., extra_payload={"top_p": 0.9})` succeeds and the merged value reaches the upstream provider on a real eval.
  • `client.models.update_custom(model_id, extra_payload={"temperature": 1})` replaces the stored payload.
  • `client.models.update_custom(model_id, extra_payload={})` clears the stored payload.
  • Old SDK call sites without the new arg still work (no breakage).

Dependencies

  • Backend support lands in atlas-app#1932 — merge that first; SDK release should follow.
  • Backend now also enforces Key uniqueness within (org, project) at create time and rejects loopback / IMDS api_urls. SDK callers that previously relied on the silent-overwrite behavior for duplicate keys, or pointed at restricted IPs, will now see 409 / 400 respectively.

🤖 Generated with Claude Code

stepdi and others added 4 commits May 19, 2026 16:24
Mirrors the new atlas-app backend field (LayerLens/stratix-app#1932).
``extra_payload`` is an optional JSON object that's deep-merged into
every outgoing chat-completions request body — customer values win on
conflict with our hardcoded defaults. Lets callers override
``temperature`` (we send 0 for reproducible evals) or add
provider-specific fields like ``max_completion_tokens`` that some
endpoints require.

Updated:

- ``models.create_custom`` (sync + async) — new ``extra_payload``
  keyword argument.
- ``models.update_custom`` (sync + async) — same, plus a docstring
  note that ``{}`` clears the existing payload.
- Reference docs, README example, samples, CHANGELOG.
- Example ``api_url`` values now show the full ``/v1/chat/completions``
  form to match the backend hint.

Existing SDK callers keep working — the field is optional and additive
on the backend.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Five new cases exercising the new parameter:
- create_custom forwards extra_payload verbatim
- create_custom omits the key when None
- update_custom supports extra_payload-only updates
- update_custom({}) sends an empty dict so the backend clears the
  stored payload (semantics added in atlas-app#1932)
- update_custom omits the key when None

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Releases the ``extra_payload`` parameter added in the previous commits.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The previous commit bumped to 1.6.2, but adding the new
``extra_payload`` parameter counts as new functionality, so the
semver-correct minor bump is 1.7.0.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@m-peko m-peko merged commit 6205930 into main May 20, 2026
7 checks passed
@m-peko m-peko deleted the dstepanov/custom-model-extra-payload branch May 20, 2026 12:54
m-peko added a commit that referenced this pull request May 20, 2026
* Add CHANGELOG.md

* Create auto release GH workflow

* Update CHANGELOG.md, make updates in auto-release workflow

* models: add extra_payload to create_custom / update_custom (#171)

* models: add extra_payload to create_custom / update_custom

Mirrors the new atlas-app backend field (LayerLens/stratix-app#1932).
``extra_payload`` is an optional JSON object that's deep-merged into
every outgoing chat-completions request body — customer values win on
conflict with our hardcoded defaults. Lets callers override
``temperature`` (we send 0 for reproducible evals) or add
provider-specific fields like ``max_completion_tokens`` that some
endpoints require.

Updated:

- ``models.create_custom`` (sync + async) — new ``extra_payload``
  keyword argument.
- ``models.update_custom`` (sync + async) — same, plus a docstring
  note that ``{}`` clears the existing payload.
- Reference docs, README example, samples, CHANGELOG.
- Example ``api_url`` values now show the full ``/v1/chat/completions``
  form to match the backend hint.

Existing SDK callers keep working — the field is optional and additive
on the backend.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* test(models): cover extra_payload in create_custom / update_custom

Five new cases exercising the new parameter:
- create_custom forwards extra_payload verbatim
- create_custom omits the key when None
- update_custom supports extra_payload-only updates
- update_custom({}) sends an empty dict so the backend clears the
  stored payload (semantics added in atlas-app#1932)
- update_custom omits the key when None

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* chore: bump version to 1.6.2

Releases the ``extra_payload`` parameter added in the previous commits.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* chore: re-bump version to 1.7.0

The previous commit bumped to 1.6.2, but adding the new
``extra_payload`` parameter counts as new functionality, so the
semver-correct minor bump is 1.7.0.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Dmitry Stepanov <stepdi@gmail.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants