Skip to content

feat: allow custom models#51

Merged
AdirAmsalem merged 8 commits into
mainfrom
rene/custom-realtime-models
May 6, 2026
Merged

feat: allow custom models#51
AdirAmsalem merged 8 commits into
mainfrom
rene/custom-realtime-models

Conversation

@AdirAmsalem
Copy link
Copy Markdown
Contributor

@AdirAmsalem AdirAmsalem commented May 6, 2026

Summary

Lets callers use models that aren't in the SDK's built-in registry (preview, experimental, private) by constructing a ModelDefinition directly. Mirrors the JS SDK's CustomModelDefinition.

Usage

Built-in helpers stay the same:

from decart import DecartClient, models

client = DecartClient(api_key=os.getenv("DECART_API_KEY"))

# Process (sync image)
await client.process({"model": models.image("lucy-image-2"), "prompt": "...", "data": b"..."})

# Queue (async video)
await client.queue.submit({"model": models.video("lucy-clip"), "prompt": "...", "data": b"..."})

For non-registry models, construct a ModelDefinition directly:

from decart import DecartClient, ModelDefinition, RealtimeClient, RealtimeConnectOptions

# Realtime
custom_realtime = ModelDefinition(
    name="lucy_2_rt_preview",
    url_path="/v1/stream",
    fps=20,
    width=1280,
    height=720,
)
await RealtimeClient.connect(..., options=RealtimeConnectOptions(model=custom_realtime, ...))

# Process (sync)
custom_image = ModelDefinition(
    name="lucy_image_preview",
    url_path="/v1/generate/lucy_image_preview",
    fps=25,
    width=1280,
    height=704,
)
await client.process({"model": custom_image, "prompt": "...", "data": b"..."})

# Queue (async)
custom_video = ModelDefinition(
    name="lucy_video_preview",
    url_path="/v1/jobs/lucy_video_preview",
    fps=20,
    width=1280,
    height=720,
)
await client.queue.submit({"model": custom_video, "prompt": "...", "data": b"..."})

If input_schema is omitted, inputs are forwarded as-is and the backend validates them.

Validation

  • pytest -q — 128 passed (135 with decart[realtime] extras)
  • black / ruff / mypy clean
  • E2E smoke against the live API: built-in and custom ModelDefinition both succeed via client.process()

Note

Medium Risk
Expands supported model definitions beyond the SDK registry and changes request routing to rely on model.url_path, which could surface new runtime errors if callers supply incorrect paths or inputs.

Overview
Adds support for custom/non-registry models by letting callers construct ModelDefinition directly (exported as CustomModelDefinition) and making input_schema optional for passthrough validation.

Updates DecartClient.process() and QueueClient.submit() to accept any ModelDefinition[str], skip registry checks, and (when no schema is provided) forward non-None inputs as-is; queue submission now posts to base_url + model.url_path instead of /v1/jobs/{model.name}. Realtime connect typing is loosened to accept arbitrary model names, and tests are updated/added to cover custom models and backend error surfacing.

Reviewed by Cursor Bugbot for commit de8805b. Bugbot is set up for automated code reviews on this repo. Configure here.

@AdirAmsalem AdirAmsalem changed the title feat: allow custom realtime models feat: allow custom models May 6, 2026
@AdirAmsalem AdirAmsalem force-pushed the rene/custom-realtime-models branch 2 times, most recently from 8c032d6 to fdae6f3 Compare May 6, 2026 12:53
@AdirAmsalem AdirAmsalem force-pushed the rene/custom-realtime-models branch from fdae6f3 to b846107 Compare May 6, 2026 13:11
Drop the `models.custom(...)` factory in favor of constructing
`ModelDefinition(...)` directly, matching how the JS SDK exposes
custom models via the `CustomModelDefinition` type.

- `input_schema` becomes truly optional (`Optional[type[BaseModel]] = None`)
  instead of using `BaseModel` as a sentinel.
- Add `queue_url_path` to `ModelDefinition`; populate it on the video
  and image registry entries and use it from `submit_job`. Submitting a
  model without `queue_url_path` raises `InvalidInputError` (mirrors the
  JS check in `queue/request.ts`).
- Video registry `url_path` flips to `/v1/generate/<name>` to align
  with the JS SDK; queue submissions now go through `queue_url_path`.
Copy link
Copy Markdown

@cursor cursor Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, have a team admin enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit 985db94. Configure here.

Comment thread decart/models.py Outdated
Each model now has a single `url_path` field that points at its primary
endpoint:

- realtime: `/v1/stream`
- image:    `/v1/generate/<name>`
- video:    `/v1/jobs/<name>`

Queue, process, and realtime clients all submit/connect to
`{base_url}{model.url_path}`. This drops the JS-style split between
`urlPath` and `queueUrlPath` — JS only carries both because TypeScript
forces every model to have a `urlPath`. In Python a single field
suffices, since video models are queue-only and image models are
process-only via the registry.

Custom models continue to work: pass whichever endpoint is appropriate
for the API you intend to call (e.g. `url_path="/v1/jobs/<name>"` for a
custom queue model).
The realtime unit test and two docstrings still referenced the removed
`models.custom()` factory. Switch the test to constructing
`ModelDefinition` directly and update the docstrings to match.

Resolves the CI failure on Python 3.11 (and the Cursor Bugbot review).
Drop the "Custom models can omit an input schema..." comments — the
`if model.input_schema is None:` branch is self-explanatory.

Also tighten docstrings the PR over-extended:
- `DecartClient.queue` / `QueueClient`: drop the second-line "SDK accepts
  any model definition..." preamble
- `DecartClient.process`: drop the "backend validates whether the model
  supports this API" line and over-broad model list
- `QueueClient.submit`: drop the "Video models and custom queue model
  definitions are supported" line
- `CustomModelDefinition`: collapse the two-paragraph docstring to one line
Remove four tests that asserted process() accepts video/realtime
ModelDefinitions and queue.submit() accepts image/realtime
ModelDefinitions. They mocked the underlying request layer to pin
"the SDK does not gate by registry bucket", but they don't reflect
real usage — those calls would 404 against the backend because the
url_path of one model is wrong for the other API.
- test_models.py: drop the validates-required-shape test (Pydantic's
  own concern), and trim the over-asserting custom-model test to the
  meaningful invariants (str name accepted, input_schema defaults None).
- test_process.py: drop test_process_allows_custom_model_definition_for_realtime_url_path,
  same cross-bucket weirdness as the tests just removed.
- test_queue.py: drop test_queue_custom_model_uses_url_path (URL
  construction is identical for registry and custom models, already
  covered by test_queue_includes_user_agent_header). Refocus the
  bouncer-error test as test_queue_submit_surfaces_backend_error using
  a registry video model — the "custom" framing was incidental.
- Drop the misleading "accepts any model definition" file-level notes.
@AdirAmsalem AdirAmsalem merged commit 36445c7 into main May 6, 2026
9 checks passed
@AdirAmsalem AdirAmsalem deleted the rene/custom-realtime-models branch May 6, 2026 21:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant