Skip to content

feat(llm): add image/audio output caps, ImageGenOptions, public OneChunkStream#71

Merged
lIang70 merged 1 commit into
mainfrom
feat/llm-image-output-caps
May 7, 2026
Merged

feat(llm): add image/audio output caps, ImageGenOptions, public OneChunkStream#71
lIang70 merged 1 commit into
mainfrom
feat/llm-image-output-caps

Conversation

@lIang70

@lIang70 lIang70 commented May 7, 2026

Copy link
Copy Markdown
Collaborator

Summary

Extends sdk/llm with the primitives needed by upcoming
image-generation adapters in sdkx (MiniMax / Seedream /
Qwen-Image), without coupling sdk/llm to any specific provider.

  • New capabilities CapImageOutput / CapAudioOutput so output
    modalities are declarable in the blacklist-based ModelCaps
    matrix. Existing chat-only catalogs will get these disabled in a
    follow-up sdkx PR.
  • ImageGenOptions + WithImageGen() carry size / aspect ratio
    / N / seed / response-format knobs through
    llm.GenerateOptions.
  • NewOneChunkStream extracts the previously-private
    oneChunkStream helper from with_caps.go so synchronous-only
    adapters can satisfy llm.StreamMessage with a single shared
    implementation. with_caps.go now delegates to the public
    helper.

Why this PR exists

Three image-generation providers are about to land in sdkx. They
all need:

  1. A way to declare image-output capability without inventing a
    parallel sdk/image interface.
  2. A typed channel for image-specific knobs (size etc.) that the
    text-oriented LLM interface lacks.
  3. A canonical wrapper to expose synchronous image APIs as a
    single-chunk stream so llm.StreamMessage callers don't break.

Keeping these in sdk/llm (rather than copy-pasting per adapter)
makes the matrix consistent and unblocks sdkx.

Test plan

  • go test ./sdk/llm/... -count=1 -race — covers the new
    onechunk_test.go (text, tool calls, multimodal parts) and
    the existing with_caps_test.go which now exercises the
    public helper via the streaming-downgrade path.
  • go vet ./sdk/...

Follow-ups

  • sdkx: catalog updates (disable new caps on chat models) + the
    three image adapters that consume ImageGenOptions and
    NewOneChunkStream.
  • tests/conformance: live API integration tests for the image
    providers (gated on env vars).

Made with Cursor

…unkStream

Extends sdk/llm with the primitives needed by image-generation
adapters in sdkx (MiniMax / Seedream / Qwen-Image) without coupling
sdk/llm to any specific provider:

- New capabilities CapImageOutput / CapAudioOutput so output
  modalities are declarable in the blacklist-based ModelCaps
  matrix; chat-only catalogs get them disabled in a follow-up
  sdkx PR.
- ImageGenOptions + WithImageGen() carry size / aspect-ratio /
  N / seed / response-format through llm.GenerateOptions.
- NewOneChunkStream extracts the previously-private
  oneChunkStream helper from with_caps.go so synchronous-only
  adapters can satisfy llm.StreamMessage with a single shared
  implementation. with_caps.go now delegates to the public
  helper.

Co-authored-by: Cursor <cursoragent@cursor.com>
@lIang70
lIang70 merged commit 09c05f2 into main May 7, 2026
17 checks passed
@lIang70
lIang70 deleted the feat/llm-image-output-caps branch May 7, 2026 10:27
lIang70 added a commit that referenced this pull request May 7, 2026
sdkx now requires sdk v0.2.8 (PR #71); the local replace makes the
e2e module pick up that requirement transitively, so its indirect
line needs syncing. CI was failing the 'go: updates to go.mod
needed' check before this.

Co-authored-by: Cursor <cursoragent@cursor.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant