feat(tests/conformance): add image-generation conformance suite#73
Merged
Conversation
Live-API conformance scenarios for the three image-generation adapters added in sdkx v0.2.6 (#72): minimax-image, bytedance-image, qwen-image. Each test self-skips when its FLOWCRAFT_TEST_*_IMAGE env var is unset, so credential-less runs are a no-op (matches the chat conformance pattern). Five scenarios across the three providers: - Basic text-to-image (all three). - Explicit Width/Height routing — verifies the per-family size syntax (WxH for MiniMax/Seedream, W*H for Qwen) and known-valid dimensions per provider (1024x1024 / 2048x2048 / 1024x1024). - Streaming via NewOneChunkStream — validates the StreamMessage contract (single chunk + final message carrying image parts). - Image-to-image with a reference URL (minimax-image, bytedance-image; reference image hosted on Aliyun help-static CDN so it stays reachable from China-region inference servers). - qwen-image rejection of PartImage inputs — pins the t2i-only contract; image editing lives on a separate qwen-image-edit endpoint not yet adapted. Optional offline review path: setting SAVE_GENERATED_IMAGES=1 downloads every returned image to tests/conformance/llm/_out/ (directory git-ignored) for visual inspection. Off by default so CI and credential-less runs stay disk-clean. Bumps the conformance module's sdk pin to v0.2.8 (#71) and sdkx pin to v0.2.6 (#72), and drops the local replace directives that were temporarily added while those PRs were in flight. The module now exclusively tests against released versions, restoring the README's stated "isolate dependency graph from sdk/sdkx releases" property. Co-authored-by: Cursor <cursoragent@cursor.com>
…itignore Tiny housekeeping: the per-example ignores examples/voice-pipeline/voice-pipeline examples/chatbot-with-recall/chatbot-with-recall were lodged at the repo root, which couples adding/renaming an example with editing the top-level .gitignore. Moves them into a local examples/.gitignore so future examples can manage their build artefacts in isolation. Net behaviour identical. Co-authored-by: Cursor <cursoragent@cursor.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Live-API conformance scenarios for the three image-generation
adapters that landed in
sdkx/v0.2.6via #72:minimax-image/bytedance-image/qwen-image.Each test self-skips when its
FLOWCRAFT_TEST_*_IMAGEenv var isunset, so credential-less runs (CI default) are a no-op. Same
pattern as the existing chat conformance suite.
Scenarios
BasicTextToImageExplicitDimensionsWidth/Heightrouting —WxH(MiniMax / Seedream) vsW*H(Qwen)StreamingNewOneChunkStreamcontract: single chunk + final messageImageToImageQwenRejectsImageInputOptional offline review
SAVE_GENERATED_IMAGES=1 make test-conformancedownloads everyreturned image into
tests/conformance/llm/_out/(git-ignored)for visual inspection. Off by default, so CI / credential-less
runs stay disk-clean.
Dependency / replace housekeeping
sdkpin:v0.2.6→v0.2.8(consumes feat(llm): add image/audio output caps, ImageGenOptions, public OneChunkStream #71'sCapImageOutput/ImageGenOptions/NewOneChunkStream).sdkxpin:v0.2.2→v0.2.6(consumes feat(sdkx/llm): add image-generation adapters + align chat caps #72's three imageadapters).
replacedirectives that were temporarilyadded while feat(llm): add image/audio output caps, ImageGenOptions, public OneChunkStream #71 / feat(sdkx/llm): add image-generation adapters + align chat caps #72 were in flight. The module now
exclusively tests against released versions, restoring the
README's stated "isolate dependency graph from sdk/sdkx
releases" property.
Bonus housekeeping (separate commit)
chore(.gitignore): relocate per-example binary ignores— movesexamples/voice-pipeline/voice-pipelineandexamples/chatbot-with-recall/chatbot-with-recallfrom thetop-level
.gitignoreinto a newexamples/.gitignoreso futureexamples can manage their build artefacts in isolation. Net
behaviour identical.
Test plan
cd tests/conformance && GOWORK=off go vet ./...cd tests/conformance && GOWORK=off go test -count=1 ./...(creds-less, all skip cleanly)
SAVE_GENERATED_IMAGES=1produced and visually verified the11 sample images.
Made with Cursor