Skip to content

Feature/new models#841

Merged
lchoquel merged 13 commits into
devfrom
feature/New-models
Apr 28, 2026
Merged

Feature/new models#841
lchoquel merged 13 commits into
devfrom
feature/New-models

Conversation

@lchoquel
Copy link
Copy Markdown
Member

@lchoquel lchoquel commented Apr 28, 2026

Summary by cubic

Adds gpt-5.5 and gpt-image-2, moves image generation to a rule‑driven pipeline with parameter checks, and fixes model routing across openai/huggingface/fal. Also declares PDF input for GPT‑5.4/5.5 on azure_openai and the gateway. Error messages now include the model_name for clarity.

  • New Features

    • Added gpt-5.5 on openai, azure_openai, and the gateway (text/images/pdf).
    • Added gpt-image-2 with per‑model rules (no transparent background); updated alias best-gptgpt-image-2.
    • Introduced ImgGenParamSupport and PipeImgGen static validation; added e2e/integration/unit tests.
    • Declared pdf input for GPT‑5.4 series and gpt-5.5 on azure_openai and the gateway.
  • Refactors

    • Unified image‑gen arg building in ImgGenArgsFactory; OpenAI worker now rule‑driven with defaults quality=medium and output_compression=100; positive size validation for OpenAI img‑gen.
    • Renamed taxonomy MODEL_NAMEMODEL_CHOICE (supports model_id/model_name); standardized taxonomies (gpt_image vs gpt_image_legacy); added InputFidelity and OutputCompression; threaded model_name through all workers and errors, and now forward it to the arg factory.
    • Enforced model_choice = "model_id" on openai/huggingface/fal rules; workers require/pass the correct model (prevents OpenAI fallback to DALL‑E 2).
    • Improved Azure img‑gen reliability and observability (token usage capture, 600s timeout); refreshed gateway/docs and bumped remote config URL.
    • Removed TODOS.md.

Written for commit f93bed7. Summary will update on new commits. Review in cubic

lchoquel and others added 8 commits April 26, 2026 10:17
- Add gpt-image-2 to OpenAI direct and Azure OpenAI backends with
  background = "unavailable" rule (model rejects transparent bg)
- Generalize BackgroundTaxonomy: GPT -> AVAILABLE / UNAVAILABLE
- Refactor OpenAI img-gen worker to consult rules; defaults to
  AVAILABLE for models without a rules table so gpt-image-1 family
  keeps transparent support unchanged
- Update Azure rules tables for gpt-image-1 family to "available"
- Refresh auto-generated gateway model docs (gemini 3.x renames)

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…mpatibility

- Set `quality` default to `medium` for OpenAI direct image generation, aligning with Azure OpenAI.
- Introduce `output_compression` parameter for OpenAI models, defaulting to `100` for JPEG/WEBP outputs.
- Update TOML configuration files for both OpenAI and Azure backends to include `output_compression`.
- Modify `ImgGenArgsFactory` to handle new output compression logic and ensure compatibility with existing models.
- Enhance unit tests to cover new defaults and parameter handling, ensuring robust functionality across image generation workflows.
Convention: `gpt_image` for taxonomies/values shared across all OpenAI GPT
Image models (legacy + gpt-image-2); `gpt_image_legacy` when the value
applies only to gpt-image-1/-1-mini/-1.5 (gpt-image-2 = `unavailable`).
Local backend TOMLs and the remote Pipelex Gateway config updated in
lockstep.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- Parse usage tokens from Azure img gen response into job report
- Bump Azure img gen HTTP timeout to 600s for slow gpt-image-2 calls
- Pass model name in gateway img gen args; add debug logging
- Drop hardcoded gpt-image-2 model_id to use deck default
- Simplify img_gen test fixtures and prompt

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- Rename ImgGenArgTopic.MODEL_NAME → MODEL_CHOICE; ModelNameTaxonomy → ModelChoiceTaxonomy
- Replace STANDARD arm with MODEL_ID (passes model_id) and MODEL_NAME (passes model name)
- Thread model_name through ImgGenArgsFactory and all img-gen workers (Azure, OpenAI, Gateway, Fal, HuggingFace)
- Drop now-redundant model_choice rule from legacy gpt-image-* entries (default behavior)
- Gateway worker no longer overrides args_dict["model"] — driven by rules instead
- Refresh gateway model docs to include gpt-image-2
- Update unit tests for the new taxonomy and model_name plumbing

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds gpt-5.5 (Azure deployment gpt-5.5-2026-04-24, fixed_temperature=1)
on openai, azure_openai, and the gateway. Live LLM inference verified on
all three backends.

Also declares pdf in inputs for gpt-5.4, gpt-5.4-mini, gpt-5.4-nano,
gpt-5.4-pro, and gpt-5.5 on azure_openai and the gateway, matching the
existing OpenAI direct entries. PDF capability verified end-to-end with
TestLLMDocument (path + URL) on all three backends.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@lchoquel lchoquel requested a review from thomashebrard April 28, 2026 11:45
@lchoquel
Copy link
Copy Markdown
Member Author

@cubic-dev-ai

@cubic-dev-ai
Copy link
Copy Markdown

cubic-dev-ai Bot commented Apr 28, 2026

@cubic-dev-ai

@lchoquel I have started the AI code review. It will take a few minutes to complete.

@greptile-apps
Copy link
Copy Markdown

greptile-apps Bot commented Apr 28, 2026

Greptile Summary

  • Adds gpt-image-2 (OpenAI & Azure), gpt-5.5 LLM, and refactors the OpenAI image worker to a taxonomy-driven ImgGenArgsFactory pattern that also supports image-to-image via images.edit.
  • Introduces ImgGenParamSupport for static blueprint validation and InputFidelity / OutputCompression taxonomy types.
  • P1: All five OpenAI image model rule blocks in both openai.toml and azure_openai.toml are missing model_choice = \"model_id\". Because openai_img_gen_worker.py now delegates entirely to ImgGenArgsFactory, the model key is never inserted into args_dict, so every images.generate() / images.edit() call is sent without a model parameter.

Confidence Score: 3/5

Not safe to merge — all OpenAI image API calls will use the wrong default model or fail outright until model_choice is added to TOML rules.

A single P1 bug (missing model_choice in every OpenAI image model's TOML rules) means the model parameter is never passed to the OpenAI SDK, breaking all image generation and editing calls for this PR's new models. The rest of the refactor is well-structured.

pipelex/kit/configs/inference/backends/openai.toml and pipelex/kit/configs/inference/backends/azure_openai.toml — every [*.rules] block for image generation models needs model_choice = "model_id".

Important Files Changed

Filename Overview
pipelex/kit/configs/inference/backends/openai.toml Adds rules blocks for all 5 OpenAI image models but all rules are missing model_choice = "model_id", so model is never passed to the OpenAI SDK (P1 bug).
pipelex/kit/configs/inference/backends/azure_openai.toml Adds gpt-image-2 and updates existing image model rules; same missing model_choice issue as openai.toml. Also adds gpt-5.5 LLM and pdf input for gpt-5.4 series.
pipelex/cogt/img_gen/img_gen_args_factory.py New taxonomy-driven factory; passes model_id instead of model_name to error-message params in three places (P2). Logic is otherwise sound.
pipelex/plugins/openai/openai_img_gen_worker.py Refactored to use ImgGenArgsFactory and adds images.edit path for image-to-image; size fallback parsing and format detection look correct. Relies on rules having model_choice (which is missing from TOML configs).
pipelex/plugins/openai/openai_img_gen_factory.py Good refactor adding gpt-image-2 size validation; potential ZeroDivisionError if min_edge is 0 (P2). LEGACY_SIZE_TO_DIMENSIONS class var dict comprehension is correct.
pipelex/cogt/img_gen/img_gen_param_support.py New module providing yes/no parameter support checks; clean implementation with proper abstain-on-unknown-taxonomy behavior.
pipelex/cogt/img_gen/img_gen_model_rules.py Adds InputFidelityTaxonomy, OutputCompressionTaxonomy, ModelChoiceTaxonomy; renames/expands existing taxonomies for GPT Image 2. Clean additions.
pipelex/pipe_operators/img_gen/pipe_img_gen.py Adds static blueprint validation against model rules; correctly skips unresolvable presets and models without rules.
pipelex/plugins/azure_rest/azure_img_gen_worker.py Adds usage token extraction and increases timeout to 600s for gpt-image-2 generation time. Minor: model_name added to factory call.

Sequence Diagram

sequenceDiagram
    participant PipeImgGen
    participant ImgGenParamSupport
    participant OpenAIImgGenWorker
    participant ImgGenArgsFactory
    participant OpenAIAPI

    PipeImgGen->>ImgGenParamSupport: check_blueprint_params(rules, aspect_ratio, background, output_format)
    ImgGenParamSupport-->>PipeImgGen: unsupported reasons (or empty)
    PipeImgGen->>OpenAIImgGenWorker: _gen_image_list(img_gen_job, nb_images)
    OpenAIImgGenWorker->>ImgGenArgsFactory: make_args_for_model(rules, job, nb_images, model_id, model_name)
    ImgGenArgsFactory-->>OpenAIImgGenWorker: args_dict (prompt, size, quality, n, ...)
    alt has input images
        OpenAIImgGenWorker->>OpenAIAPI: images.edit(**args_dict)
    else text-to-image
        OpenAIImgGenWorker->>OpenAIAPI: images.generate(**args_dict)
    end
    OpenAIAPI-->>OpenAIImgGenWorker: ImagesResponse (b64_json, size, output_format)
    OpenAIImgGenWorker-->>PipeImgGen: list[GeneratedImageRawDetails]
Loading

Comments Outside Diff (1)

  1. pipelex/cogt/img_gen/img_gen_args_factory.py, line 95-113 (link)

    P2 model_id passed where model_name expected

    Lines 102, 111, and 178 all call make_args_from_* with model_name=model_id. The model_name argument is used exclusively in user-facing error messages (ImgGenParameterError). Since model_name (config name, e.g. "gpt-image-2") and model_id (provider ID, e.g. "gpt-image-2-2026-04-21") can differ, errors will cite the wrong identifier, making debugging harder. The model_name parameter was explicitly added to this function for this reason, so it should be forwarded correctly.

    Prompt To Fix With AI
    This is a comment left during a code review.
    Path: pipelex/cogt/img_gen/img_gen_args_factory.py
    Line: 95-113
    
    Comment:
    **`model_id` passed where `model_name` expected**
    
    Lines 102, 111, and 178 all call `make_args_from_*` with `model_name=model_id`. The `model_name` argument is used exclusively in user-facing error messages (`ImgGenParameterError`). Since `model_name` (config name, e.g. `"gpt-image-2"`) and `model_id` (provider ID, e.g. `"gpt-image-2-2026-04-21"`) can differ, errors will cite the wrong identifier, making debugging harder. The `model_name` parameter was explicitly added to this function for this reason, so it should be forwarded correctly.
    
    How can I resolve this? If you propose a fix, please make it concise.
Prompt To Fix All With AI
This is a comment left during a code review.
Path: pipelex/kit/configs/inference/backends/openai.toml
Line: 263-275

Comment:
**`model_choice` missing from all OpenAI image model rules**

None of the five new OpenAI image model rule blocks (`gpt-image-1`, `gpt-image-1-mini`, `gpt-image-1.5`, `gpt-image-2-2026-04-21`, `gpt-image-2`) include `model_choice = "model_id"`. Because `openai_img_gen_worker.py` now calls `openai_client.images.generate(**args_dict)` (and `.edit(**args_dict)`) entirely via the factory-built dict, and `ImgGenArgTopic.MODEL_CHOICE` is never hit, the `model` key is never added. The SDK call therefore omits the required `model` parameter, causing it to either fail or silently use an unintended default model. The same omission is present in all five matching blocks in `azure_openai.toml`. Adding `model_choice = "model_id"` to every `[*.rules]` block, the same way the unit-test helper `_make_gpt_image_2_rules()` already does, will fix this.

How can I resolve this? If you propose a fix, please make it concise.

---

This is a comment left during a code review.
Path: pipelex/cogt/img_gen/img_gen_args_factory.py
Line: 95-113

Comment:
**`model_id` passed where `model_name` expected**

Lines 102, 111, and 178 all call `make_args_from_*` with `model_name=model_id`. The `model_name` argument is used exclusively in user-facing error messages (`ImgGenParameterError`). Since `model_name` (config name, e.g. `"gpt-image-2"`) and `model_id` (provider ID, e.g. `"gpt-image-2-2026-04-21"`) can differ, errors will cite the wrong identifier, making debugging harder. The `model_name` parameter was explicitly added to this function for this reason, so it should be forwarded correctly.

How can I resolve this? If you propose a fix, please make it concise.

---

This is a comment left during a code review.
Path: pipelex/plugins/openai/openai_img_gen_factory.py
Line: 99-103

Comment:
**Potential division by zero in size validation**

`long_short_ratio = max_edge / min_edge` divides before the pixel-area floor check runs. If an `ImageSize` with `width=0` or `height=0` is supplied, `min_edge` is 0 and this raises `ZeroDivisionError` rather than the intended `ImgGenParameterError`. Adding a guard like `if min_edge == 0: raise ImgGenParameterError(...)` before this division would keep error handling consistent.

How can I resolve this? If you propose a fix, please make it concise.

Reviews (1): Last reviewed commit: "cleanup TODO" | Re-trigger Greptile

Comment thread pipelex/kit/configs/inference/backends/openai.toml
Comment thread pipelex/plugins/openai/openai_img_gen_factory.py
Copy link
Copy Markdown

@cubic-dev-ai cubic-dev-ai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

2 issues found across 34 files

Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="pipelex/plugins/openai/openai_img_gen_worker.py">

<violation number="1" location="pipelex/plugins/openai/openai_img_gen_worker.py:69">
P2: Don’t drop the `moderation` argument for `images.edit`. The OpenAI API supports `moderation` on edits, so this removes a requested safety setting.</violation>
</file>

<file name="pipelex/cogt/img_gen/img_gen_param_support.py">

<violation number="1" location="pipelex/cogt/img_gen/img_gen_param_support.py:196">
P2: `check_blueprint_params` omits `input_fidelity` validation, so unsupported blueprint `input_fidelity` values are not caught during static blueprint checks.</violation>
</file>

Reply with feedback, questions, or to request a fix. Tag @cubic-dev-ai to re-run a review.

Comment thread pipelex/plugins/openai/openai_img_gen_worker.py
Comment thread pipelex/cogt/img_gen/img_gen_param_support.py
Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 19bae2c459

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread pipelex/plugins/openai/openai_img_gen_worker.py
Comment thread pipelex/plugins/openai/openai_img_gen_factory.py
lchoquel and others added 2 commits April 28, 2026 14:55
OpenAI img-gen was silently falling back to DALL-E 2: rules omitted
model_choice, so args_dict reached images.generate() without "model".
HF and Fal workers hardcoded inference_model.model_id and ignored
the rule entirely.

- Declare `model_choice = "model_id"` on OpenAI gpt-image-* rules,
  HF qwen-image, and all 5 Fal models — in both kit configs and
  the .pipelex/ project overrides.
- HF and Fal workers now pop "model" from args_dict (mirroring
  prompt) and raise ImgGenParameterError if missing.
- Auto-regenerated gateway model docs reflect PDF inputs on the
  gpt-5.4 series and gpt-5.5.

Verified live against fal/fast-lightning-sdxl,
huggingface/qwen-image, google/nano-banana.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
thomashebrard
thomashebrard previously approved these changes Apr 28, 2026
lchoquel and others added 2 commits April 28, 2026 15:27
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@lchoquel lchoquel merged commit 9ae38b6 into dev Apr 28, 2026
27 checks passed
@github-actions github-actions Bot locked and limited conversation to collaborators Apr 28, 2026
@lchoquel lchoquel deleted the feature/New-models branch April 28, 2026 13:47
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants