Skip to content

Add inference models & template optional insertion#32

Merged
lchoquel merged 18 commits intodevfrom
feature/Add-inference-models
Jun 2, 2025
Merged

Add inference models & template optional insertion#32
lchoquel merged 18 commits intodevfrom
feature/Add-inference-models

Conversation

@lchoquel
Copy link
Copy Markdown
Member

@lchoquel lchoquel commented Jun 1, 2025

Features

  • Template preprocessor new token prefix @? for optional insertion: if the variable doesn't exist in the context, nothing is inserted (instead of raising an exception).

Models

  • Added Claude 4 Opus and Sonnet, usable through the Anthropic SDK at Anthropic and Bedrock and usable with the Bedrock SDK. Includes a specific setting to reduce the max_tokens limit to avoid a timeout/streaming issue yet to be addressed later.
  • Added support for the Grok 3 family through the OpenAI SDK.
  • Added support for GPT4o image generation through the OpenAI SDK, with the gpt-image-1 model. Available through PipeImgGen as an alternative to the Forest Lab (FAL) supported models. Currently saves local files, to be addressed.
  • Added latest gemini-2.5-pro

Base Library

  • Added Pipe: extract_page_contents_and_views_from_pdf which not only grabs the text and linked images from PDF but also includes page_view images, i.e. pages rendered as images (very useful for using Vision down the line). This one used to be defined in the cookbook samples so, congrats on the promotion.

Refactor

  • Improved error handling in several places regarding inference
  • Moved cogt tests into the tests/pipelex so they can share fixtures, currently used for imgg_handle (image generation handle)

Fixes

  • Fixed integration of Perplexity, broken by recent changes

Dependencies

  • Updated instructor to 1.8.3
  • Dev: replaced pytest-pretty by pytest-sugar because pytest-pretty truncates test-case names just to make it fit into a pretty rich table. Function before fashion.

🔄 Type of Change

  • 🐛 Bug fix
  • ✨ New feature
  • 💥 Breaking change
  • 📚 Documentation update
  • 🧹 Code refactor
  • ⚡ Performance improvement
  • ✅ Test update

🧪 Tests

  • TestTemplatePreprocessor
  • TestImggByOpenAIGpt
  • TestImageGeneration
  • TestPipeImgg

@lchoquel lchoquel requested a review from thomashebrard June 1, 2025 11:05
Copy link
Copy Markdown
Member

@thomashebrard thomashebrard left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Really nice!
non breaking remarks

Comment on lines +41 to +45
background: Background = Field(strict=False)
quality: Optional[Quality] = Field(default=None, strict=False)
nb_steps: Optional[int] = Field(default=None, gt=0)
guidance_scale: float = Field(..., gt=0)
is_safety_checker_enabled: bool
is_moderated: bool
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe this model would need docstrings to explain the fields

from pipelex.pipe_operators.pipe_operator import PipeOperator

# TODO: refacto this as part of the PipeImgGen blueprint/params
IMGG_PROMPT_NAME = "imgg_prompt"
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"imgg" is kind of confusing. Hard to guess it means “image generation"

@lchoquel lchoquel merged commit e76c9b8 into dev Jun 2, 2025
11 checks passed
@github-actions github-actions Bot locked and limited conversation to collaborators Jun 2, 2025
@lchoquel lchoquel deleted the feature/Add-inference-models branch June 3, 2025 09:43
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants