Genkit Python SDK v0.11.0

Genkit Python SDK v0.11.0 Release Notes

Genkit Python SDK v0.11.0 is here. This release is about generate() keeping the reply when something goes wrong after the model has already answered, Veo as a background job you poll, and provider failures arriving as one GenkitError you can actually branch on.

uv add genkit genkit-google-genai

What's New

generate() returns the leftover instead of throwing (#6100, #6271)

generate() used to throw when the model refused, hit max tool turns, or wrote something that was not the schema you asked for. That discarded the reply. After this, you get the ModelResponse back. This is extremely helpful when you want to retry from the tool rounds that already succeeded.

from pydantic import BaseModel, Field
from genkit import Genkit
from genkit_google_genai import GoogleAI

ai = Genkit(
    plugins=[GoogleAI()],
    model=GoogleAI.gemini_model('gemini-flash-latest'),
)


class Recipe(BaseModel):
  title: str
  minutes: int = Field(description='time to cook')


response = await ai.generate(prompt='give me a recipe', output_schema=Recipe)
if response.output is not None:
  print(response.output.title)
else:
  # Still have the conversation. Send it back and try again.
  print(response.finish_reason, response.text)
  retry = await ai.generate(
      messages=response.messages,
      prompt='that was not a recipe. try again as JSON.',
      output_schema=Recipe,
  )
  print(retry.output)

Matching JSON is still stop, and .output is the Recipe. A leftover that is not a Recipe comes back as finish_reason=failed; the leftover stays on .text. A safety refusal stays blocked. Hitting max tool turns is aborted. A tool name that is not registered is failed.

.messages only keeps completed rounds (model message plus every tool response). An unanswered tool request is dropped, including the model message that opened it, because no provider will accept that history. response.message is None in those cases.

generate() still throws when there was no turn: empty messages, an unknown output format or model, or an output schema that is not JSON Schema. A tool that raises still raises.

Veo and background models (#6129, #6130, #6131, #6237)

Video is a job, not a round-trip. generate_operation starts it and hands back a ticket; check_operation is how you find out when the video is ready.

operation = await ai.generate_operation(
    model=GoogleAI.veo_model('veo-3.1-fast-generate-preview'),
    prompt='A paper airplane gliding through a bright classroom',
)
while not operation.done:
  operation = await ai.check_operation(operation)
print(operation.output.media[0].url)

generate() on a Veo model returns a ModelResponse whose .operation is the ticket and whose .message is None. It does not poll.

GoogleAI.veo_model / VertexAI.veo_model stamp the plugin namespace. On Vertex, a finished operation unpacks uri / GCS / inline bytes into a playable media.url. If safety filters drop every sample, that comes back as an error instead of an empty success.

check_operation and cancel_operation take a live Operation. Reload a persist dump with Operation.model_validate(dumped) first.

One GenkitError from every provider (#6098)

A Gemini 400, an OpenAI 429, an Ollama mid-stream failure, and an Anthropic 529 used to leak as whatever that SDK raises. Retry then retried a bad request, and your except had to know three exception types.

After this, plugins wrap provider HTTP errors as GenkitError. You write one branch. Retry already skips INVALID_ARGUMENT and waits out UNAVAILABLE / RESOURCE_EXHAUSTED, so a lot of that code goes away.

from genkit import Genkit, GenkitError
from genkit_google_genai import GoogleAI
from genkit_middleware import Retry

ai = Genkit(plugins=[GoogleAI()])
try:
  response = await ai.generate(
      model=GoogleAI.gemini_model('gemini-flash-latest'),
      prompt='Draft a weekend in Paris.',
      use=[Retry()],
  )
  print(response.text)
except GenkitError as err:
  # Same type whether this was Gemini, Claude, or GPT.
  print(err.status, err.original_message)
  if err.status == 'RESOURCE_EXHAUSTED' and err.response_metadata:
    print('retry after', err.response_metadata.get('retry_after_ms'), 'ms')

Status names, not raw HTTP codes:

• 400 → INVALID_ARGUMENT (retry skips)
• 429 + Retry-After: 60 → RESOURCE_EXHAUSTED (wait 60s, then retry)
• 503 → UNAVAILABLE (retry)
• Anthropic 529 → UNAVAILABLE (overloaded)
• Anthropic 404 → NOT_FOUND (Fallback can try the next model)

The reflection wire shows the provider text (original_message), not the SDK repr. Veo start() and check() polling are wrapped the same way.

Typed model refs (#6104, #6105, #6074, #6138)

model= now takes a name or a ModelRef. The constructor is what picks the config type, so a Gemini temperature and a Claude max_output_tokens cannot silently ride onto the wrong model. Same slot on Genkit(), generate, generate_stream, generate_operation, define_prompt, and agents.

from genkit import Genkit
from genkit_anthropic import Anthropic, AnthropicConfig
from genkit_google_genai import GeminiConfigSchema, GoogleAI, VertexAI
from genkit_openai import OpenAI, OpenAIConfig

flash = GoogleAI.gemini_model(
    'gemini-flash-latest',
    config=GeminiConfigSchema(temperature=0.2),
)
vertex_flash = VertexAI.gemini_model('gemini-flash-latest')
veo = GoogleAI.veo_model('veo-3.1-fast-generate-preview')
sonnet = Anthropic.claude_model(
    'claude-sonnet-4-5',
    config=AnthropicConfig(max_output_tokens=1024),
)
gpt = OpenAI.gpt_model('gpt-4o', config=OpenAIConfig(temperature=0.2))

ai = Genkit(plugins=[GoogleAI(), Anthropic(), OpenAI()], model=flash)
print((await ai.generate(model=flash, prompt='Say hi in one word.')).text)
print((await ai.generate(model=sonnet, prompt='Say hi in one word.')).text)
print((await ai.generate(model=gpt, prompt='Say hi in one word.')).text)

A pasted vertexai/… or models/… prefix is stripped and this plugin's namespace is stamped. Family constructors refuse ids they do not own (GoogleAI.gemini_model('veo-3.1-generate-001') tells you to use veo_model). Bedrock and Ollama still take a string name; they do not have a ref constructor yet.

A typed config object has to belong to the model this call is about to hit. A dict, None, or omitting config is left alone.

await ai.generate(model=flash, prompt='hi', config=OpenAIConfig())
# GenkitError: googleai/gemini-flash-latest: config must be
# genkit_google_genai.GeminiConfigSchema or a mapping, got genkit_openai.OpenAIConfig

Veneer config= is ModelConfigDict, so autocomplete works at generate() (#5989).

Debug logs in the Dev UI (#6099)

Under genkit start, generate() leaves named breadcrumbs on the span in the trace viewer's Logs panel: generate request resolved, calling model, executing tool requests, model responded. The terminal still honours GENKIT_LOG, so you can keep the shared TTY quiet and still see the trail next to the span.

Fixes & Polish

• Telemetry export no longer stalls generate() or dumps httpx tracebacks onto the shared genkit start TTY (#5978).
• The Python reflection server no longer sends a wildcard Access-Control-Allow-Origin (#6198).
• OpenAI gpt-image-1 no longer sends response_format (the endpoint rejects it). DALL-E still defaults to b64_json (#6167).
• OpenAI streaming now requests and populates token usage (include_usage) (#6229).
• Responses-API-only ids (gpt-5.1-codex, o3-pro, …) are gone from the OpenAI chat catalog. This plugin speaks Chat Completions (#6223).
• Whisper honours config={'translate': True} and routes to the translations API. gpt-4o-transcribe rejects translate as INVALID_ARGUMENT (#6168).

Existing API Changes

generate() no longer throws after the model has already replied. Code that caught a schema-mismatch, blocked, max-turns, or unknown-tool exception will not see it. Branch on the response:

# OLD (v0.10.0): leftover / blocked / max turns raised
try:
  response = await ai.generate(prompt='give me a recipe', output_schema=Recipe)
  recipe = response.output
except GenkitError:
  ...

# NEW (v0.11.0): the leftover is on the response
response = await ai.generate(prompt='give me a recipe', output_schema=Recipe)
if response.output is not None:
  recipe = response.output
else:
  print(response.finish_reason, response.text)
  print(response.messages)

Provider HTTP failures are GenkitError, not the raw SDK exception. Code that caught APIError / BadRequestError / APIStatusError from a plugin call will not see those types. Branch on err.status:

# OLD (v0.10.0): caught provider-specific HTTP exceptions
from google.genai.errors import ClientError
from openai import BadRequestError
try:
  response = await ai.generate(prompt='Draft a weekend in Paris.')
except (ClientError, BadRequestError) as err:
  ...

# NEW (v0.11.0): catch one GenkitError and branch on status
from genkit import GenkitError
try:
  response = await ai.generate(prompt='Draft a weekend in Paris.')
except GenkitError as err:
  if err.status == 'INVALID_ARGUMENT':
    ...

generate() on a Veo (or any background) model returns a ticket. .message is None. Use generate_operation / check_operation to poll:

# OLD (v0.10.0): background video models were unsupported
# NEW (v0.11.0): generate_operation returns a ticket; poll until done
operation = await ai.generate_operation(
    model=GoogleAI.veo_model('veo-3.1-fast-generate-preview'),
    prompt='A paper airplane gliding through a bright classroom',
)
while not operation.done:
  operation = await ai.check_operation(operation)
print(operation.output.media[0].url)

check_operation / cancel_operation take a live Operation. A persist dump is Operation.model_validate(...) first. Passing the dump, a boxed ModelResponse, or a str is INVALID_ARGUMENT:

# Rehydrate persisted dictionary before checking status
from genkit import Operation

dumped = op.model_dump()  # retrieved from database / queue
operation = await ai.check_operation(Operation.model_validate(dumped))

A typed config class that does not belong to the model is INVALID_ARGUMENT at generate / prompt / agent definition time:

# Raises GenkitError(INVALID_ARGUMENT)
await ai.generate(
    model=flash,
    prompt='hi',
    config=OpenAIConfig(temperature=0.2),
)
# => GenkitError: googleai/gemini-flash-latest: config must be
# genkit_google_genai.GeminiConfigSchema or a mapping, got genkit_openai.OpenAIConfig

Responses-only OpenAI ids no longer resolve from the chat catalog:

# OLD (v0.10.0): catalog included non-chat endpoints (gpt-5.1-codex, o3-pro)
# NEW (v0.11.0): catalog restricted to Chat Completions models
response = await ai.generate(
    model=OpenAI.gpt_model('gpt-4o'),
    prompt='Draft a weekend in Paris.',
)

Decisions

• 0.11.0 (not 0.10.1) because generate() no longer throws after a model reply, Veo/background is a new surface, and typed config rejection is a new INVALID_ARGUMENT.
• Notes live in the PR description (create_release copies that body). No CHANGELOG.md file.
• Version bump via py/bin/bump_version 0.11.0 only.