fix(py): add custom evaluator sample to match JS by MengqinShen · Pull Request #4670 · genkit-ai/genkit

MengqinShen · 2026-02-14T08:09:26Z

Ported static .prompt test fixtures from JavaScript to Python and created the framework-custom-evaluators sample. Refactored the model configuration to match JavaScript patterns (model-in-code rather than model-in-prompt) and ensured full Dev UI support.

gemini-code-assist · 2026-02-14T08:10:00Z

Summary of Changes

Hello @MengqinShen, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request significantly enhances the Python Genkit framework by integrating static prompt definitions and introducing a robust sample for custom evaluators. The changes aim to bring the Python implementation closer to its JavaScript counterpart, particularly in how models are configured and prompts are managed. This includes porting existing prompt test fixtures and establishing a new sample that showcases various evaluation techniques, all while ensuring full compatibility and visibility within the Genkit Dev UI.

Highlights

Static Prompt Fixtures Ported to Python: The pull request ports static .prompt test fixtures from JavaScript to Python, enabling the definition of prompts in external files and enhancing consistency across language implementations.
New Custom Evaluator Sample Introduced: A new framework-custom-evaluators sample has been created, showcasing both LLM-based (PII detection, funniness, deliciousness) and regex-based custom evaluators within the Genkit framework.
Model Configuration Refactoring: Model configuration within the Python framework has been refactored to align with JavaScript's 'model-in-code' pattern, improving the clarity and maintainability of model definitions.
Enhanced Dev UI Support: Full Dev UI support has been ensured for the newly added static prompts and custom evaluators, providing a seamless development experience.

🧠 New Feature in Public Preview: You can now enable Memory to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console.

Changelog

py/packages/genkit/tests/genkit/blocks/prompt_test.py
- Added a new test case test_load_static_prompts to validate the loading and rendering of static prompt files, including those with configurations, tools, and subdirectories.
py/packages/genkit/tests/genkit/blocks/prompts/badSchemaRef.prompt
- Added a static prompt file to test schema reference handling.
py/packages/genkit/tests/genkit/blocks/prompts/chat_preamble.prompt
- Added a static prompt file for chat preamble testing.
py/packages/genkit/tests/genkit/blocks/prompts/kitchensink.prompt
- Added a comprehensive static prompt file demonstrating various configurations like model, tools, output format, and templating.
py/packages/genkit/tests/genkit/blocks/prompts/output.prompt
- Added a static prompt file to test output schema.
py/packages/genkit/tests/genkit/blocks/prompts/schemaRef.prompt
- Added a static prompt file to test schema references.
py/packages/genkit/tests/genkit/blocks/prompts/sub/test.prompt
- Added a static prompt file within a subdirectory to test nested prompt loading.
py/packages/genkit/tests/genkit/blocks/prompts/test.prompt
- Added a basic static prompt file for general testing.
py/packages/genkit/tests/genkit/blocks/prompts/test.variant.prompt
- Added a static prompt file to test prompt variants.
py/packages/genkit/tests/genkit/blocks/prompts/toolPrompt.prompt
- Added a static prompt file to test tool integration.
py/pyproject.toml
- Registered the new framework-custom-evaluators sample as a workspace dependency.
py/samples/framework-custom-evaluators/README.md
- Added documentation for the custom evaluators sample, detailing setup, evaluators, and testing instructions.
py/samples/framework-custom-evaluators/datasets/deliciousness_dataset.json
- Added a dataset for testing the deliciousness evaluator.
py/samples/framework-custom-evaluators/datasets/funniness_dataset.json
- Added a dataset for testing the funniness evaluator.
py/samples/framework-custom-evaluators/datasets/pii_detection_dataset.json
- Added a dataset for testing the PII detection evaluator.
py/samples/framework-custom-evaluators/datasets/regex_dataset.json
- Added a dataset for testing the regex evaluators.
py/samples/framework-custom-evaluators/local.env.example
- Added an example environment file for local development.
py/samples/framework-custom-evaluators/prompts/deliciousness.prompt
- Added a prompt definition for the LLM-based deliciousness evaluator.
py/samples/framework-custom-evaluators/prompts/funniness.prompt
- Added a prompt definition for the LLM-based funniness evaluator.
py/samples/framework-custom-evaluators/prompts/pii_detection.prompt
- Added a prompt definition for the LLM-based PII detection evaluator.
py/samples/framework-custom-evaluators/pyproject.toml
- Defined project metadata and dependencies for the custom evaluators sample.
py/samples/framework-custom-evaluators/run.sh
- Added a shell script to run the custom evaluators sample, including dependency installation and auto-restart.
py/samples/framework-custom-evaluators/src/init.py
- Added package initialization for the custom evaluators sample.
py/samples/framework-custom-evaluators/src/constants.py
- Added constants for regex patterns and permissive safety settings used by evaluators.
py/samples/framework-custom-evaluators/src/deliciousness_evaluator.py
- Implemented the LLM-based deliciousness evaluator.
py/samples/framework-custom-evaluators/src/funniness_evaluator.py
- Implemented the LLM-based funniness evaluator.
py/samples/framework-custom-evaluators/src/main.py
- Implemented the main application logic for the custom evaluators sample, initializing Genkit and registering all evaluators.
py/samples/framework-custom-evaluators/src/pii_evaluator.py
- Implemented the LLM-based PII detection evaluator.
py/samples/framework-custom-evaluators/src/regex_evaluator.py
- Implemented a factory for regex-based evaluators.
py/samples/framework-prompt-demo/prompts/recipe.robot.prompt
- Added a new prompt variant for robot-themed recipes.
py/samples/framework-prompt-demo/src/main.py
- Updated the main application to include and demonstrate the new robot_chef_flow and updated the main execution block to call this new flow.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	`/gemini review`	Performs a code review for the current pull request in its current state.
Pull Request Summary	`/gemini summary`	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	`/gemini help`	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

gemini-code-assist

Code Review

This pull request is a great addition, aligning the Python version of Genkit with the JavaScript version by adding static prompt fixtures and a new sample for custom evaluators. The new tests for static prompts are comprehensive. The custom evaluators sample is well-structured and provides clear examples of both LLM-based and non-LLM evaluators. I've found a few areas for improvement, mainly in the prompt files and configuration of the new sample, to enhance robustness and correctness. My detailed comments are below.

…tch JS

… in the framework-custom-evaluators

# Release v0.6.0 This release includes **67** package(s). | Package | Version heading | |---------|----------------| | `conform` | 0.2.0 (2026-02-17) | | `dev-local-vectorstore-hello` | 0.2.0 (2026-02-17) | | `framework-context-demo` | 0.2.0 (2026-02-17) | | `framework-custom-evaluators` | 0.1.0 (2026-02-17) | | `framework-dynamic-tools-demo` | 0.2.0 (2026-02-17) | | `framework-evaluator-demo` | 0.1.0 (2026-02-17) | | `framework-format-demo` | 0.2.0 (2026-02-17) | | `framework-middleware-demo` | 0.2.0 (2026-02-17) | | `framework-prompt-demo` | 0.1.0 (2026-02-17) | | `framework-realtime-tracing-demo` | 0.2.0 (2026-02-17) | | `framework-restaurant-demo` | 0.2.0 (2026-02-17) | | `framework-tool-interrupts` | 0.2.0 (2026-02-17) | | `genkit` | 0.6.0 (2026-02-17) | | `genkit-plugin-amazon-bedrock` | 0.6.0 (2026-02-17) | | `genkit-plugin-anthropic` | 0.6.0 (2026-02-17) | | `genkit-plugin-checks` | 0.6.0 (2026-02-17) | | `genkit-plugin-cloudflare-workers-ai` | 0.6.0 (2026-02-17) | | `genkit-plugin-cohere` | 0.6.0 (2026-02-17) | | `genkit-plugin-compat-oai` | 0.6.0 (2026-02-17) | | `genkit-plugin-deepseek` | 0.6.0 (2026-02-17) | | `genkit-plugin-dev-local-vectorstore` | 0.6.0 (2026-02-17) | | `genkit-plugin-evaluators` | 0.6.0 (2026-02-17) | | `genkit-plugin-fastapi` | 0.6.0 (2026-02-17) | | `genkit-plugin-firebase` | 0.6.0 (2026-02-17) | | `genkit-plugin-flask` | 0.6.0 (2026-02-17) | | `genkit-plugin-google-cloud` | 0.6.0 (2026-02-17) | | `genkit-plugin-google-genai` | 0.6.0 (2026-02-17) | | `genkit-plugin-huggingface` | 0.6.0 (2026-02-17) | | `genkit-plugin-mcp` | 0.6.0 (2026-02-17) | | `genkit-plugin-microsoft-foundry` | 0.6.0 (2026-02-17) | | `genkit-plugin-mistral` | 0.6.0 (2026-02-17) | | `genkit-plugin-observability` | 0.6.0 (2026-02-17) | | `genkit-plugin-ollama` | 0.6.0 (2026-02-17) | | `genkit-plugin-vertex-ai` | 0.6.0 (2026-02-17) | | `genkit-plugin-xai` | 0.6.0 (2026-02-17) | | `genkit-tools-model-config-test` | 0.2.0 (2026-02-17) | | `genkit-tools-sample-flows` | 0.1.1 (2026-02-17) | | `provider-amazon-bedrock-hello` | 0.2.0 (2026-02-17) | | `provider-anthropic-hello` | 0.2.0 (2026-02-17) | | `provider-checks-hello` | 0.2.0 (2026-02-17) | | `provider-cloudflare-workers-ai-hello` | 0.2.0 (2026-02-17) | | `provider-cohere-hello` | 0.2.0 (2026-02-17) | | `provider-compat-oai-hello` | 0.2.0 (2026-02-17) | | `provider-deepseek-hello` | 0.2.0 (2026-02-17) | | `provider-firestore-retriever` | 0.2.0 (2026-02-17) | | `provider-google-genai-code-execution` | 0.2.0 (2026-02-17) | | `provider-google-genai-context-caching` | 0.2.0 (2026-02-17) | | `provider-google-genai-hello` | 0.2.0 (2026-02-17) | | `provider-google-genai-media-models-demo` | 0.2.0 (2026-02-17) | | `provider-google-genai-vertexai-hello` | 0.2.0 (2026-02-17) | | `provider-google-genai-vertexai-image` | 0.2.0 (2026-02-17) | | `provider-huggingface-hello` | 0.2.0 (2026-02-17) | | `provider-microsoft-foundry-hello` | 0.1.0 (2026-02-17) | | `provider-mistral-hello` | 0.2.0 (2026-02-17) | | `provider-observability-hello` | 0.2.0 (2026-02-17) | | `provider-ollama-hello` | 0.2.0 (2026-02-17) | | `provider-vertex-ai-model-garden` | 0.2.0 (2026-02-17) | | `provider-vertex-ai-rerank-eval` | 0.2.0 (2026-02-17) | | `provider-vertex-ai-vector-search-bigquery` | 0.2.0 (2026-02-17) | | `provider-vertex-ai-vector-search-firestore` | 0.2.0 (2026-02-17) | | `provider-xai-hello` | 0.2.0 (2026-02-17) | | `releasekit` | 0.2.0 (2026-02-17) | | `web-endpoints-hello` | 0.2.0 (2026-02-17) | | `web-fastapi-bugbot` | 0.2.0 (2026-02-17) | | `web-flask-hello` | 0.2.0 (2026-02-17) | | `web-multi-server` | 0.2.0 (2026-02-17) | | `web-short-n-long` | 0.2.0 (2026-02-17) | ## Changelogs <details><summary>genkit-tools-sample-flows</summary> ## 0.1.1 (2026-02-17) ### Bug Fixes - **py**: relocate tools of model-config test and sample-flow test (1043245, #4669) — @Elisa Shen </details> <details><summary>genkit-tools-model-config-test</summary> ## 0.2.0 (2026-02-17) ### Features - **releasekit**: genkit.ai (f2dde35, #4702) — @Yesudeep Mangalapilly ### Bug Fixes - **py**: relocate tools of model-config test and sample-flow test (1043245, #4669) — @Elisa Shen </details> <details><summary>framework-custom-evaluators</summary> ## 0.1.0 (2026-02-17) ### Features - **releasekit**: supply-chain security, multi-ecosystem orchestration, and CI hardening (29d3ec1, #4682) — @Yesudeep Mangalapilly ### Bug Fixes - **py**: add custom evaluator sample to match JS (3472939, #4670) — @Elisa Shen </details> <details><summary>genkit-plugin-fastapi</summary> ## 0.6.0 (2026-02-17) ### Features - **py**: Add fastAPI plugin and sample (417158d, #4543) — @huangjeff5 ### Bug Fixes - **py**: address releasekit check warnings for metadata and grouping (4f5a910, #4595) — @Yesudeep Mangalapilly - **ci**: enable releasekit check in bin/lint and PR workflow (f245d6b, #4590) — @Yesudeep Mangalapilly </details> <details><summary>provider-checks-hello</summary> ## 0.2.0 (2026-02-17) ### Features - **releasekit**: add Forge protocol extensions, transitive propagation, and multi-backend conformance (d6dbb44, #4577) — @Yesudeep Mangalapilly - **py/checks**: add Google Checks AI Safety plugin (a8bf3c1, #4504) — @Yesudeep Mangalapilly ### Bug Fixes - **py**: resolve CI license check failures and lint diagnostics (e8519ef, #4524) — @Yesudeep Mangalapilly </details> <details><summary>web-fastapi-bugbot</summary> ## 0.2.0 (2026-02-17) ### Features - **py**: Add fastAPI plugin and sample (417158d, #4543) — @huangjeff5 ### Bug Fixes - **py**: handle nullable JSON Schema types in Gemini plugin and clean up samples (4daef62, #4629) — @Yesudeep Mangalapilly - **py**: address releasekit check warnings for metadata and grouping (4f5a910, #4595) — @Yesudeep Mangalapilly - **ci**: enable releasekit check in bin/lint and PR workflow (f245d6b, #4590) — @Yesudeep Mangalapilly </details> <details><summary>provider-cohere-hello</summary> ## 0.2.0 (2026-02-17) ### Features - **releasekit**: add Forge protocol extensions, transitive propagation, and multi-backend conformance (d6dbb44, #4577) — @Yesudeep Mangalapilly - **py/plugins**: add Cohere provider plugin (e424dcd, #4518) — @Yesudeep Mangalapilly ### Bug Fixes - **py**: add missing LICENSE file and license metadata to samples (4ddd8a3, #4571) — @Yesudeep Mangalapilly ### Refactoring - **py/plugins**: extract converters, add tests, community labeling (ebd0a2e, #4520) — @Yesudeep Mangalapilly </details> <details><summary>web-flask-hello</summary> ## 0.2.0 (2026-02-17) ### Features - **releasekit**: add Forge protocol extensions, transitive propagation, and multi-backend conformance (d6dbb44, #4577) — @Yesudeep Mangalapilly - **py/samples**: add web-endpoints-hello — REST + gRPC kitchen-sink sample (8614e5e, #4498) — @Yesudeep Mangalapilly ### Refactoring - **py/samples**: standardize naming taxonomy, consolidate shared logic, and close feature coverage gaps (1996c7c, #4488) — @Yesudeep Mangalapilly ### Documentation - **py**: update readmes for all samples and plugins (e02ea9f, #4624) — @Yesudeep Mangalapilly </details> <details><summary>framework-format-demo</summary> ## 0.2.0 (2026-02-17) ### Features - **releasekit**: add Forge protocol extensions, transitive propagation, and multi-backend conformance (d6dbb44, #4577) — @Yesudeep Mangalapilly - **py/samples**: add web-endpoints-hello — REST + gRPC kitchen-sink sample (8614e5e, #4498) — @Yesudeep Mangalapilly ### Refactoring - **py/samples**: standardize naming taxonomy, consolidate shared logic, and close feature coverage gaps (1996c7c, #4488) — @Yesudeep Mangalapilly ### Documentation - **py**: update readmes for all samples and plugins (e02ea9f, #4624) — @Yesudeep Mangalapilly </details> <details><summary>framework-context-demo</summary> ## 0.2.0 (2026-02-17) ### Features - **releasekit**: add Forge protocol extensions, transitive propagation, and multi-backend conformance (d6dbb44, #4577) — @Yesudeep Mangalapilly - **py/samples**: add web-endpoints-hello — REST + gRPC kitchen-sink sample (8614e5e, #4498) — @Yesudeep Mangalapilly ### Refactoring - **py/samples**: standardize naming taxonomy, consolidate shared logic, and close feature coverage gaps (1996c7c, #4488) — @Yesudeep Mangalapilly ### Documentation - **py**: update readmes for all samples and plugins (e02ea9f, #4624) — @Yesudeep Mangalapilly </details> <details><summary>framework-middleware-demo</summary> ## 0.2.0 (2026-02-17) ### Features - **releasekit**: add Forge protocol extensions, transitive propagation, and multi-backend conformance (d6dbb44, #4577) — @Yesudeep Mangalapilly - **py/samples**: add web-endpoints-hello — REST + gRPC kitchen-sink sample (8614e5e, #4498) — @Yesudeep Mangalapilly ### Refactoring - **py/samples**: standardize naming taxonomy, consolidate shared logic, and close feature coverage gaps (1996c7c, #4488) — @Yesudeep Mangalapilly ### Documentation - **py**: update readmes for all samples and plugins (e02ea9f, #4624) — @Yesudeep Mangalapilly </details> <details><summary>provider-firestore-retriever</summary> ## 0.2.0 (2026-02-17) ### Features - **releasekit**: add Forge protocol extensions, transitive propagation, and multi-backend conformance (d6dbb44, #4577) — @Yesudeep Mangalapilly - **py/samples**: add web-endpoints-hello — REST + gRPC kitchen-sink sample (8614e5e, #4498) — @Yesudeep Mangalapilly ### Bug Fixes - **py**: migrate default embedding model to gemini-embedding-001 (051f75f, #4557) — @Elisa Shen ### Refactoring - **py/samples**: standardize naming taxonomy, consolidate shared logic, and close feature coverage gaps (1996c7c, #4488) — @Yesudeep Mangalapilly </details> <details><summary>framework-dynamic-tools-demo</summary> ## 0.2.0 (2026-02-17) ### Features - **releasekit**: add Forge protocol extensions, transitive propagation, and multi-backend conformance (d6dbb44, #4577) — @Yesudeep Mangalapilly - **py/samples**: add web-endpoints-hello — REST + gRPC kitchen-sink sample (8614e5e, #4498) — @Yesudeep Mangalapilly ### Refactoring - **py/samples**: standardize naming taxonomy, consolidate shared logic, and close feature coverage gaps (1996c7c, #4488) — @Yesudeep Mangalapilly ### Documentation - **py**: update readmes for all samples and plugins (e02ea9f, #4624) — @Yesudeep Mangalapilly </details> <details><summary>provider-ollama-hello</summary> ## 0.2.0 (2026-02-17) ### Features - **releasekit**: add Forge protocol extensions, transitive propagation, and multi-backend conformance (d6dbb44, #4577) — @Yesudeep Mangalapilly - **py/samples**: add web-endpoints-hello — REST + gRPC kitchen-sink sample (8614e5e, #4498) — @Yesudeep Mangalapilly ### Bug Fixes - **py**: resolve CI license check failures and lint diagnostics (e8519ef, #4524) — @Yesudeep Mangalapilly ### Refactoring - **py/samples**: standardize naming taxonomy, consolidate shared logic, and close feature coverage gaps (1996c7c, #4488) — @Yesudeep Mangalapilly </details> <details><summary>provider-google-genai-code-execution</summary> ## 0.2.0 (2026-02-17) ### Features - **releasekit**: add Forge protocol extensions, transitive propagation, and multi-backend conformance (d6dbb44, #4577) — @Yesudeep Mangalapilly - **py/samples**: add web-endpoints-hello — REST + gRPC kitchen-sink sample (8614e5e, #4498) — @Yesudeep Mangalapilly ### Refactoring - **py/samples**: standardize naming taxonomy, consolidate shared logic, and close feature coverage gaps (1996c7c, #4488) — @Yesudeep Mangalapilly ### Documentation - **py**: update readmes for all samples and plugins (e02ea9f, #4624) — @Yesudeep Mangalapilly </details> <details><summary>provider-google-genai-context-caching</summary> ## 0.2.0 (2026-02-17) ### Features - **releasekit**: add Forge protocol extensions, transitive propagation, and multi-backend conformance (d6dbb44, #4577) — @Yesudeep Mangalapilly - **py/samples**: add web-endpoints-hello — REST + gRPC kitchen-sink sample (8614e5e, #4498) — @Yesudeep Mangalapilly ### Refactoring - **py/samples**: standardize naming taxonomy, consolidate shared logic, and close feature coverage gaps (1996c7c, #4488) — @Yesudeep Mangalapilly ### Documentation - **py**: update readmes for all samples and plugins (e02ea9f, #4624) — @Yesudeep Mangalapilly </details> <details><summary>provider-vertex-ai-vector-search-bigquery</summary> ## 0.2.0 (2026-02-17) ### Features - **releasekit**: add Forge protocol extensions, transitive propagation, and multi-backend conformance (d6dbb44, #4577) — @Yesudeep Mangalapilly - **py/samples**: add web-endpoints-hello — REST + gRPC kitchen-sink sample (8614e5e, #4498) — @Yesudeep Mangalapilly ### Bug Fixes - **py**: migrate default embedding model to gemini-embedding-001 (051f75f, #4557) — @Elisa Shen ### Refactoring - **py/samples**: standardize naming taxonomy, consolidate shared logic, and close feature coverage gaps (1996c7c, #4488) — @Yesudeep Mangalapilly </details> <details><summary>genkit-plugin-checks</summary> ## 0.6.0 (2026-02-17) ### Features - **releasekit**: add Forge protocol extensions, transitive propagation, and multi-backend conformance (d6dbb44, #4577) — @Yesudeep Mangalapilly - **py/checks**: add Google Checks AI Safety plugin (a8bf3c1, #4504) — @Yesudeep Mangalapilly ### Bug Fixes - **releasekit**: replace literal null byte with git %x00 escape in changelog format (4866724, #4661) — @Yesudeep Mangalapilly - issues reported by releasekit (fba9ed1, #4646) — @Yesudeep Mangalapilly - **py**: resolve CI license check failures and lint diagnostics (e8519ef, #4524) — @Yesudeep Mangalapilly </details> <details><summary>provider-microsoft-foundry-hello</summary> ## 0.1.0 (2026-02-17) ### Features - **releasekit**: add Forge protocol extensions, transitive propagation, and multi-backend conformance (d6dbb44, #4577) — @Yesudeep Mangalapilly - **py/samples**: add web-endpoints-hello — REST + gRPC kitchen-sink sample (8614e5e, #4498) — @Yesudeep Mangalapilly ### Bug Fixes - **py**: resolve CI license check failures and lint diagnostics (e8519ef, #4524) — @Yesudeep Mangalapilly ### Refactoring - **py/samples**: standardize naming taxonomy, consolidate shared logic, and close feature coverage gaps (1996c7c, #4488) — @Yesudeep Mangalapilly </details> <details><summary>provider-vertex-ai-vector-search-firestore</summary> ## 0.2.0 (2026-02-17) ### Features - **releasekit**: add Forge protocol extensions, transitive propagation, and multi-backend conformance (d6dbb44, #4577) — @Yesudeep Mangalapilly - **py/samples**: add web-endpoints-hello — REST + gRPC kitchen-sink sample (8614e5e, #4498) — @Yesudeep Mangalapilly ### Bug Fixes - **py**: migrate default embedding model to gemini-embedding-001 (051f75f, #4557) — @Elisa Shen ### Refactoring - **py/samples**: standardize naming taxonomy, consolidate shared logic, and close feature coverage gaps (1996c7c, #4488) — @Yesudeep Mangalapilly </details> <details><summary>provider-vertex-ai-rerank-eval</summary> ## 0.2.0 (2026-02-17) ### Features - **releasekit**: add Forge protocol extensions, transitive propagation, and multi-backend conformance (d6dbb44, #4577) — @Yesudeep Mangalapilly - **py/samples**: add web-endpoints-hello — REST + gRPC kitchen-sink sample (8614e5e, #4498) — @Yesudeep Mangalapilly ### Bug Fixes - **py**: handle nullable JSON Schema types in Gemini plugin and clean up samples (4daef62, #4629) — @Yesudeep Mangalapilly ### Refactoring - **py/samples**: standardize naming taxonomy, consolidate shared logic, and close feature coverage gaps (1996c7c, #4488) — @Yesudeep Mangalapilly </details> <details><summary>framework-realtime-tracing-demo</summary> ## 0.2.0 (2026-02-17) ### Features - **releasekit**: add Forge protocol extensions, transitive propagation, and multi-backend conformance (d6dbb44, #4577) — @Yesudeep Mangalapilly - **py/samples**: add web-endpoints-hello — REST + gRPC kitchen-sink sample (8614e5e, #4498) — @Yesudeep Mangalapilly ### Bug Fixes - **py**: handle nullable JSON Schema types in Gemini plugin and clean up samples (4daef62, #4629) — @Yesudeep Mangalapilly ### Refactoring - **py/samples**: standardize naming taxonomy, consolidate shared logic, and close feature coverage gaps (1996c7c, #4488) — @Yesudeep Mangalapilly </details> <details><summary>genkit-plugin-evaluators</summary> ## 0.6.0 (2026-02-17) ### Features - **releasekit**: add Forge protocol extensions, transitive propagation, and multi-backend conformance (d6dbb44, #4577) — @Yesudeep Mangalapilly - **py/samples**: add web-endpoints-hello — REST + gRPC kitchen-sink sample (8614e5e, #4498) — @Yesudeep Mangalapilly ### Bug Fixes - **py**: correct evaluator scoring bug (40bbdbc, #4573) — @Elisa Shen - **py/plugins**: fix wheel build duplicate files in PEP 420 namespace packages (0c396b6, #4441) — @Yesudeep Mangalapilly ### Refactoring - **py**: rename aws-bedrock plugin to amazon-bedrock (8acd6b0, #4448) — @Yesudeep Mangalapilly </details> <details><summary>genkit-plugin-mcp</summary> ## 0.6.0 (2026-02-17) ### Features - **releasekit**: add Forge protocol extensions, transitive propagation, and multi-backend conformance (d6dbb44, #4577) — @Yesudeep Mangalapilly - **py/samples**: add web-endpoints-hello — REST + gRPC kitchen-sink sample (8614e5e, #4498) — @Yesudeep Mangalapilly ### Bug Fixes - **py/plugins**: move in-function import to top level in google-genai (7235768, #4461) — @Yesudeep Mangalapilly - **py/plugins**: fix wheel build duplicate files in PEP 420 namespace packages (0c396b6, #4441) — @Yesudeep Mangalapilly ### Refactoring - **py**: rename aws-bedrock plugin to amazon-bedrock (8acd6b0, #4448) — @Yesudeep Mangalapilly </details> <details><summary>web-short-n-long</summary> ## 0.2.0 (2026-02-17) ### Features - **releasekit**: add Forge protocol extensions, transitive propagation, and multi-backend conformance (d6dbb44, #4577) — @Yesudeep Mangalapilly - **py/samples**: add web-endpoints-hello — REST + gRPC kitchen-sink sample (8614e5e, #4498) — @Yesudeep Mangalapilly ### Bug Fixes - **py**: fix broken flows in shot-n-long and combat-oai (9bdf5fd, #4610) — @Elisa Shen - **py**: migrate default embedding model to gemini-embedding-001 (051f75f, #4557) — @Elisa Shen ### Refactoring - **py/samples**: standardize naming taxonomy, consolidate shared logic, and close feature coverage gaps (1996c7c, #4488) — @Yesudeep Mangalapilly </details> <details><summary>provider-observability-hello</summary> ## 0.2.0 (2026-02-17) ### Features - **releasekit**: genkit.ai (f2dde35, #4702) — @Yesudeep Mangalapilly - **releasekit**: add Forge protocol extensions, transitive propagation, and multi-backend conformance (d6dbb44, #4577) — @Yesudeep Mangalapilly - **py/samples**: add web-endpoints-hello — REST + gRPC kitchen-sink sample (8614e5e, #4498) — @Yesudeep Mangalapilly ### Bug Fixes - **py**: resolve CI license check failures and lint diagnostics (e8519ef, #4524) — @Yesudeep Mangalapilly ### Refactoring - **py/samples**: standardize naming taxonomy, consolidate shared logic, and close feature coverage gaps (1996c7c, #4488) — @Yesudeep Mangalapilly </details> <details><summary>framework-prompt-demo</summary> ## 0.1.0 (2026-02-17) ### Features - **releasekit**: add Forge protocol extensions, transitive propagation, and multi-backend conformance (d6dbb44, #4577) — @Yesudeep Mangalapilly - **py/samples**: add web-endpoints-hello — REST + gRPC kitchen-sink sample (8614e5e, #4498) — @Yesudeep Mangalapilly ### Bug Fixes - **py**: add custom evaluator sample to match JS (3472939, #4670) — @Elisa Shen ### Refactoring - **py/samples**: standardize naming taxonomy, consolidate shared logic, and close feature coverage gaps (1996c7c, #4488) — @Yesudeep Mangalapilly ### Documentation - **py**: update readmes for all samples and plugins (e02ea9f, #4624) — @Yesudeep Mangalapilly </details> <details><summary>genkit-plugin-flask</summary> ## 0.6.0 (2026-02-17) ### Features - **releasekit**: add Forge protocol extensions, transitive propagation, and multi-backend conformance (d6dbb44, #4577) — @Yesudeep Mangalapilly - **py/samples**: add web-endpoints-hello — REST + gRPC kitchen-sink sample (8614e5e, #4498) — @Yesudeep Mangalapilly ### Bug Fixes - **py/plugins/flask**: remove self-referencing cyclical dependency using releasekit (edaf88c, #4561) — @Yesudeep Mangalapilly - **py/plugins**: fix wheel build duplicate files in PEP 420 namespace packages (0c396b6, #4441) — @Yesudeep Mangalapilly ### Refactoring - **py**: rename aws-bedrock plugin to amazon-bedrock (8acd6b0, #4448) — @Yesudeep Mangalapilly </details> <details><summary>dev-local-vectorstore-hello</summary> ## 0.2.0 (2026-02-17) ### Features - **releasekit**: add Forge protocol extensions, transitive propagation, and multi-backend conformance (d6dbb44, #4577) — @Yesudeep Mangalapilly - **py/samples**: add web-endpoints-hello — REST + gRPC kitchen-sink sample (8614e5e, #4498) — @Yesudeep Mangalapilly ### Bug Fixes - **py**: migrate default embedding model to gemini-embedding-001 (051f75f, #4557) — @Elisa Shen ### Refactoring - **py/samples**: standardize naming taxonomy, consolidate shared logic, and close feature coverage gaps (1996c7c, #4488) — @Yesudeep Mangalapilly ### Documentation - **py**: update readmes for all samples and plugins (e02ea9f, #4624) — @Yesudeep Mangalapilly </details> <details><summary>framework-tool-interrupts</summary> ## 0.2.0 (2026-02-17) ### Features - **releasekit**: add Forge protocol extensions, transitive propagation, and multi-backend conformance (d6dbb44, #4577) — @Yesudeep Mangalapilly - **py/samples**: add web-endpoints-hello — REST + gRPC kitchen-sink sample (8614e5e, #4498) — @Yesudeep Mangalapilly ### Bug Fixes - **py**: resolve CI license check failures and lint diagnostics (e8519ef, #4524) — @Yesudeep Mangalapilly ### Refactoring - **py/samples**: standardize naming taxonomy, consolidate shared logic, and close feature coverage gaps (1996c7c, #4488) — @Yesudeep Mangalapilly ### Documentation - **py**: update readmes for all samples and plugins (e02ea9f, #4624) — @Yesudeep Mangalapilly </details> <details><summary>provider-deepseek-hello</summary> ## 0.2.0 (2026-02-17) ### Features - **releasekit**: add Forge protocol extensions, transitive propagation, and multi-backend conformance (d6dbb44, #4577) — @Yesudeep Mangalapilly - **py/samples**: add web-endpoints-hello — REST + gRPC kitchen-sink sample (8614e5e, #4498) — @Yesudeep Mangalapilly ### Bug Fixes - **py**: add missing LICENSE file and license metadata to samples (4ddd8a3, #4571) — @Yesudeep Mangalapilly ### Refactoring - **py/samples**: standardize naming taxonomy, consolidate shared logic, and close feature coverage gaps (1996c7c, #4488) — @Yesudeep Mangalapilly ### Documentation - **py**: update readmes for all samples and plugins (e02ea9f, #4624) — @Yesudeep Mangalapilly </details> <details><summary>provider-google-genai-hello</summary> ## 0.2.0 (2026-02-17) ### Features - **releasekit**: add Forge protocol extensions, transitive propagation, and multi-backend conformance (d6dbb44, #4577) — @Yesudeep Mangalapilly - **py/samples**: add web-endpoints-hello — REST + gRPC kitchen-sink sample (8614e5e, #4498) — @Yesudeep Mangalapilly ### Bug Fixes - **py**: resolve CI license check failures and lint diagnostics (e8519ef, #4524) — @Yesudeep Mangalapilly ### Refactoring - **py/samples**: standardize naming taxonomy, consolidate shared logic, and close feature coverage gaps (1996c7c, #4488) — @Yesudeep Mangalapilly ### Documentation - **py**: update readmes for all samples and plugins (e02ea9f, #4624) — @Yesudeep Mangalapilly </details> <details><summary>provider-cloudflare-workers-ai-hello</summary> ## 0.2.0 (2026-02-17) ### Features - **releasekit**: add Forge protocol extensions, transitive propagation, and multi-backend conformance (d6dbb44, #4577) — @Yesudeep Mangalapilly - **py/samples**: add web-endpoints-hello — REST + gRPC kitchen-sink sample (8614e5e, #4498) — @Yesudeep Mangalapilly ### Bug Fixes - **py**: add missing LICENSE file and license metadata to samples (4ddd8a3, #4571) — @Yesudeep Mangalapilly - **py**: resolve CI license check failures and lint diagnostics (e8519ef, #4524) — @Yesudeep Mangalapilly ### Refactoring - **py/samples**: standardize naming taxonomy, consolidate shared logic, and close feature coverage gaps (1996c7c, #4488) — @Yesudeep Mangalapilly </details> <details><summary>provider-vertex-ai-model-garden</summary> ## 0.2.0 (2026-02-17) ### Features - **releasekit**: add Forge protocol extensions, transitive propagation, and multi-backend conformance (d6dbb44, #4577) — @Yesudeep Mangalapilly - **py/samples**: add web-endpoints-hello — REST + gRPC kitchen-sink sample (8614e5e, #4498) — @Yesudeep Mangalapilly ### Bug Fixes - **py**: resolve CI license check failures and lint diagnostics (e8519ef, #4524) — @Yesudeep Mangalapilly ### Refactoring - **py/samples**: standardize naming taxonomy, consolidate shared logic, and close feature coverage gaps (1996c7c, #4488) — @Yesudeep Mangalapilly ### Documentation - **py**: update readmes for all samples and plugins (e02ea9f, #4624) — @Yesudeep Mangalapilly </details> <details><summary>provider-google-genai-media-models-demo</summary> ## 0.2.0 (2026-02-17) ### Features - **releasekit**: add Forge protocol extensions, transitive propagation, and multi-backend conformance (d6dbb44, #4577) — @Yesudeep Mangalapilly - **py/samples**: add web-endpoints-hello — REST + gRPC kitchen-sink sample (8614e5e, #4498) — @Yesudeep Mangalapilly ### Bug Fixes - **py**: add missing LICENSE file and license metadata to samples (4ddd8a3, #4571) — @Yesudeep Mangalapilly - **py**: resolve CI license check failures and lint diagnostics (e8519ef, #4524) — @Yesudeep Mangalapilly ### Refactoring - **py/samples**: standardize naming taxonomy, consolidate shared logic, and close feature coverage gaps (1996c7c, #4488) — @Yesudeep Mangalapilly </details> <details><summary>provider-compat-oai-hello</summary> ## 0.2.0 (2026-02-17) ### Features - **releasekit**: add Forge protocol extensions, transitive propagation, and multi-backend conformance (d6dbb44, #4577) — @Yesudeep Mangalapilly - **py/samples**: add web-endpoints-hello — REST + gRPC kitchen-sink sample (8614e5e, #4498) — @Yesudeep Mangalapilly ### Bug Fixes - **py**: fix structlog config blowaway, DeepSeek reasoning, and double JSON encoding (ea0223b, #4625) — @Yesudeep Mangalapilly ### Refactoring - **py/samples**: standardize naming taxonomy, consolidate shared logic, and close feature coverage gaps (1996c7c, #4488) — @Yesudeep Mangalapilly ### Documentation - **py**: update readmes for all samples and plugins (e02ea9f, #4624) — @Yesudeep Mangalapilly </details> <details><summary>provider-amazon-bedrock-hello</summary> ## 0.2.0 (2026-02-17) ### Features - **releasekit**: add Forge protocol extensions, transitive propagation, and multi-backend conformance (d6dbb44, #4577) — @Yesudeep Mangalapilly - **py/plugins**: add Cohere provider plugin (e424dcd, #4518) — @Yesudeep Mangalapilly - **py/samples**: add web-endpoints-hello — REST + gRPC kitchen-sink sample (8614e5e, #4498) — @Yesudeep Mangalapilly ### Bug Fixes - **py**: add missing LICENSE file and license metadata to samples (4ddd8a3, #4571) — @Yesudeep Mangalapilly - **py**: resolve CI license check failures and lint diagnostics (e8519ef, #4524) — @Yesudeep Mangalapilly ### Refactoring - **py/samples**: standardize naming taxonomy, consolidate shared logic, and close feature coverage gaps (1996c7c, #4488) — @Yesudeep Mangalapilly </details> <details><summary>provider-mistral-hello</summary> ## 0.2.0 (2026-02-17) ### Features - **releasekit**: add Forge protocol extensions, transitive propagation, and multi-backend conformance (d6dbb44, #4577) — @Yesudeep Mangalapilly - **py/plugins**: add Cohere provider plugin (e424dcd, #4518) — @Yesudeep Mangalapilly - **py/samples**: add web-endpoints-hello — REST + gRPC kitchen-sink sample (8614e5e, #4498) — @Yesudeep Mangalapilly ### Bug Fixes - **py**: add missing LICENSE file and license metadata to samples (4ddd8a3, #4571) — @Yesudeep Mangalapilly ### Refactoring - **py/samples**: standardize naming taxonomy, consolidate shared logic, and close feature coverage gaps (1996c7c, #4488) — @Yesudeep Mangalapilly ### Documentation - **py**: update readmes for all samples and plugins (e02ea9f, #4624) — @Yesudeep Mangalapilly </details> <details><summary>web-multi-server</summary> ## 0.2.0 (2026-02-17) ### Features - **releasekit**: add Forge protocol extensions, transitive propagation, and multi-backend conformance (d6dbb44, #4577) — @Yesudeep Mangalapilly - **py/samples**: add web-endpoints-hello — REST + gRPC kitchen-sink sample (8614e5e, #4498) — @Yesudeep Mangalapilly ### Bug Fixes - **py**: fix living log and test flows in sample-flow-test tool (a9b9132, #4545) — @Elisa Shen - **py**: resolve CI license check failures and lint diagnostics (e8519ef, #4524) — @Yesudeep Mangalapilly ### Refactoring - **py/samples**: standardize naming taxonomy, consolidate shared logic, and close feature coverage gaps (1996c7c, #4488) — @Yesudeep Mangalapilly ### Documentation - **py**: update readmes for all samples and plugins (e02ea9f, #4624) — @Yesudeep Mangalapilly </details> <details><summary>framework-evaluator-demo</summary> ## 0.1.0 (2026-02-17) ### Features - **releasekit**: add Forge protocol extensions, transitive propagation, and multi-backend conformance (d6dbb44, #4577) — @Yesudeep Mangalapilly - **py/samples**: add web-endpoints-hello — REST + gRPC kitchen-sink sample (8614e5e, #4498) — @Yesudeep Mangalapilly ### Bug Fixes - **py**: add missing LICENSE file and license metadata to samples (4ddd8a3, #4571) — @Yesudeep Mangalapilly - **py**: migrate default embedding model to gemini-embedding-001 (051f75f, #4557) — @Elisa Shen - **py**: resolve CI license check failures and lint diagnostics (e8519ef, #4524) — @Yesudeep Mangalapilly ### Refactoring - **py/samples**: standardize naming taxonomy, consolidate shared logic, and close feature coverage gaps (1996c7c, #4488) — @Yesudeep Mangalapilly </details> <details><summary>genkit-plugin-google-cloud</summary> ## 0.6.0 (2026-02-17) ### Features - **releasekit**: add Forge protocol extensions, transitive propagation, and multi-backend conformance (d6dbb44, #4577) — @Yesudeep Mangalapilly - **py/samples**: add web-endpoints-hello — REST + gRPC kitchen-sink sample (8614e5e, #4498) — @Yesudeep Mangalapilly ### Bug Fixes - Fixed firebase telemetry , refactored telemetry implementation fixed failing tests (05c1b55, #4530) — @Niraj Nepal - **py**: fix structlog config blowaway, DeepSeek reasoning, and double JSON encoding (ea0223b, #4625) — @Yesudeep Mangalapilly - **py/plugins**: fix wheel build duplicate files in PEP 420 namespace packages (0c396b6, #4441) — @Yesudeep Mangalapilly ### Refactoring - **py**: rename aws-bedrock plugin to amazon-bedrock (8acd6b0, #4448) — @Yesudeep Mangalapilly </details> <details><summary>framework-restaurant-demo</summary> ## 0.2.0 (2026-02-17) ### Features - **releasekit**: add Forge protocol extensions, transitive propagation, and multi-backend conformance (d6dbb44, #4577) — @Yesudeep Mangalapilly - **py/samples**: add web-endpoints-hello — REST + gRPC kitchen-sink sample (8614e5e, #4498) — @Yesudeep Mangalapilly ### Bug Fixes - **py**: fix living log and test flows in sample-flow-test tool (a9b9132, #4545) — @Elisa Shen - **py**: resolve CI license check failures and lint diagnostics (e8519ef, #4524) — @Yesudeep Mangalapilly ### Refactoring - **py/samples**: standardize naming taxonomy, consolidate shared logic, and close feature coverage gaps (1996c7c, #4488) — @Yesudeep Mangalapilly ### Documentation - **py**: update readmes for all samples and plugins (e02ea9f, #4624) — @Yesudeep Mangalapilly </details> <details><summary>provider-anthropic-hello</summary> ## 0.2.0 (2026-02-17) ### Features - **releasekit**: add Forge protocol extensions, transitive propagation, and multi-backend conformance (d6dbb44, #4577) — @Yesudeep Mangalapilly - **py/samples**: add web-endpoints-hello — REST + gRPC kitchen-sink sample (8614e5e, #4498) — @Yesudeep Mangalapilly ### Bug Fixes - **py**: add missing LICENSE file and license metadata to samples (4ddd8a3, #4571) — @Yesudeep Mangalapilly - **py**: resolve CI license check failures and lint diagnostics (e8519ef, #4524) — @Yesudeep Mangalapilly ### Refactoring - **py/samples**: standardize naming taxonomy, consolidate shared logic, and close feature coverage gaps (1996c7c, #4488) — @Yesudeep Mangalapilly ### Documentation - **py**: update readmes for all samples and plugins (e02ea9f, #4624) — @Yesudeep Mangalapilly </details> <details><summary>provider-google-genai-vertexai-hello</summary> ## 0.2.0 (2026-02-17) ### Features - **releasekit**: add Forge protocol extensions, transitive propagation, and multi-backend conformance (d6dbb44, #4577) — @Yesudeep Mangalapilly - **py/samples**: add web-endpoints-hello — REST + gRPC kitchen-sink sample (8614e5e, #4498) — @Yesudeep Mangalapilly ### Bug Fixes - **py**: migrate default embedding model to gemini-embedding-001 (051f75f, #4557) — @Elisa Shen - **py**: resolve CI license check failures and lint diagnostics (e8519ef, #4524) — @Yesudeep Mangalapilly ### Refactoring - **py/samples**: standardize naming taxonomy, consolidate shared logic, and close feature coverage gaps (1996c7c, #4488) — @Yesudeep Mangalapilly ### Documentation - **py**: update readmes for all samples and plugins (e02ea9f, #4624) — @Yesudeep Mangalapilly </details> <details><summary>provider-google-genai-vertexai-image</summary> ## 0.2.0 (2026-02-17) ### Features - **releasekit**: add Forge protocol extensions, transitive propagation, and multi-backend conformance (d6dbb44, #4577) — @Yesudeep Mangalapilly - **py/samples**: add web-endpoints-hello — REST + gRPC kitchen-sink sample (8614e5e, #4498) — @Yesudeep Mangalapilly ### Bug Fixes - **releasekit**: fix git push argument order, boost test coverage to 92%, fix lint errors (59d9dc3, #4667) — @Yesudeep Mangalapilly - **py**: resolve CI license check failures and lint diagnostics (e8519ef, #4524) — @Yesudeep Mangalapilly ### Refactoring - **py/samples**: standardize naming taxonomy, consolidate shared logic, and close feature coverage gaps (1996c7c, #4488) — @Yesudeep Mangalapilly ### Documentation - **py**: update readmes for all samples and plugins (e02ea9f, #4624) — @Yesudeep Mangalapilly </details> > **Note:** 22 changelog(s) omitted to stay within the PR body size limit. See individual CHANGELOG.md files for full details.  ```json { "git_sha": "3fc96674ec066181634177f90e9362858c5e6534", "umbrella_tag": "py/v0.6.0", "packages": [ { "name": "conform", "old_version": "0.1.0", "new_version": "0.2.0", "bump": "minor", "reason": "fix(conform,anthropic): native executors, tool schema handling, and CLI consolidation (#4698)", "skipped": false, "tag": "/conform-v0.2.0" }, { "name": "dev-local-vectorstore-hello", "old_version": "0.1.0", "new_version": "0.2.0", "bump": "minor", "reason": "feat(releasekit): add Forge protocol extensions, transitive propagation, and multi-backend conformance (#4577)", "skipped": false, "tag": "/dev-local-vectorstore-hello-v0.2.0" }, { "name": "framework-context-demo", "old_version": "0.1.0", "new_version": "0.2.0", "bump": "minor", "reason": "feat(releasekit): add Forge protocol extensions, transitive propagation, and multi-backend conformance (#4577)", "skipped": false, "tag": "/framework-context-demo-v0.2.0" }, { "name": "framework-custom-evaluators", "old_version": "0.0.1", "new_version": "0.1.0", "bump": "minor", "reason": "feat(releasekit): supply-chain security, multi-ecosystem orchestration, and CI hardening (#4682)", "skipped": false, "tag": "/framework-custom-evaluators-v0.1.0" }, { "name": "framework-dynamic-tools-demo", "old_version": "0.1.0", "new_version": "0.2.0", "bump": "minor", "reason": "feat(releasekit): add Forge protocol extensions, transitive propagation, and multi-backend conformance (#4577)", "skipped": false, "tag": "/framework-dynamic-tools-demo-v0.2.0" }, { "name": "framework-evaluator-demo", "old_version": "0.0.1", "new_version": "0.1.0", "bump": "minor", "reason": "feat(releasekit): add Forge protocol extensions, transitive propagation, and multi-backend conformance (#4577)", "skipped": false, "tag": "/framework-evaluator-demo-v0.1.0" }, { "name": "framework-format-demo", "old_version": "0.1.0", "new_version": "0.2.0", "bump": "minor", "reason": "feat(releasekit): add Forge protocol extensions, transitive propagation, and multi-backend conformance (#4577)", "skipped": false, "tag": "/framework-format-demo-v0.2.0" }, { "name": "framework-middleware-demo", "old_version": "0.1.0", "new_version": "0.2.0", "bump": "minor", "reason": "feat(releasekit): add Forge protocol extensions, transitive propagation, and multi-backend conformance (#4577)", "skipped": false, "tag": "/framework-middleware-demo-v0.2.0" }, { "name": "framework-prompt-demo", "old_version": "0.0.1", "new_version": "0.1.0", "bump": "minor", "reason": "fix(py): add custom evaluator sample to match JS (#4670)", "skipped": false, "tag": "/framework-prompt-demo-v0.1.0" }, { "name": "framework-realtime-tracing-demo", "old_version": "0.1.0", "new_version": "0.2.0", "bump": "minor", "reason": "fix(py): handle nullable JSON Schema types in Gemini plugin and clean up samples (#4629)", "skipped": false, "tag": "/framework-realtime-tracing-demo-v0.2.0" }, { "name": "framework-restaurant-demo", "old_version": "0.1.0", "new_version": "0.2.0", "bump": "minor", "reason": "feat(releasekit): add Forge protocol extensions, transitive propagation, and multi-backend conformance (#4577)", "skipped": false, "tag": "/framework-restaurant-demo-v0.2.0" }, { "name": "framework-tool-interrupts", "old_version": "0.1.0", "new_version": "0.2.0", "bump": "minor", "reason": "feat(releasekit): add Forge protocol extensions, transitive propagation, and multi-backend conformance (#4577)", "skipped": false, "tag": "/framework-tool-interrupts-v0.2.0" }, { "name": "genkit", "old_version": "0.5.0", "new_version": "0.6.0", "bump": "minor", "reason": "fix: Path fix for logging (#4642)", "skipped": false, "tag": "/genkit-v0.6.0" }, { "name": "genkit-plugin-amazon-bedrock", "old_version": "0.5.0", "new_version": "0.6.0", "bump": "minor", "reason": "fix(releasekit): replace literal null byte with git %x00 escape in changelog format (#4661)", "skipped": false, "tag": "/genkit-plugin-amazon-bedrock-v0.6.0" }, { "name": "genkit-plugin-anthropic", "old_version": "0.5.0", "new_version": "0.6.0", "bump": "minor", "reason": "fix(releasekit): replace literal null byte with git %x00 escape in changelog format (#4661)", "skipped": false, "tag": "/genkit-plugin-anthropic-v0.6.0" }, { "name": "genkit-plugin-checks", "old_version": "0.5.0", "new_version": "0.6.0", "bump": "minor", "reason": "fix(releasekit): replace literal null byte with git %x00 escape in changelog format (#4661)", "skipped": false, "tag": "/genkit-plugin-checks-v0.6.0" }, { "name": "genkit-plugin-cloudflare-workers-ai", "old_version": "0.5.0", "new_version": "0.6.0", "bump": "minor", "reason": "fix(releasekit): replace literal null byte with git %x00 escape in changelog format (#4661)", "skipped": false, "tag": "/genkit-plugin-cloudflare-workers-ai-v0.6.0" }, { "name": "genkit-plugin-cohere", "old_version": "0.5.0", "new_version": "0.6.0", "bump": "minor", "reason": "fix(releasekit): replace literal null byte with git %x00 escape in changelog format (#4661)", "skipped": false, "tag": "/genkit-plugin-cohere-v0.6.0" }, { "name": "genkit-plugin-compat-oai", "old_version": "0.5.0", "new_version": "0.6.0", "bump": "minor", "reason": "fix(releasekit): replace literal null byte with git %x00 escape in changelog format (#4661)", "skipped": false, "tag": "/genkit-plugin-compat-oai-v0.6.0" }, { "name": "genkit-plugin-deepseek", "old_version": "0.5.0", "new_version": "0.6.0", "bump": "minor", "reason": "fix(releasekit): replace literal null byte with git %x00 escape in changelog format (#4661)", "skipped": false, "tag": "/genkit-plugin-deepseek-v0.6.0" }, { "name": "genkit-plugin-dev-local-vectorstore", "old_version": "0.5.0", "new_version": "0.6.0", "bump": "minor", "reason": "fix: issues reported by releasekit (#4646)", "skipped": false, "tag": "/genkit-plugin-dev-local-vectorstore-v0.6.0" }, { "name": "genkit-plugin-evaluators", "old_version": "0.5.0", "new_version": "0.6.0", "bump": "minor", "reason": "feat(releasekit): add Forge protocol extensions, transitive propagation, and multi-backend conformance (#4577)", "skipped": false, "tag": "/genkit-plugin-evaluators-v0.6.0" }, { "name": "genkit-plugin-fastapi", "old_version": "0.5.0", "new_version": "0.6.0", "bump": "minor", "reason": "fix(py): address releasekit check warnings for metadata and grouping (#4595)", "skipped": false, "tag": "/genkit-plugin-fastapi-v0.6.0" }, { "name": "genkit-plugin-firebase", "old_version": "0.5.0", "new_version": "0.6.0", "bump": "minor", "reason": "feat(releasekit): graph based licensing (#4705)", "skipped": false, "tag": "/genkit-plugin-firebase-v0.6.0" }, { "name": "genkit-plugin-flask", "old_version": "0.5.0", "new_version": "0.6.0", "bump": "minor", "reason": "feat(releasekit): add Forge protocol extensions, transitive propagation, and multi-backend conformance (#4577)", "skipped": false, "tag": "/genkit-plugin-flask-v0.6.0" }, { "name": "genkit-plugin-google-cloud", "old_version": "0.5.0", "new_version": "0.6.0", "bump": "minor", "reason": "fix: Fixed firebase telemetry , refactored telemetry implementation fixed failing tests (#4530)", "skipped": false, "tag": "/genkit-plugin-google-cloud-v0.6.0" }, { "name": "genkit-plugin-google-genai", "old_version": "0.5.0", "new_version": "0.6.0", "bump": "minor", "reason": "fix(py): update google-genai evaluators and cleanup sample-test (#4648)", "skipped": false, "tag": "/genkit-plugin-google-genai-v0.6.0" }, { "name": "genkit-plugin-huggingface", "old_version": "0.5.0", "new_version": "0.6.0", "bump": "minor", "reason": "fix(releasekit): replace literal null byte with git %x00 escape in changelog format (#4661)", "skipped": false, "tag": "/genkit-plugin-huggingface-v0.6.0" }, { "name": "genkit-plugin-mcp", "old_version": "0.5.0", "new_version": "0.6.0", "bump": "minor", "reason": "feat(releasekit): add Forge protocol extensions, transitive propagation, and multi-backend conformance (#4577)", "skipped": false, "tag": "/genkit-plugin-mcp-v0.6.0" }, { "name": "genkit-plugin-microsoft-foundry", "old_version": "0.5.0", "new_version": "0.6.0", "bump": "minor", "reason": "fix(microsoft-foundry): conform tests and sanitize credentials to prevent Unicode encoding errors (#4689)", "skipped": false, "tag": "/genkit-plugin-microsoft-foundry-v0.6.0" }, { "name": "genkit-plugin-mistral", "old_version": "0.5.0", "new_version": "0.6.0", "bump": "minor", "reason": "fix(releasekit): replace literal null byte with git %x00 escape in changelog format (#4661)", "skipped": false, "tag": "/genkit-plugin-mistral-v0.6.0" }, { "name": "genkit-plugin-observability", "old_version": "0.5.0", "new_version": "0.6.0", "bump": "minor", "reason": "fix(releasekit): replace literal null byte with git %x00 escape in changelog format (#4661)", "skipped": false, "tag": "/genkit-plugin-observability-v0.6.0" }, { "name": "genkit-plugin-ollama", "old_version": "0.5.0", "new_version": "0.6.0", "bump": "minor", "reason": "fix(releasekit): replace literal null byte with git %x00 escape in changelog format (#4661)", "skipped": false, "tag": "/genkit-plugin-ollama-v0.6.0" }, { "name": "genkit-plugin-vertex-ai", "old_version": "0.5.0", "new_version": "0.6.0", "bump": "minor", "reason": "fix(py/vertex-ai): async client creation with threaded credential refresh (#4608)", "skipped": false, "tag": "/genkit-plugin-vertex-ai-v0.6.0" }, { "name": "genkit-plugin-xai", "old_version": "0.5.0", "new_version": "0.6.0", "bump": "minor", "reason": "fix(conform,anthropic): native executors, tool schema handling, and CLI consolidation (#4698)", "skipped": false, "tag": "/genkit-plugin-xai-v0.6.0" }, { "name": "genkit-tools-model-config-test", "old_version": "0.1.0", "new_version": "0.2.0", "bump": "minor", "reason": "feat(releasekit): genkit.ai (#4702)", "skipped": false, "tag": "/genkit-tools-model-config-test-v0.2.0" }, { "name": "genkit-tools-sample-flows", "old_version": "0.1.0", "new_version": "0.1.1", "bump": "patch", "reason": "fix(py): relocate tools of model-config test and sample-flow test (#4669)", "skipped": false, "tag": "/genkit-tools-sample-flows-v0.1.1" }, { "name": "provider-amazon-bedrock-hello", "old_version": "0.1.0", "new_version": "0.2.0", "bump": "minor", "reason": "feat(releasekit): add Forge protocol extensions, transitive propagation, and multi-backend conformance (#4577)", "skipped": false, "tag": "/provider-amazon-bedrock-hello-v0.2.0" }, { "name": "provider-anthropic-hello", "old_version": "0.1.0", "new_version": "0.2.0", "bump": "minor", "reason": "feat(releasekit): add Forge protocol extensions, transitive propagation, and multi-backend conformance (#4577)", "skipped": false, "tag": "/provider-anthropic-hello-v0.2.0" }, { "name": "provider-checks-hello", "old_version": "0.1.0", "new_version": "0.2.0", "bump": "minor", "reason": "feat(releasekit): add Forge protocol extensions, transitive propagation, and multi-backend conformance (#4577)", "skipped": false, "tag": "/provider-checks-hello-v0.2.0" }, { "name": "provider-cloudflare-workers-ai-hello", "old_version": "0.1.0", "new_version": "0.2.0", "bump": "minor", "reason": "feat(releasekit): add Forge protocol extensions, transitive propagation, and multi-backend conformance (#4577)", "skipped": false, "tag": "/provider-cloudflare-workers-ai-hello-v0.2.0" }, { "name": "provider-cohere-hello", "old_version": "0.1.0", "new_version": "0.2.0", "bump": "minor", "reason": "feat(releasekit): add Forge protocol extensions, transitive propagation, and multi-backend conformance (#4577)", "skipped": false, "tag": "/provider-cohere-hello-v0.2.0" }, { "name": "provider-compat-oai-hello", "old_version": "0.1.0", "new_version": "0.2.0", "bump": "minor", "reason": "fix(py): fix structlog config blowaway, DeepSeek reasoning, and double JSON encoding (#4625)", "skipped": false, "tag": "/provider-compat-oai-hello-v0.2.0" }, { "name": "provider-deepseek-hello", "old_version": "0.1.0", "new_version": "0.2.0", "bump": "minor", "reason": "feat(releasekit): add Forge protocol extensions, transitive propagation, and multi-backend conformance (#4577)", "skipped": false, "tag": "/provider-deepseek-hello-v0.2.0" }, { "name": "provider-firestore-retriever", "old_version": "0.1.0", "new_version": "0.2.0", "bump": "minor", "reason": "feat(releasekit): add Forge protocol extensions, transitive propagation, and multi-backend conformance (#4577)", "skipped": false, "tag": "/provider-firestore-retriever-v0.2.0" }, { "name": "provider-google-genai-code-execution", "old_version": "0.1.0", "new_version": "0.2.0", "bump": "minor", "reason": "feat(releasekit): add Forge protocol extensions, transitive propagation, and multi-backend conformance (#4577)", "skipped": false, "tag": "/provider-google-genai-code-execution-v0.2.0" }, { "name": "provider-google-genai-context-caching", "old_version": "0.1.0", "new_version": "0.2.0", "bump": "minor", "reason": "feat(releasekit): add Forge protocol extensions, transitive propagation, and multi-backend conformance (#4577)", "skipped": false, "tag": "/provider-google-genai-context-caching-v0.2.0" }, { "name": "provider-google-genai-hello", "old_version": "0.1.0", "new_version": "0.2.0", "bump": "minor", "reason": "feat(releasekit): add Forge protocol extensions, transitive propagation, and multi-backend conformance (#4577)", "skipped": false, "tag": "/provider-google-genai-hello-v0.2.0" }, { "name": "provider-google-genai-media-models-demo", "old_version": "0.1.0", "new_version": "0.2.0", "bump": "minor", "reason": "feat(releasekit): add Forge protocol extensions, transitive propagation, and multi-backend conformance (#4577)", "skipped": false, "tag": "/provider-google-genai-media-models-demo-v0.2.0" }, { "name": "provider-google-genai-vertexai-hello", "old_version": "0.1.0", "new_version": "0.2.0", "bump": "minor", "reason": "feat(releasekit): add Forge protocol extensions, transitive propagation, and multi-backend conformance (#4577)", "skipped": false, "tag": "/provider-google-genai-vertexai-hello-v0.2.0" }, { "name": "provider-google-genai-vertexai-image", "old_version": "0.1.0", "new_version": "0.2.0", "bump": "minor", "reason": "fix(releasekit): fix git push argument order, boost test coverage to 92%, fix lint errors (#4667)", "skipped": false, "tag": "/provider-google-genai-vertexai-image-v0.2.0" }, { "name": "provider-huggingface-hello", "old_version": "0.1.0", "new_version": "0.2.0", "bump": "minor", "reason": "feat(releasekit): add Forge protocol extensions, transitive propagation, and multi-backend conformance (#4577)", "skipped": false, "tag": "/provider-huggingface-hello-v0.2.0" }, { "name": "provider-microsoft-foundry-hello", "old_version": "0.0.0", "new_version": "0.1.0", "bump": "minor", "reason": "feat(releasekit): add Forge protocol extensions, transitive propagation, and multi-backend conformance (#4577)", "skipped": false, "tag": "/provider-microsoft-foundry-hello-v0.1.0" }, { "name": "provider-mistral-hello", "old_version": "0.1.0", "new_version": "0.2.0", "bump": "minor", "reason": "feat(releasekit): add Forge protocol extensions, transitive propagation, and multi-backend conformance (#4577)", "skipped": false, "tag": "/provider-mistral-hello-v0.2.0" }, { "name": "provider-observability-hello", "old_version": "0.1.0", "new_version": "0.2.0", "bump": "minor", "reason": "feat(releasekit): genkit.ai (#4702)", "skipped": false, "tag": "/provider-observability-hello-v0.2.0" }, { "name": "provider-ollama-hello", "old_version": "0.1.0", "new_version": "0.2.0", "bump": "minor", "reason": "feat(releasekit): add Forge protocol extensions, transitive propagation, and multi-backend conformance (#4577)", "skipped": false, "tag": "/provider-ollama-hello-v0.2.0" }, { "name": "provider-vertex-ai-model-garden", "old_version": "0.1.0", "new_version": "0.2.0", "bump": "minor", "reason": "feat(releasekit): add Forge protocol extensions, transitive propagation, and multi-backend conformance (#4577)", "skipped": false, "tag": "/provider-vertex-ai-model-garden-v0.2.0" }, { "name": "provider-vertex-ai-rerank-eval", "old_version": "0.1.0", "new_version": "0.2.0", "bump": "minor", "reason": "fix(py): handle nullable JSON Schema types in Gemini plugin and clean up samples (#4629)", "skipped": false, "tag": "/provider-vertex-ai-rerank-eval-v0.2.0" }, { "name": "provider-vertex-ai-vector-search-bigquery", "old_version": "0.1.0", "new_version": "0.2.0", "bump": "minor", "reason": "feat(releasekit): add Forge protocol extensions, transitive propagation, and multi-backend conformance (#4577)", "skipped": false, "tag": "/provider-vertex-ai-vector-search-bigquery-v0.2.0" }, { "name": "provider-vertex-ai-vector-search-firestore", "old_version": "0.1.0", "new_version": "0.2.0", "bump": "minor", "reason": "feat(releasekit): add Forge protocol extensions, transitive propagation, and multi-backend conformance (#4577)", "skipped": false, "tag": "/provider-vertex-ai-vector-search-firestore-v0.2.0" }, { "name": "provider-xai-hello", "old_version": "0.1.0", "new_version": "0.2.0", "bump": "minor", "reason": "feat(releasekit): add default branch detection, and packaging (#4650)", "skipped": false, "tag": "/provider-xai-hello-v0.2.0" }, { "name": "releasekit", "old_version": "0.1.0", "new_version": "0.2.0", "bump": "minor", "reason": "fix(releasekit): skip git hooks in release commits and pushes (#4723)", "skipped": false, "tag": "/releasekit-v0.2.0" }, { "name": "web-endpoints-hello", "old_version": "0.1.0", "new_version": "0.2.0", "bump": "minor", "reason": "feat(releasekit): supply-chain security, multi-ecosystem orchestration, and CI hardening (#4682)", "skipped": false, "tag": "/web-endpoints-hello-v0.2.0" }, { "name": "web-fastapi-bugbot", "old_version": "0.1.0", "new_version": "0.2.0", "bump": "minor", "reason": "fix(py): handle nullable JSON Schema types in Gemini plugin and clean up samples (#4629)", "skipped": false, "tag": "/web-fastapi-bugbot-v0.2.0" }, { "name": "web-flask-hello", "old_version": "0.1.0", "new_version": "0.2.0", "bump": "minor", "reason": "feat(releasekit): add Forge protocol extensions, transitive propagation, and multi-backend conformance (#4577)", "skipped": false, "tag": "/web-flask-hello-v0.2.0" }, { "name": "web-multi-server", "old_version": "0.1.0", "new_version": "0.2.0", "bump": "minor", "reason": "feat(releasekit): add Forge protocol extensions, transitive propagation, and multi-backend conformance (#4577)", "skipped": false, "tag": "/web-multi-server-v0.2.0" }, { "name": "web-short-n-long", "old_version": "0.1.0", "new_version": "0.2.0", "bump": "minor", "reason": "fix(py): fix broken flows in shot-n-long and combat-oai (#4610)", "skipped": false, "tag": "/web-short-n-long-v0.2.0" } ], "created_at": "2026-02-17T23:04:44.937419+00:00" } ```

github-project-automation Bot added this to Genkit Backlog Feb 14, 2026

github-actions Bot added docs Improvements or additions to documentation python Python config fix labels Feb 14, 2026

MengqinShen requested a review from yesudeep February 14, 2026 08:10

gemini-code-assist Bot reviewed Feb 14, 2026

View reviewed changes

github-actions Bot added the js label Feb 15, 2026

MengqinShen-GL added 3 commits February 14, 2026 23:31

fix(py): add static prompt fixtures and custom evaluator sample to ma…

bc889f8

…tch JS

fix(py): fix with gemini

591c5c7

fix(py): fix lint errors and remove prompt file changes in JS

ec9b252

MengqinShen force-pushed the elisa/fix/add-prompt branch from e5a0306 to ec9b252 Compare February 15, 2026 07:32

MengqinShen-GL added 2 commits February 14, 2026 23:49

fix(py): fix type check errors

8fede59

fix(py): remove static prompt fixtures to a new pr

3efe7df

MengqinShen changed the title ~~fix(py): add static prompt fixtures and custom evaluator sample to match JS~~ fix(py): add custom evaluator sample to match JS Feb 15, 2026

fix(py): update pyproject and fixed the pyrefly missing-import errors…

bd99923

… in the framework-custom-evaluators

github-actions Bot added the root label Feb 15, 2026

fix(py): update pyproject to fix Ty check errors in new sample

8161a58

yesudeep approved these changes Feb 15, 2026

View reviewed changes

MengqinShen marked this pull request as ready for review February 15, 2026 09:25

MengqinShen enabled auto-merge (squash) February 15, 2026 09:26

MengqinShen merged commit 3472939 into main Feb 15, 2026
21 checks passed

MengqinShen deleted the elisa/fix/add-prompt branch February 15, 2026 09:36

github-project-automation Bot moved this to Done in Genkit Backlog Feb 15, 2026

This was referenced Feb 17, 2026

chore(release): v0.2.0 #4711

Closed

chore(release): genkit python v0.2.0 #4721

Closed

yesudeep mentioned this pull request Feb 17, 2026

chore(release): genkit python v0.6.0 #4725

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

fix(py): add custom evaluator sample to match JS#4670

fix(py): add custom evaluator sample to match JS#4670
MengqinShen merged 7 commits into
mainfrom
elisa/fix/add-prompt

MengqinShen commented Feb 14, 2026

Uh oh!

gemini-code-assist Bot commented Feb 14, 2026

Uh oh!

gemini-code-assist Bot left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

MengqinShen commented Feb 14, 2026

Uh oh!

gemini-code-assist Bot commented Feb 14, 2026

Summary of Changes

Highlights

Footnotes

Uh oh!

gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

Code Review

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants