fix(models): Allow context caches with empty content prefixes - #6367
Closed
hxaxd wants to merge 5 commits into
Closed
fix(models): Allow context caches with empty content prefixes#6367hxaxd wants to merge 5 commits into
hxaxd wants to merge 5 commits into
Conversation
|
Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA). View this failed invocation of the CLA check for more information. For the most up to date status, view the checks section at the bottom of the pull request. |
Omit the contents field when the cacheable prefix is empty so the Google Gen AI SDK can create a cache from the system instruction and tools. Add a regression assertion for the zero-prefix request. Fixes google#6363
hxaxd
force-pushed
the
fix/gemini-cache-empty-contents
branch
from
July 10, 2026 15:31
d51915b to
7b2808b
Compare
copybara-service Bot
pushed a commit
that referenced
this pull request
Jul 13, 2026
Merge #6367 ### Link to Issue or Description of Change - Closes: #6363 **Problem:** [PR #6067](#6067) correctly changed fingerprint-only cache metadata to use the cacheable conversation prefix rather than all request contents. This keeps trailing user contents, including request-scoped dynamic instructions, out of the stable cache identity. For an initial request containing only the latest user content, the cacheable conversation prefix is legitimately empty. ADK therefore stores fingerprint-only metadata with `contents_count == 0`. It does not create a cache on this initial request because no token count from a previous response is available yet. On the next request, ADK uses the stored count to calculate the current fingerprint. When the system instruction and tools remain unchanged, that fingerprint matches the stored fingerprint. If the system instruction and tools are large enough to meet Gemini’s cache-size requirement, ADK makes its first real cache-creation attempt. The stored count is deliberately reused because it defines the prefix covered by the matched fingerprint. Even if the later request now has a non-empty cacheable conversation prefix, ADK still slices it with the stored zero. This produces `llm_request.contents[:0] == []`, which is passed as `CreateCachedContentConfig(contents=[])`. The Google Gen AI SDK distinguishes an omitted `contents` field from an explicit empty list. It rejects `contents=[]` with `ValueError: contents are required`, while `contents=None` omits only the conversation contents and still allows the system instruction and tools to be included in the cache request. ADK catches the exception, logs a warning, and continues the model request without a cache, so the request itself does not crash. Because no `cache_name` was created, the caller preserves the fingerprint-only metadata and its original `contents_count == 0`. This behavior is intentional so transient cache-creation failures can retry the same cache identity. As long as the system instruction and tools remain unchanged, the next request again matches the same fingerprint using the preserved count. ADK then retries the same invalid `contents=[]` configuration, logs another warning, and again receives no `cache_name`. This cycle repeats on every matching request, so the cache is never established. **Solution:** Pass `None` when the cacheable conversation prefix is empty so the SDK omits the `contents` field. Preserve the existing list behavior for non-empty prefixes. The existing zero-prefix lifecycle test now verifies that the SDK-bound configuration uses `config.contents is None`. ### Testing Plan **Unit Tests:** - [x] I have added or updated unit tests for my change. - [x] All unit tests pass locally. Focused test result: - `tests/unittests/agents/test_gemini_context_cache_manager.py`: 33 passed, 5 warnings. The five warnings are unrelated to this change: four are `BaseAgentConfig` deprecation warnings, and one reports that the experimental `AGENT_CONFIG` feature is enabled. The relevant context-cache tests passed under tox with Python 3.10 through 3.14. Additional validation: - Ruff passed. - Pyink passed. - Pre-commit and compliance checks for the changed files passed. - The source distribution and wheel built successfully. - The wheel installed and imported successfully in a clean Python 3.12 environment. - `git diff --check` passed. **Manual End-to-End (E2E) Tests:** Not run because this environment cannot create and inspect a real Google cache resource. The official SDK documentation and source confirm the relevant boundary behavior: - [`CreateCachedContentConfig`](https://googleapis.github.io/python-genai/genai.html#genai.types.CreateCachedContentConfig) allows `contents` to be `None` and defines `system_instruction` and `tools` as separate fields. - The [SDK cache request builder](https://github.com/googleapis/python-genai/blob/v2.11.0/google/genai/caches.py#L270-L292) only transforms `contents` when it is not `None`; `system_instruction` and `tools` are handled independently. - The SDK’s [`t_contents()` implementation](https://github.com/googleapis/python-genai/blob/v2.11.0/google/genai/_transformers.py#L476-L482) explicitly rejects an empty list with `ValueError("contents are required.")`. Therefore, `None` omits the optional field, while `[]` enters the transformer and is rejected. The unit test verifies that ADK produces the valid side of this boundary. ### Checklist - [x] I have read the [CONTRIBUTING.md](https://github.com/google/adk-python/blob/main/CONTRIBUTING.md) document. - [x] I have performed a self-review of my own code. - [x] I have commented my code, particularly in hard-to-understand areas. No additional comments were needed for this focused boundary conversion. - [x] I have added tests that prove my fix is effective or that my feature works. - [x] New and existing unit tests pass locally with my changes. - [ ] I have manually tested my changes end-to-end. - [x] Any dependent changes have been merged and published in downstream modules. Not applicable; there are no dependent changes. ### Additional context The root cause of the missed regression in the existing tests is that `caches.create()` is mocked and therefore does not reproduce the Google Gen AI SDK’s complete request transformation and validation behavior. The mock accepted `contents=[]` and returned success, so the existing zero-prefix lifecycle test passed even though the real SDK rejects that value before sending the request. No user-facing API or documentation changes are required. Co-authored-by: Yi Liu <yiliuly@google.com> COPYBARA_INTEGRATE_REVIEW=#6367 from hxaxd:fix/gemini-cache-empty-contents 22f1a8b PiperOrigin-RevId: 947285615
Collaborator
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Link to Issue or Description of Change
Problem:
PR #6067 correctly changed fingerprint-only cache metadata to use the cacheable conversation prefix rather than all request contents. This keeps trailing user contents, including request-scoped dynamic instructions, out of the stable cache identity.
For an initial request containing only the latest user content, the cacheable conversation prefix is legitimately empty. ADK therefore stores fingerprint-only metadata with
contents_count == 0. It does not create a cache on this initial request because no token count from a previous response is available yet.On the next request, ADK uses the stored count to calculate the current fingerprint. When the system instruction and tools remain unchanged, that fingerprint matches the stored fingerprint. If the system instruction and tools are large enough to meet Gemini’s cache-size requirement, ADK makes its first real cache-creation attempt.
The stored count is deliberately reused because it defines the prefix covered by the matched fingerprint. Even if the later request now has a non-empty cacheable conversation prefix, ADK still slices it with the stored zero. This produces
llm_request.contents[:0] == [], which is passed asCreateCachedContentConfig(contents=[]).The Google Gen AI SDK distinguishes an omitted
contentsfield from an explicit empty list. It rejectscontents=[]withValueError: contents are required, whilecontents=Noneomits only the conversation contents and still allows the system instruction and tools to be included in the cache request.ADK catches the exception, logs a warning, and continues the model request without a cache, so the request itself does not crash. Because no
cache_namewas created, the caller preserves the fingerprint-only metadata and its originalcontents_count == 0. This behavior is intentional so transient cache-creation failures can retry the same cache identity.As long as the system instruction and tools remain unchanged, the next request again matches the same fingerprint using the preserved count. ADK then retries the same invalid
contents=[]configuration, logs another warning, and again receives nocache_name. This cycle repeats on every matching request, so the cache is never established.Solution:
Pass
Nonewhen the cacheable conversation prefix is empty so the SDK omits thecontentsfield. Preserve the existing list behavior for non-empty prefixes.The existing zero-prefix lifecycle test now verifies that the SDK-bound configuration uses
config.contents is None.Testing Plan
Unit Tests:
Focused test result:
tests/unittests/agents/test_gemini_context_cache_manager.py: 33 passed, 5 warnings.The five warnings are unrelated to this change: four are
BaseAgentConfigdeprecation warnings, and one reports that the experimentalAGENT_CONFIGfeature is enabled.The relevant context-cache tests passed under tox with Python 3.10 through 3.14.
Additional validation:
git diff --checkpassed.Manual End-to-End (E2E) Tests:
Not run because this environment cannot create and inspect a real Google cache resource.
The official SDK documentation and source confirm the relevant boundary behavior:
CreateCachedContentConfigallowscontentsto beNoneand definessystem_instructionandtoolsas separate fields.contentswhen it is notNone;system_instructionandtoolsare handled independently.t_contents()implementation explicitly rejects an empty list withValueError("contents are required.").Therefore,
Noneomits the optional field, while[]enters the transformer and is rejected. The unit test verifies that ADK produces the valid side of this boundary.Checklist
Additional context
The root cause of the missed regression in the existing tests is that
caches.create()is mocked and therefore does not reproduce the Google Gen AI SDK’s complete request transformation and validation behavior. The mock acceptedcontents=[]and returned success, so the existing zero-prefix lifecycle test passed even though the real SDK rejects that value before sending the request.No user-facing API or documentation changes are required.