Skip to content

fix(models): Allow context caches with empty content prefixes - #6367

Closed
hxaxd wants to merge 5 commits into
google:mainfrom
hxaxd:fix/gemini-cache-empty-contents
Closed

fix(models): Allow context caches with empty content prefixes#6367
hxaxd wants to merge 5 commits into
google:mainfrom
hxaxd:fix/gemini-cache-empty-contents

Conversation

@hxaxd

@hxaxd hxaxd commented Jul 10, 2026

Copy link
Copy Markdown
Contributor

Link to Issue or Description of Change

Problem:

PR #6067 correctly changed fingerprint-only cache metadata to use the cacheable conversation prefix rather than all request contents. This keeps trailing user contents, including request-scoped dynamic instructions, out of the stable cache identity.

For an initial request containing only the latest user content, the cacheable conversation prefix is legitimately empty. ADK therefore stores fingerprint-only metadata with contents_count == 0. It does not create a cache on this initial request because no token count from a previous response is available yet.

On the next request, ADK uses the stored count to calculate the current fingerprint. When the system instruction and tools remain unchanged, that fingerprint matches the stored fingerprint. If the system instruction and tools are large enough to meet Gemini’s cache-size requirement, ADK makes its first real cache-creation attempt.

The stored count is deliberately reused because it defines the prefix covered by the matched fingerprint. Even if the later request now has a non-empty cacheable conversation prefix, ADK still slices it with the stored zero. This produces llm_request.contents[:0] == [], which is passed as CreateCachedContentConfig(contents=[]).

The Google Gen AI SDK distinguishes an omitted contents field from an explicit empty list. It rejects contents=[] with ValueError: contents are required, while contents=None omits only the conversation contents and still allows the system instruction and tools to be included in the cache request.

ADK catches the exception, logs a warning, and continues the model request without a cache, so the request itself does not crash. Because no cache_name was created, the caller preserves the fingerprint-only metadata and its original contents_count == 0. This behavior is intentional so transient cache-creation failures can retry the same cache identity.

As long as the system instruction and tools remain unchanged, the next request again matches the same fingerprint using the preserved count. ADK then retries the same invalid contents=[] configuration, logs another warning, and again receives no cache_name. This cycle repeats on every matching request, so the cache is never established.

Solution:

Pass None when the cacheable conversation prefix is empty so the SDK omits the contents field. Preserve the existing list behavior for non-empty prefixes.

The existing zero-prefix lifecycle test now verifies that the SDK-bound configuration uses config.contents is None.

Testing Plan

Unit Tests:

  • I have added or updated unit tests for my change.
  • All unit tests pass locally.

Focused test result:

  • tests/unittests/agents/test_gemini_context_cache_manager.py: 33 passed, 5 warnings.

The five warnings are unrelated to this change: four are BaseAgentConfig deprecation warnings, and one reports that the experimental AGENT_CONFIG feature is enabled.

The relevant context-cache tests passed under tox with Python 3.10 through 3.14.

Additional validation:

  • Ruff passed.
  • Pyink passed.
  • Pre-commit and compliance checks for the changed files passed.
  • The source distribution and wheel built successfully.
  • The wheel installed and imported successfully in a clean Python 3.12 environment.
  • git diff --check passed.

Manual End-to-End (E2E) Tests:

Not run because this environment cannot create and inspect a real Google cache resource.

The official SDK documentation and source confirm the relevant boundary behavior:

Therefore, None omits the optional field, while [] enters the transformer and is rejected. The unit test verifies that ADK produces the valid side of this boundary.

Checklist

  • I have read the CONTRIBUTING.md document.
  • I have performed a self-review of my own code.
  • I have commented my code, particularly in hard-to-understand areas. No additional comments were needed for this focused boundary conversion.
  • I have added tests that prove my fix is effective or that my feature works.
  • New and existing unit tests pass locally with my changes.
  • I have manually tested my changes end-to-end.
  • Any dependent changes have been merged and published in downstream modules. Not applicable; there are no dependent changes.

Additional context

The root cause of the missed regression in the existing tests is that caches.create() is mocked and therefore does not reproduce the Google Gen AI SDK’s complete request transformation and validation behavior. The mock accepted contents=[] and returned success, so the existing zero-prefix lifecycle test passed even though the real SDK rejects that value before sending the request.

No user-facing API or documentation changes are required.

@google-cla

google-cla Bot commented Jul 10, 2026

Copy link
Copy Markdown

Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

View this failed invocation of the CLA check for more information.

For the most up to date status, view the checks section at the bottom of the pull request.

Omit the contents field when the cacheable prefix is empty so the Google Gen AI SDK can create a cache from the system instruction and tools. Add a regression assertion for the zero-prefix request.

Fixes google#6363
@hxaxd
hxaxd force-pushed the fix/gemini-cache-empty-contents branch from d51915b to 7b2808b Compare July 10, 2026 15:31
@rohityan rohityan self-assigned this Jul 10, 2026
@rohityan rohityan added the models [Component] This issue is related to model support label Jul 10, 2026
@i-yliu i-yliu assigned i-yliu and unassigned rohityan Jul 13, 2026
copybara-service Bot pushed a commit that referenced this pull request Jul 13, 2026
Merge #6367

### Link to Issue or Description of Change

- Closes: #6363

**Problem:**

[PR #6067](#6067) correctly changed fingerprint-only cache metadata to use the cacheable conversation prefix rather than all request contents. This keeps trailing user contents, including request-scoped dynamic instructions, out of the stable cache identity.

For an initial request containing only the latest user content, the cacheable conversation prefix is legitimately empty. ADK therefore stores fingerprint-only metadata with `contents_count == 0`. It does not create a cache on this initial request because no token count from a previous response is available yet.

On the next request, ADK uses the stored count to calculate the current fingerprint. When the system instruction and tools remain unchanged, that fingerprint matches the stored fingerprint. If the system instruction and tools are large enough to meet Gemini’s cache-size requirement, ADK makes its first real cache-creation attempt.

The stored count is deliberately reused because it defines the prefix covered by the matched fingerprint. Even if the later request now has a non-empty cacheable conversation prefix, ADK still slices it with the stored zero. This produces `llm_request.contents[:0] == []`, which is passed as `CreateCachedContentConfig(contents=[])`.

The Google Gen AI SDK distinguishes an omitted `contents` field from an explicit empty list. It rejects `contents=[]` with `ValueError: contents are required`, while `contents=None` omits only the conversation contents and still allows the system instruction and tools to be included in the cache request.

ADK catches the exception, logs a warning, and continues the model request without a cache, so the request itself does not crash. Because no `cache_name` was created, the caller preserves the fingerprint-only metadata and its original `contents_count == 0`. This behavior is intentional so transient cache-creation failures can retry the same cache identity.

As long as the system instruction and tools remain unchanged, the next request again matches the same fingerprint using the preserved count. ADK then retries the same invalid `contents=[]` configuration, logs another warning, and again receives no `cache_name`. This cycle repeats on every matching request, so the cache is never established.

**Solution:**

Pass `None` when the cacheable conversation prefix is empty so the SDK omits the `contents` field. Preserve the existing list behavior for non-empty prefixes.

The existing zero-prefix lifecycle test now verifies that the SDK-bound configuration uses `config.contents is None`.

### Testing Plan

**Unit Tests:**

- [x] I have added or updated unit tests for my change.
- [x] All unit tests pass locally.

Focused test result:

- `tests/unittests/agents/test_gemini_context_cache_manager.py`: 33 passed, 5 warnings.

The five warnings are unrelated to this change: four are `BaseAgentConfig` deprecation warnings, and one reports that the experimental `AGENT_CONFIG` feature is enabled.

The relevant context-cache tests passed under tox with Python 3.10 through 3.14.

Additional validation:

- Ruff passed.
- Pyink passed.
- Pre-commit and compliance checks for the changed files passed.
- The source distribution and wheel built successfully.
- The wheel installed and imported successfully in a clean Python 3.12 environment.
- `git diff --check` passed.

**Manual End-to-End (E2E) Tests:**

Not run because this environment cannot create and inspect a real Google cache resource.

The official SDK documentation and source confirm the relevant boundary behavior:

- [`CreateCachedContentConfig`](https://googleapis.github.io/python-genai/genai.html#genai.types.CreateCachedContentConfig) allows `contents` to be `None` and defines `system_instruction` and `tools` as separate fields.
- The [SDK cache request builder](https://github.com/googleapis/python-genai/blob/v2.11.0/google/genai/caches.py#L270-L292) only transforms `contents` when it is not `None`; `system_instruction` and `tools` are handled independently.
- The SDK’s [`t_contents()` implementation](https://github.com/googleapis/python-genai/blob/v2.11.0/google/genai/_transformers.py#L476-L482) explicitly rejects an empty list with `ValueError("contents are required.")`.

Therefore, `None` omits the optional field, while `[]` enters the transformer and is rejected. The unit test verifies that ADK produces the valid side of this boundary.

### Checklist

- [x] I have read the [CONTRIBUTING.md](https://github.com/google/adk-python/blob/main/CONTRIBUTING.md) document.
- [x] I have performed a self-review of my own code.
- [x] I have commented my code, particularly in hard-to-understand areas. No additional comments were needed for this focused boundary conversion.
- [x] I have added tests that prove my fix is effective or that my feature works.
- [x] New and existing unit tests pass locally with my changes.
- [ ] I have manually tested my changes end-to-end.
- [x] Any dependent changes have been merged and published in downstream modules. Not applicable; there are no dependent changes.

### Additional context

The root cause of the missed regression in the existing tests is that `caches.create()` is mocked and therefore does not reproduce the Google Gen AI SDK’s complete request transformation and validation behavior. The mock accepted `contents=[]` and returned success, so the existing zero-prefix lifecycle test passed even though the real SDK rejects that value before sending the request.

No user-facing API or documentation changes are required.

Co-authored-by: Yi Liu <yiliuly@google.com>
COPYBARA_INTEGRATE_REVIEW=#6367 from hxaxd:fix/gemini-cache-empty-contents 22f1a8b
PiperOrigin-RevId: 947285615
@adk-bot

adk-bot commented Jul 13, 2026

Copy link
Copy Markdown
Collaborator

Thank you @hxaxd for your contribution! 🎉

Your changes have been successfully imported and merged via Copybara in commit ee7174d.

Closing this PR as the changes are now in the main branch.

@adk-bot adk-bot added the merged [Status] This PR is merged label Jul 13, 2026
@adk-bot adk-bot closed this Jul 13, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

merged [Status] This PR is merged models [Component] This issue is related to model support

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Context cache creation fails with 'contents are required' when cacheable prefix is empty (cache_contents_count == 0)

4 participants