Add 'last two messages' cache breakpoint strategy for Messages API#312472
Merged
Add 'last two messages' cache breakpoint strategy for Messages API#312472
Conversation
Contributor
There was a problem hiding this comment.
Pull request overview
Adds an experiment-gated alternative cache breakpoint placement strategy for the Anthropic Messages API, shifting cache_control markers to the last two merged messages (instead of relying on prompt/heuristic breakpoint placement) while preserving the existing tools/system prefix caching behavior.
Changes:
- Introduces
addLastTwoMessagesCacheControland wires it intocreateMessagesRequestBodybehindchat.anthropic.cacheBreakpoints.lastTwoMessages. - Disables prompt-tsx + heuristic cache breakpoints in the agent prompt path when the experiment is enabled.
- Adds the new experiment-based setting to configuration + extension settings metadata and adds unit tests for the new strategy.
Show a summary per file
| File | Description |
|---|---|
| extensions/copilot/src/platform/endpoint/node/messagesApi.ts | Adds the new “last two messages” cache marker strategy and enables it via experiment in request body construction. |
| extensions/copilot/src/platform/endpoint/test/node/messagesApi.spec.ts | Adds a dedicated test suite validating expected cache marker placement and limit behavior. |
| extensions/copilot/src/platform/configuration/common/configurationService.ts | Registers the new experiment-based config key. |
| extensions/copilot/src/extension/intents/node/agentIntent.ts | Suppresses upstream prompt/heuristic cache breakpoint insertion when the new strategy is enabled. |
| extensions/copilot/package.json | Exposes the new setting in extension contributes/configuration. |
| extensions/copilot/package.nls.json | Adds the localized description for the new setting. |
Copilot's findings
- Files reviewed: 6/6 changed files
- Comments generated: 2
bhavyaus
force-pushed
the
dev/bhavyau/last-two-messages-cache-breakpoints
branch
2 times, most recently
from
daebb6e to
3d438e1
Compare
Adds an experiment-gated alternative to the heuristic-based cache breakpoint placement. Places cache_control on the last two merged Anthropic messages instead of using prompt-tsx markers and addCacheBreakpoints. Gated behind AnthropicCacheBreakpointsLastTwoMessages (default false, onExp).
bhavyaus
force-pushed
the
dev/bhavyau/last-two-messages-cache-breakpoints
branch
from
3d438e1 to
e487cd3
Compare
roblourens
approved these changes
Apr 26, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Experiment-gated alternative cache breakpoint placement for Anthropic Messages API. Places
cache_controlon the last two merged messages instead of heuristic-based placement.Behind
chat.anthropic.cacheBreakpoints.lastTwoMessages(default false, onExp).Ref: microsoft/vscode-internalbacklog#7315