feat(prompts): expose prompt cache operations#89
Merged
Conversation
There was a problem hiding this comment.
Pull request overview
This PR adds public, flat prompt-cache APIs and fetch metadata so SDK consumers can inspect cache keys/status, refresh prompts, and perform invalidation without relying on private cache internals. It also introduces generation-based invalidation for both in-memory and Rails cache backends, plus per-call TTL overrides and event hooks.
Changes:
- Introduce
PromptFetchResult/PromptCacheKeyand addClient#get_prompt_resultplus flat cache operations (refresh/invalidate/clear/stats/key inspection/validation). - Add generation-based storage keys and name/global logical invalidation across memory + Rails cache adapters (avoids prefix scans).
- Add per-call
cache_ttl(including0bypass) and prompt cache events via observer + ActiveSupport instrumentation; addcache_backend = :auto.
Reviewed changes
Copilot reviewed 16 out of 16 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
| spec/langfuse/prompt_cache_spec.rb | Adds coverage for generation-based storage keys, logical invalidation, and delete semantics. |
| spec/langfuse/config_spec.rb | Adds validation tests for cache_backend: :auto and callable prompt_cache_observer. |
| spec/langfuse/client_spec.rb | Adds tests for :auto backend selection and client-side fetch metadata / cache inspection APIs. |
| spec/langfuse/api_client_spec.rb | Adds tests for API-client-level cache operations (metadata, bypass, refresh, invalidation, events). |
| lib/langfuse/stale_while_revalidate.rb | Extends SWR to support TTL overrides, async refresh scheduling, and write helpers. |
| lib/langfuse/rails_cache_adapter.rb | Adds generation-based invalidation + key generation, stats, delete/entry helpers, TTL override support. |
| lib/langfuse/prompt_fetch_result.rb | Introduces public result/key objects for prompt fetch metadata and cache key inspection. |
| lib/langfuse/prompt_cache.rb | Adds generation-based keying/invalidation, stats, delete/entry helpers, TTL override support. |
| lib/langfuse/config.rb | Adds :auto backend + prompt_cache_observer configuration and validation. |
| lib/langfuse/client.rb | Exposes flat client cache APIs and get_prompt_result; wires observer into ApiClient and supports per-call cache_ttl. |
| lib/langfuse/api_client.rb | Implements prompt fetch metadata, cache operations, mutation invalidation, and cache event instrumentation. |
| lib/langfuse.rb | Requires new public prompt fetch result types. |
| docs/PROMPTS.md | Documents per-call cache TTL/bypass and operational visibility APIs. |
| docs/CONFIGURATION.md | Documents cache_backend: :auto and prompt_cache_observer. |
| docs/CACHING.md | Adds “Public Cache Operations” section, cache key semantics, invalidation model, and events. |
| docs/API_REFERENCE.md | Updates signatures and documents get_prompt_result + flat cache operations. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
- Centralize label-defaulting on PromptCacheKey#resolved_label - Share event payload builder between ApiClient and Client - Memoize cache backend name and ActiveSupport::Notifications lookup - Skip event emission early when no observers are configured - Drop dead to_sym coercions and uncalled fetch_with_simple_cache helpers
3 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
TL;DRExpose flat prompt cache inspection, refresh, invalidation, and fetch metadata APIs.
WhyPrompt consumers need a supported way to inspect cache hit or miss source, bypass or refresh prompt fetches, and invalidate cache entries without relying on private cache internals.
ChecklistLinear: AAI-106
SummaryPromptFetchResult/PromptCacheKeyand flat client APIs for cache metadata, refresh, invalidation, stats, and key inspection.cache_ttl,cache_ttl: 0bypass, mutation invalidation, observer / ActiveSupport cache events, andcache_backend = :auto.VerificationRBENV_VERSION=3.2.0 rbenv exec bundle _2.4.7_ exec rspecRBENV_VERSION=3.2.0 rbenv exec bundle _2.4.7_ exec rubocop --format simpleRBENV_VERSION=3.2.0 rbenv exec bundle _2.4.7_ exec ruby scratchpad/aai_106_prompt_cache_validate.rbnpx --yes langfuse-cli api prompts get <validated prompt>npx --yes langfuse-cli api prompts list --limit 10npx --yes langfuse-cli api prompts delete <validated prompt>