feat: add block-level caching for ByteSource#7
Merged
Conversation
Adds a new block cache that caches fixed-size blocks from ByteSources, optimizing random access reads from remote sources. Key features: - BlockCache interface wraps ByteSource with block-level caching - Disk-backed implementation with configurable block size (default 64KB) - MaxBlocksPerRead bypass prevents caching large sequential reads - LRU-style pruning by modification time - Singleflight deduplication for concurrent block fetches - SourceID method on ByteSource for stable cache key generation Also adds cache size management to existing disk cache: - WithMaxBytes option for size limits - MaxBytes(), SizeBytes(), Prune() methods on Cache interface Documentation updated with block caching guide, cache pruning docs, and API reference for all new interfaces and options. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Deploying with
|
| Status | Name | Latest Commit | Preview URL | Updated (UTC) |
|---|---|---|---|---|
| ✅ Deployment successful! View logs |
blob | d11bdca | Commit Preview URL Branch Preview URL |
Jan 17 2026, 09:10 PM |
- Add nolint comment for intentionally ignored writeBlock error - Add nolint comment for safe file read from hash-derived path - Add nolint comments for validated int64->uint64 conversions - Combine parameter types in pruneDir signature (gocritic) - Use string concatenation instead of fmt.Sprintf (perfsprint) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
ByteSourceto optimize random access reads from remote sourcesBlock Cache
New
BlockCacheinterface wrapsByteSourcewith fixed-size block caching:MaxBlocksPerReadbypass prevents caching large sequential reads (e.g., CopyDir)SourceID()method onByteSourceprovides stable cache key generationCache Size Management
Extended
Cacheinterface with size tracking and pruning:WithMaxBytesoption sets cache size limitMaxBytes(),SizeBytes()for monitoringPrune(targetBytes)for manual evictionDocumentation
docs/guides/block-caching.mdcaching.mdwith cache size management sectionperformance-tuning.mdwith block cache tuningTest plan
just cito verify linting and tests passgo test -bench=BlockCache -run=^$ ./...cd docs && npm run buildcd docs && npm run start🤖 Generated with Claude Code