Skip to content

fix: storage preview cache misses and stale cache eviction#11842

Merged
loks0n merged 1 commit into
1.9.xfrom
fix/storage-cache
Apr 9, 2026
Merged

fix: storage preview cache misses and stale cache eviction#11842
loks0n merged 1 commit into
1.9.xfrom
fix/storage-cache

Conversation

@loks0n

@loks0n loks0n commented Apr 9, 2026

Copy link
Copy Markdown
Member

Summary

  • Token in cache key: The token auth param was included in cache keys for storage preview, so every request using a resource token generated a unique key and never hit cache. Introduced a cache.params route label to opt-in specific params; preview now declares only the image transform params.
  • Cache hits never refreshed TTL: accessedAt and filesystem mtime were only updated in the shutdown hook, which is skipped on cache hits (response already sent in init hook). After 30 days the maintenance job evicted still-active entries, causing periodic full re-renders. The cache-hit path now refreshes both on the 24h APP_CACHE_UPDATE interval.
  • Full document update in preview action: updateDocument was passing the full file document when only transformedAt changed; changed to a sparse document to match conventions.

Test plan

  • Request a file preview with a resource token multiple times — confirm only the first renders (X-Appwrite-Cache: hit on subsequent requests)
  • Request a file preview with different tokens but same transform params — confirm same cache entry is reused
  • Request a file preview with different transform params (width, height, etc.) — confirm separate cache entries
  • Verify accessedAt in the cache collection updates on cache hits after 24h
  • Confirm the maintenance job no longer evicts active preview cache entries after 30 days

🤖 Generated with Claude Code

Three bugs causing storage preview cache to be ineffective:

1. Cache keys included the `token` auth parameter, so requests using
   resource tokens always generated unique keys and never hit cache.
   Introduced `cache.params` label for routes to opt-in specific params
   into the cache key; preview now declares only the transform params.

2. Cache hits never refreshed `accessedAt` in the DB or the filesystem
   file mtime, because `$response->send()` in the init hook skips the
   shutdown hook. After 30 days the maintenance job evicted still-active
   cache entries, and after the original 30-day filesystem TTL the cache
   file expired — causing periodic full re-renders. The cache-hit path
   now updates both on the APP_CACHE_UPDATE (24h) interval.

3. `updateDocument` in the preview action passed the full file document
   instead of a sparse one when updating `transformedAt`.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@greptile-apps

greptile-apps Bot commented Apr 9, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

This PR fixes three related storage-preview cache bugs: resource tokens were poisoning cache keys causing every token request to miss; accessedAt was never refreshed on cache hits (response sent in init hook, bypassing the shutdown hook), letting the maintenance job evict still-active entries; and updateDocument for transformedAt was needlessly passing the full file document. The fixes are well-scoped — the new cache.params label is opt-in and doesn't affect other cached routes, the init-hook accessedAt refresh correctly mirrors the existing shutdown-hook logic, and the sparse-document conversions follow project conventions.

Confidence Score: 5/5

Safe to merge; all remaining findings are P2 style suggestions with no correctness impact.

All three bug fixes are logically sound: the cache-key filtering preserves uniqueness via the URI, the accessedAt refresh correctly mirrors the shutdown-hook pattern, and the sparse-document updates follow established conventions. No P0 or P1 issues found.

No files require special attention.

Vulnerabilities

No security concerns identified. The cache.params filtering correctly excludes the token auth param from the cache key while the existing authorization checks in the cache-hit path (bucket/file read permissions, resource-token validation) remain intact, so token exclusion does not widen access to cached responses.

Important Files Changed

Filename Overview
src/Appwrite/Utopia/Request.php Adds opt-in cache.params label support to cacheIdentifier(), filtering query params to only those declared by the route; unchanged routes are unaffected (null guard).
app/controllers/shared/api.php Cache-hit path now refreshes accessedAt in the DB and re-saves the filesystem entry (updating mtime) within the APP_CACHE_UPDATE window, mirroring the shutdown-hook logic that is skipped when the response is sent early; also folds in the sparse-document fix for transformedAt in the storage-preview hot path.
src/Appwrite/Platform/Modules/Storage/Http/Buckets/Files/Preview/Get.php Declares cache.params with the 11 image-transform query params, and converts the transformedAt updateDocument call from a full file document to a sparse document per project conventions.

Reviews (1): Last reviewed commit: "fix: storage preview cache misses and st..." | Re-trigger Greptile

Comment on lines 280 to +283
$file->setAttribute('transformedAt', DateTime::now());
$authorization->skip(fn () => $dbForProject->updateDocument('bucket_' . $file->getAttribute('bucketInternalId'), $file->getId(), $file));
$authorization->skip(fn () => $dbForProject->updateDocument('bucket_' . $file->getAttribute('bucketInternalId'), $file->getId(), new Document([
'transformedAt' => $file->getAttribute('transformedAt'),
])));

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Unnecessary setAttribute/getAttribute round-trip

$file->setAttribute('transformedAt', …) is called only so that getAttribute can be passed to the sparse document; the intermediate write to $file is never used again. You can drop both calls and construct the value inline, matching the pattern used elsewhere in the codebase.

Suggested change
$file->setAttribute('transformedAt', DateTime::now());
$authorization->skip(fn () => $dbForProject->updateDocument('bucket_' . $file->getAttribute('bucketInternalId'), $file->getId(), $file));
$authorization->skip(fn () => $dbForProject->updateDocument('bucket_' . $file->getAttribute('bucketInternalId'), $file->getId(), new Document([
'transformedAt' => $file->getAttribute('transformedAt'),
])));
$authorization->skip(fn () => $dbForProject->updateDocument('bucket_' . $file->getAttribute('bucketInternalId'), $file->getId(), new Document([
'transformedAt' => DateTime::now(),
])));

@github-actions

github-actions Bot commented Apr 9, 2026

Copy link
Copy Markdown

🔄 PHP-Retry Summary

Flaky tests detected across commits:

Commit f2df9cb - 4 flaky tests
Test Retries Total Time Details
UsageTest::testFunctionsStats 1 10.21s Logs
UsageTest::testPrepareSitesStats 1 7ms Logs
UsageTest::testEmbeddingsTextUsageDoesNotBreakProjectUsage 1 5ms Logs
TablesDBTransactionsCustomServerTest::testBulkUpsertWithDependentDocuments 1 240.49s Logs

@github-actions

github-actions Bot commented Apr 9, 2026

Copy link
Copy Markdown

✨ Benchmark results

  • Requests per second: 1,708
  • Requests with 200 status code: 307,508
  • P99 latency: 0.106613733

⚡ Benchmark Comparison

Metric This PR Latest version
RPS 1,708 1,247
200 307,508 224,541
P99 0.106613733 0.16312342

@loks0n loks0n merged commit 53a114e into 1.9.x Apr 9, 2026
44 of 45 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants