Respect ESI Expires header to skip HTTP requests within TTL window#10
Merged
ronaldvdmeer merged 5 commits intomainfrom Mar 29, 2026
Merged
Respect ESI Expires header to skip HTTP requests within TTL window#10ronaldvdmeer merged 5 commits intomainfrom
ronaldvdmeer merged 5 commits intomainfrom
Conversation
- Parse the Expires header on successful GET responses and store it as expires_at (datetime | None) alongside the ETag in the cache (cache tuple extended from 3 to 4 elements) - Add _get_fresh_cached() to check whether a cache entry is still within its TTL; _request_full() short-circuits immediately if so, avoiding any HTTP request and protecting the ESI error quota - Add _finalize_response() to encapsulate JSON parsing, x_pages extraction and ETag/Expires storage after a successful response - Malformed or absent Expires headers are handled gracefully (None stored; falls through to the existing ETag/If-None-Match flow) - Add TestExpiresTTLCaching with 5 tests covering storage, TTL hit, TTL miss, malformed header, and absent header - Update docs/error-handling.md to document both caching layers
- Normalize naive Expires datetimes to UTC instead of storing them as-is (would cause TypeError when comparing with datetime.now(UTC)) - Extract _parse_expires() static method to avoid duplicated parsing logic between _store_etag and the 304 branch - Update 304 branch to refresh the Expires timestamp from the response headers so re-validated entries re-enter the TTL window - Update _request_full docstring to describe both TTL and ETag layers - Fix test_fresh_cache_skips_http_request: keep both calls inside an aioresponses() context so a regression never hits the real network - Update docs/error-handling.md: accurate wording that caching only applies when a cacheable GET response with ETag was received
- Fix _parse_expires docstring: timezone-aware not UTC-aware (only naive values are patched to UTC; other timezones are kept as-is) - Fix _finalize_response docstring: applies to any 2xx response, not only GET (caching is still GET-only via _store_etag) - Add If-None-Match assertion to test_expired_cache_sends_etag_request to verify the ETag header is actually sent after TTL expiry
This was referenced Mar 29, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The client now respects the
Expiresheader returned by ESI, skipping HTTP requests entirely when cached data is still fresh. This is the most quota-efficient caching layer: no network traffic is generated until the server-side cache window closes.Details
New caching layers (two-tier)
Expires)If-None-Match/ 304)Implementation
Expiresheader is parsed viaemail.utils.parsedate_to_datetimeand stored asexpires_at: datetime | Nonealongside the ETag (cache tuple extended from 3 to 4 elements)_get_fresh_cached(method, cache_key)helper: returns(data, x_pages)if the cache entry exists and has not expired, otherwiseNone_request_full()short-circuits immediately when_get_fresh_cachedreturns a result, before any HTTP call is initiated_finalize_response(method, cache_key, response)helper extracts JSON, parses theX-Pagesheader, and stores the ETag/Expires entry — cleans up_request_fullinternalsExpiresheaders storeNone; the ETag flow proceeds as beforeDocumentation
docs/error-handling.md— updated "ESI caching headers" section to document both the TTL and ETag caching layers and removed the outdated note claiming the library does not cache responsesTesting
TestExpiresTTLCaching— 5 new tests:test_expires_stored_in_cache— validExpiresheader is parsed and storedtest_fresh_cache_skips_http_request— no HTTP call when TTL is still validtest_expired_cache_sends_etag_request— ETag flow triggered after TTL expirestest_malformed_expires_stores_none— bad header stored asNonetest_no_expires_header_stores_none— absent header stored asNoneResult: 172 tests passed, 98.40% coverage, pylint 10.00/10, ruff + mypy clean