Skip to content

Subtract the Age header when computing a cache deadline - #538

Merged
maennchen merged 2 commits into
erlef:mainfrom
ericmj:http-util-cache-age
Aug 4, 2026
Merged

Subtract the Age header when computing a cache deadline#538
maennchen merged 2 commits into
erlef:mainfrom
ericmj:http-util-cache-age

Conversation

@ericmj

@ericmj ericmj commented Aug 4, 2026

Copy link
Copy Markdown
Collaborator

headers_to_cache_deadline/2 computes freshness from max-age alone. RFC 7234 §4.2
defines it as max-age minus the Age header, and Age is what a shared cache reports
about how long it has already been holding the response. Dropping that term means a
document a CDN has held for most of its lifetime is treated as brand new.

That's measurable against a live provider. Two assertions in
oidcc_provider_configuration_SUITE hard-code 3_600_000 for Google's discovery expiry;
with the Age subtraction they come back around 1_751_000, because Google served that
response with roughly 1849 seconds of age. So oidcc reported a document with 30 minutes
of freshness left as having a full hour. As Age approaches max-age the reported
lifetime approaches double the truth. Those two assertions become range checks here,
since the exact number now depends on how long the CDN had the response.

It matters most to oidcc_provider_configuration_worker, which drives erlang:send_after/3
off this value and so schedules its refresh late by whatever the Age was. Callers that
persist the derived timestamp rather than holding it in one process carry the error for
as long as they cache it.

The subtraction alone isn't safe, which is the other half of this change. Before it, the
only route to a small deadline was a provider sending a small max-age, and exactly zero
was impossible because clamp_expiry/2 sends non-positive values to the fallback. After
it, any response a shared cache is serving near the end of its lifetime produces one, and
zero is reachable. The worker hands that straight to erlang:send_after/3 and the backoff
only fires on {error, _}, so it's an immediate refetch with nothing damping it: 6 requests
in 3 seconds against 2. So oidcc_provider_configuration floors the deadline with
max(Deadline, min(?MINIMUM_REFRESH, DefaultExpiry)), one minute. The inner min keeps a
caller that deliberately asked for a shorter fallback_expiry from having it overruled.
Flooring there rather than in the worker is what keeps Expiry :: pos_integer() true in
the loaders' specs instead of widening to admit zero.

no-cache and no-store are deliberately left alone. Honouring them means returning zero,
which is issue #370: kanidm sends no-store, no-cache, max-age=0 on discovery, and commit
31c7f3d fixed the resulting crash by falling back to the caller's expiry. Reversing that
here would reintroduce it, and an RP refetching discovery on every request is not what a
provider misconfiguring its cache headers should get.

headers_to_cache_deadline/2's return widens from pos_integer() to non_neg_integer(),
which is the one part of this that isn't strictly additive. It's the only export in
oidcc_http_util without -doc false, so it's hexdocs-visible by omission rather than by
intent, and both in-tree callers go through oidcc_provider_configuration, which floors
the value before it reaches anyone. Say the word if you'd rather it were hidden.

@ericmj
ericmj force-pushed the http-util-cache-age branch 2 times, most recently from 85aceed to 0b1959b Compare August 4, 2026 16:04
@ericmj
ericmj marked this pull request as ready for review August 4, 2026 16:06
Comment thread src/oidcc_http_util.erl Outdated
`headers_to_cache_deadline/2` read only `max-age`, so a document a shared cache
had already held for most of its stated lifetime was treated as fully fresh.
Google serves discovery through a CDN, and the suite's exact 3_600_000
assertions became a range because of it.

`Age` is subtracted from a lifetime the server stated, and only from that. The
caller's `fallback_expiry` is a refresh interval it chose rather than a claim by
the server, so an intermediary's `Age` says nothing about it and does not reduce
it. `max-age=0` still lands on the fallback, as issue erlef#370 needs.

Nothing left of a stated lifetime is reported as `0`, which widens the return
type from `pos_integer()` to `non_neg_integer()`. That is not a usable refresh
interval, so the loaders hold the result above a new `minimum_refresh` option,
itself bounded by `fallback_expiry` so a caller asking for a short interval
still gets it. Without that floor a nearly-stale response schedules an immediate
reload, which for the provider configuration worker is a sustained request loop.

`no-cache` and `no-store` are left alone. Honouring them means choosing a
refresh interval for a library that has to cache to function, and that decision
wants its own discussion.
@ericmj
ericmj force-pushed the http-util-cache-age branch from 0b1959b to 5316682 Compare August 4, 2026 20:44
`mix test.coverage` and `rebar3 coveralls send` are sibling jobs with the
same `needs`, and both downloaded every artifact in the run. The first
uploads its `cover` directory as `coverage-report`, which holds the same
three `.coverdata` files the second unpacks, so once that upload lands
`mv artifacts/*/*.coverdata` gets each basename twice and fails:

    mv: will not overwrite just-created
    '_build/test/cover/ct-OTP-27.3.4.coverdata' with
    'artifacts/ct-coverage-27.3.4/ct-OTP-27.3.4.coverdata'

Whether it fires depends on which job downloads first, so it is
intermittent on a first run and certain on a re-run, where the previous
attempt's `coverage-report` is always present.

Both jobs use only the coverdata, so restrict the download to the
artifacts that carry it.
@maennchen
maennchen merged commit 5fd55f4 into erlef:main Aug 4, 2026
28 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants