fix(handler): let the ProxyCached path request a per-call Accept-Encoding

[pinguinfuss]

Since #304 every ProxyCached caller asks upstream for identity. Right for signed indexes, but it drops transfer compression for everyone — the Homebrew API alone refetches ~33 MB of formula.jws.json that would be ~5 MB gzipped (#305).

The internal verbatim bool becomes an acceptEncoding string ("" = leave unset, identity, gzip). ProxyCached still sends identity, so nothing changes for the other callers. Homebrew asks for gzip on the API paths — brew fetches them with curl --compressed and decodes the header itself — except analytics/, which brew fetches without --compressed. Setting the header ourselves keeps Go from decompressing, so the compressed bytes and Content-Encoding are cached and replayed as they came in.

Three things had to move with that, all only harmful once a caller can ask for gzip:

• fetchOrCacheMetadata returns the encoding of the body it hands back and the writer uses it, instead of re-reading the cache row. Otherwise a failed Storage.Store served gzip bytes as application/json with no header. writeMetadataCachedResponse keeps its signature (helm/maven untouched) and delegates to a …WithEncoding sibling.
• cacheMetadataBlob no longer swallows the row-write error: it deletes the blob it just stored, so a later hit can't serve a gzip blob under the old row.
• The stale fallback re-reads the row before serving, so a request that lost the upstream race to one storing a gzip blob doesn't label it with the identity row.

Not switched globally on purpose: apt, apk, dnf, hex, R, bundler and Pkg.jl don't decode unsolicited Content-Encoding: gzip, and their upstreams send the same bytes either way.

Tests in proxy_cached_encoding_test.go and homebrew_test.go; #304's identity tests are unchanged. The ETag / Last-Modified validators still come from the cache row — that predates #304.

Fixes #305

[Copilot]

🟡 Changes recommended

Response conditional/validator handling can become inconsistent with the served body when cache writes fail, which can lead to incorrect headers or 304 responses.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Pull request overview

This PR refactors the cached-metadata fetch path to support a per-call upstream Accept-Encoding policy, enabling callers to opt into requesting gzip (and caching/replaying the encoded bytes) while keeping the default ProxyCached behavior as Accept-Encoding: identity for byte-for-byte fidelity of signed or hash-pinned metadata.

Changes:

• Replace the internal verbatim bool with acceptEncoding string and thread it through upstream fetch + streaming paths to control Go’s transparent decompression behavior.
• Return the body’s effective Content-Encoding from fetchOrCacheMetadata and use it when writing responses to avoid mislabeling bytes when cache writes fail.
• Add proxyCachedWithEncoding and introduce focused tests covering gzip caching/replay, streaming mode, store-failure behavior, and stale-cache fallback.

File summaries

File	Description
internal/handler/handler.go	Adds per-call upstream Accept-Encoding handling and plumbs Content-Encoding through fetch/cache/write paths.
internal/handler/proxy_cached_encoding_test.go	New tests validating gzip behavior across cached, streaming, store-failure, and stale-fallback scenarios.

Review details

Suppressed comments (1)

internal/handler/handler.go:1003

• writeMetadataCachedResponseWithEncoding always looks up ETag/Last-Modified from the DB cache entry, even when serving a freshly-fetched body after a cache write failure. If Storage.Store or the DB upsert fails but an older cache entry exists, this can emit validators that don’t match the bytes being served (and can incorrectly return 304 to If-None-Match/If-Modified-Since). Consider having fetchOrCacheMetadata return the validators for the body it returns (etag/lastModified/stale) and/or a flag indicating whether the cache row was successfully updated, and only apply conditional logic based on those values when they correspond to the served body.

	cm := p.lookupCachedMeta(ecosystem, cacheKey)

	if cm.etag != "" {
		w.Header().Set(headerETag, cm.etag)
	}
	if !cm.lastModified.IsZero() {
		w.Header().Set(headerLastModified, cm.lastModified.UTC().Format(http.TimeFormat))
	}
	if ifNoneMatchHits(r.Header.Get("If-None-Match"), cm.etag) {
		w.WriteHeader(http.StatusNotModified)
		return
	}

• Files reviewed: 2/2 changed files
• Comments generated: 1
• Review effort level: Lite

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.