v0.10.0

Proxy adds hosted npm and Composer packages, age-based cache retention, container image mirroring and schedules, and cache deletion through the API and web UI.

• Add Chef Supermarket and Arch Linux proxy support, Go checksum database proxying, private npm and Composer package routes, and OCI referrers support.
• Cache more Maven, Conan, and generic download artifacts. Improve archive browsing and support additional SBOM mirror inputs.
• Reduce npm and Composer metadata processing costs, gzip cached responses, preserve metadata on upstream 304 responses, and stream Conda cooldown filtering.
• Verify registry-provided artifact checksums, abort corrupt cached responses, limit upstream artifact sizes, restrict credential forwarding, and sandbox upstream content.
• Separate artifact and metadata cache metrics, expose more failure details, respect upstream rate limits, and make response and artifact download timeouts configurable.
• Update Go dependencies, including registries v0.11.0, enrichment v0.7.3, and purl v0.1.22.

Before upgrading:

• Gradle build cache uploads are disabled by default. Instances that accept CI uploads need gradle.build_cache.read_only: false or PROXY_GRADLE_BUILD_CACHE_READ_ONLY=false.
• storage.max_size now includes cached metadata. Existing installations may evict metadata or artifacts when the combined total exceeds the configured limit.
• Cache hit and miss metrics now include a kind label for artifact or metadata; check custom Prometheus queries and dashboards.
• proxy mirror returns a nonzero exit status when a package fails and refuses CLI mirroring while scanning is enabled.

Changelog

• 086abeb A database refresh now fetches the signature as well and stores it for that exact database content, keyed by its digest. A signature requested later, after the upstream has moved on, is still the one for the database the client got, and a refresh that returns the same bytes keeps the cached signature, so it is served offline too. The database is fetched once more after its signature to catch a mirror that swapped both in between. A missing signature is remembered per database, so official repos answer 404 without going upstream.
• 5ce445f A database signature that belongs to an older database is no longer served. When the database was refreshed and the signature refetch fails, the proxy answers 404 or 502 instead of falling back to the cached signature, which pacman would reject. Paths with a decoded ? or # are refused, since they would fetch one file and cache it under another name. The README now says that pacman -Sy only works offline with cache_metadata enabled.
• d7a6a89 Abort blob writes when the download fails
• 6857fad Abort the response when a cached artifact fails its integrity check (#504)
• bf0c784 Add Chef Supermarket proxy support for Berkshelf and Policyfiles (#518)
• ea696a7 Add a hosted config block and a bearer token check for writes
• 599b2e1 Add a metadata_dates table for rewritten metadata dates
• b7b7a64 Add a size limit for upstream artifacts
• b664530 Add age-based retention for cached artifacts (#428)
• d643836 Add opt-in retention for generic artifacts (#513)
• 832a441 Add package routes for private Composer and npm registries (#426)
• 80fdd16 Add push, list and delete commands for hosted packages
• 9532c6a Add upstream.go_sumdbs_offline for networks where only the proxy has egress
• 513687c Answer uncached referrers tags with 404 when the upstream is down
• 111b271 Arch Linux pacman repositories are now proxied at /arch/{name}/, with geo.mirror.pkgbuild.com as the built-in archlinux repository and more mirrors configurable under upstream.arch. Databases and file lists go through the metadata cache byte for byte, and a database signature is refreshed together with its database so the two always match. Packages and package signatures go to the artifact cache and stay available offline. The path is passed through unchanged, so Arch Linux ARM's $arch/$repo layout works too. README, config example, configuration docs and the dashboard have pacman instructions.
• 094a962 At most 64 fetches keep running after their client hangs up; beyond that a fetch is tied to its client again, so nobody can start an unbounded number of large downloads and leave. A server test now checks that client requests actually get the detach marker.
• fdd0c81 Build in version-pinned paths for well-known download hosts
• cbb1978 Bump git-pkgs/integrity to v0.2.0
• fc312a0 Bump github.com/aws/aws-sdk-go-v2/config from 1.33.6 to 1.33.7
• b8d5cf4 Bump github.com/aws/aws-sdk-go-v2/service/ecr from 1.66.1 to 1.66.2
• ba61c45 Bump github.com/prometheus/client_golang from 1.24.1 to 1.25.0
• b884a5f Bump golang.org/x/sync from 0.23.0 to 0.24.0
• 5683349 Cache Conan 2 recipe and package archives (#431)
• 6051e20 Cache Maven checksums and signatures as artifacts
• 579ab79 Cache archive artifacts from Maven repositories (#425)
• d2a069f Cache configured version-pinned paths of generic upstreams (#424)
• 2e0f132 Check gofmt in CI and reformat (#528)
• 943009c Check that a data tile holds W records before caching it. Complete tiles are never fetched again, so a block page cached as one stayed for good.
• 9a93ce0 Check tile sizes before caching them
• dcf90cf Close the Composer download gaps left after #430 (#433)
• 854182f Concurrent clients could still end up with a database and a signature that don't belong together. Databases are now stored under their digest like signatures, so a cache row and its bytes always agree, also with several replicas on shared storage. Writes for one database are serialized, and the signature recovery shares the refresh with database requests, so a slow fetch can no longer store an older database over a newer one. When the cache write fails, the proxy serves the database the cache still describes. Unchanged databases are no longer rewritten, and a remembered missing signature is asked for again after the metadata TTL. The new tests force these interleavings with storage and upstream hooks.
• d56d436 Count cached metadata against max_size and evict it LRU (#507)
• 1584499 Cut memory use of npm metadata rewriting (#413)
• cff4b51 Date rewritten metadata by when its package started serving it
• 8903a8c Date rewritten metadata by when the rewrite was made
• 8bebc56 Delete cached artifacts through the API and the UI (#491)
• bf7d17d Derive cooldown metadata ETags from the rewrite's inputs
• c0b2a20 Document how pushed versions appear in package metadata
• 9eb88e4 Document upstream.go_sumdbs
• 5332f1f Don't count a tree fresh in the cache as the database answering. The probe never asked it, and during an outage that kept lookups from remembering the failure, so offline mode waited for the upstream timeout on every request until the tree expired.
• c1a597d Don't let the supported check hang, and don't take a block page for an answer
• 1d88be5 Enable storage retention for alpine
• 70ea921 Enable storage retention for cargo
• 9c75e59 Enable storage retention for composer
• ff4d5ca Enable storage retention for conan
• bffba28 Enable storage retention for conda
• 42b4303 Enable storage retention for cran
• 3297800 Enable storage retention for deb
• 20855ca Enable storage retention for gem
• c61cb51 Enable storage retention for golang (#442)
• f3505ad Enable storage retention for helm
• 743fbe5 Enable storage retention for hex
• 8f30c6d Enable storage retention for julia
• 90989eb Enable storage retention for maven (#444)
• 4b8678c Enable storage retention for npm (#440)
• dcfb9a8 Enable storage retention for nuget
• b40a3f0 Enable storage retention for oci (#443)
• 48edbb8 Enable storage retention for pub
• 18f410d Enable storage retention for pypi (#441)
• 42f459e Enable storage retention for rpm
• f8ca190 Enable storage retention for swift
• ea4c1ba Encode file paths when loading files in the source browser
• bdec446 Exit with status 1 when proxy mirror fails to mirror a package (#502)
• 3adae30 Explain APT signing keys and add a deb822 example (#429)
• 3c09523 Fetch the pinned commit for Composer branch downloads (#430)
• 680a2da Fix gem filenames whose platform ends in a number
• c3c1204 Fix hex tarballs whose version contains a hyphen
• 0916eab Fix leftover /api/packages references to the hosted endpoints
• 42df87e Forward data tiles without storing them. The go command never fetches them, and kept for good they would add up to the whole log; the record count check from before is no longer needed.
• 3bbb5aa Give the detached fetch test time on slow runners
• e9ecde7 Hold requests to an upstream that rate limits the proxy
• b131db3 Isolate the recent failure history in tests
• dc809b0 Join every Accept header on container manifest requests
• 2ed0327 Keep fetching an artifact after the client that started the fetch disconnects. pacman gives up after 10 seconds without data, and since the proxy only starts sending once the whole file is stored, a large package on a slow upstream was cancelled on every try and never got cached. Client requests now mark their context so the shared fetch runs on context.WithoutCancel, still bounded by the artifact client timeout. Mirror jobs don't set the marker, so cancelling a job still aborts its fetches.
• b2955ef Keep holds through late successes and away from the circuit breaker
• 85b674f Keep the stored metadata on an upstream 304 (#495)
• ba5e6e8 Leave Composer dev branch archives unverified
• eca224b Leave denied pushed versions out of package metadata
• a9cff7f Limit how many version diffs run at once
• 112fc59 List pushed Composer versions in package metadata
• c566949 List pushed npm versions in package metadata
• 496f272 List recent failures under each failure count on /ui/analytics
• 4de5419 Make response write and artifact download timeouts configurable (#531)
• 83e2fb4 Make the Gradle build cache read-only by default, so a client that can reach the proxy can no longer overwrite cache entries; CI jobs that upload now need gradle.build_cache.read_only: false (or PROXY_GRADLE_BUILD_CACHE_READ_ONLY=false). The environment variable is parsed strictly, so a value like "yes" keeps uploads off and logs a warning instead of opening them. Fixes #497. (#509)
• 055540a Manifests are now cached under a key that ignores the client's Accept header: a digest names one document, and a tag is shared by all clients that accept an index type, so Docker, containerd and Podman read the same entry and a pre-fetched image is found offline. Clients without index support keep their per-Accept entry, and the old keys are still written and read as fallback.
• fd4e9b7 Mark Helm cooldown as supported in README
• 87505c0 Mark pushed artifacts as local and keep them out of eviction
• 61eeaf5 Merge remote-tracking branch 'origin/main' into issue-411-mirror-images
• 02ae49d Merge remote-tracking branch 'origin/main' into issue-411-mirror-images
• b0570bd Mirroring a tag now checks it with a HEAD first and reuses the cached manifest when the digest is unchanged, falls back to the cached tag when the upstream body is cut off, and keeps a tag that a client pull moved while the walk ran. The platform filter is checked across the whole image, so a nested index without the wanted platform no longer fails it, and a 304 for a per-Accept fallback entry only refreshes that client's own keys instead of overwriting the shared entry.
• c2ab810 Move upstream.go_sumdbs_offline next to upstream.go in the configuration table.
• 8265c7d Nest the hosted commands under proxy hosted
• e0ab227 Never replace a pushed artifact with the upstream copy
• f3f35df Note that streamed artifacts over the size limit are cut off
• 544e1a5 Note that the container registry is pull-only
• 16ebeaa On Postgres, metadata fetched_at is now stored in UTC, so metadata_ttl no longer expires hours late when the proxy runs east of UTC. Entries written before this expire late one last time and are stored in UTC on their next refresh. (#435)
• cf463c1 Only cache real pub archives on download (#520)
• e6aadb2 Only claim checksum database support when the database is reachable
• 6a90fb4 Only send credentialed registry tokens to trusted registries
• 2a957b9 Prefer the root composer.json of a pushed archive
• 0c495fd Proxy the OCI referrers API in the container handler (#427)
• 5e66688 Push, list and delete npm and Composer archives through /api/hosted
• c7f8e2f README: say when the go command still goes to the database itself
• 8d96791 Re-mirror container images on a schedule (#508)
• e7449d1 Read the package manifest of pushed archives
• 3b262f1 Record the detail behind each failure counter
• c58cdae Refuse conda rules with more than a channel and a name
• ad7f299 Refuse package mirroring from the CLI while scanning is enabled (#493)
• 729138d Reject package requests with ? or # in the decoded path (#437)
• fa7ae81 Request npm packuments in one format for downloads too (#496)
• d4e89b0 Resolve Composer downloads from the cached metadata (#494)
• 449683a Rewrite Composer metadata in place and keep it minified (#418)
• 1559be6 Route containerd ns mirror requests to configured registries (#405)
• cf7eafb Sandbox upstream content on package routes
• f47dd2f Say that the 4 KiB limit on 404 text only applies with the metadata cache
• 4f680dd Send ETag and Last-Modified with rewritten npm and Composer metadata
• a97b690 Separate artifact and metadata cache metrics on /metrics and the analytics page (#463)
• e74e5ab Serve cached npm and Composer metadata gzip-compressed
• 832c800 Serve diff2html from the proxy instead of jsDelivr
• 69313c2 Serve hosted-only npm scopes and Composer vendors from pushed packages alone
• c0ac418 Serve pushed artifacts when streaming upstream artifacts
• 96241b9 Serve pushed npm tarballs before the cooldown check
• 1c6cfae Serve the Go checksum database from its own URL
• 054c45c Share the dates of rewritten metadata across replicas
• 3dae0fb Stop UpsertPackage from clearing packages.source
• aa0b543 Stop forwarding custom auth headers across redirects
• f5e7599 Stop interpolating archive paths into inline handlers
• 5d7855f Store mirrored NuGet ids lowercased
• 8b3b933 Stream archives when listing and reading browsed files (#408)
• cae9c08 Stream conda repodata through the cooldown filter (#527)
• 1b20307 Test the probe timeout, the block pages and a damaged cached tile
• 9f42425 Test upgrading cached metadata through the metadata_dates migration
• 67fd1ad The mirror now resolves a tag without caching it, mirrors the image it points to, and stores the tag only once that image is complete, so a failed or interrupted mirror keeps the previously cached tag instead of pointing it at an image with missing layers.
• 9d0175b Try configured generic_artifacts before built-in patterns
• 5b5a848 Update Go dependencies (#534)
• d58dd3d Update x/net and remove unused production helpers
• 03f0ebd Use shared archives browser for cached artifacts (#512)
• 6d771a8 Use shared cooldown package-pattern policy
• 8e04713 Use shared registries size limits and rate-limit gate (#516)
• 5dfd01e Use shared sbom parser for mirror sources (#510)
• bf74d98 Verify npm and Composer archives against their SHA-1 shasum
• 2d258dd Verify npm tarballs against dist.integrity
• e3cdf73 Wait for the clock to move on in the fresh-tree test, since on Windows the first probe's success and the start of the lookup can get the same time.
• 59f0203 When a signature has to be fetched on request because its fetch failed during the database refresh, it is now only served if it belongs to the database the proxy had cached at that point. If the upstream has published a new database in the meantime, the request fails with 502 instead of handing out a signature that cannot verify the database the client already has. The next sync gets the new pair.
• 5b9a5fc When the shared tag entry holds a representation the client does not accept, the lookup now goes on to the client's own per-Accept entry instead of giving up, so a Docker-only client still gets its cached manifest list after an OCI index was cached for the same tag.
• 03c8e68 Withhold Composer archives still in cooldown
• 73ed6bf build(deps): bump github.com/aws/aws-sdk-go-v2/config (#422)
• a691e99 build(deps): bump github.com/aws/aws-sdk-go-v2/service/ecr (#423)
• 52e8b5c build(deps): bump modernc.org/sqlite from 1.59.0 to 1.60.1 (#421)
• 6e334b4 npm: version lookup returns 502 (#419)
• 7b6c64a proxy mirror and the mirror API now pre-fetch container images from pkg:docker and pkg:oci PURLs: the mirror walks index, platform manifests (with their attestations) and blobs through the container handler, so a later pull is served from cache even when the upstream is down. A --platform filter (platforms in the API) limits multi-arch images, repository_url only resolves to configured registries, and names, tags and digests are validated before anything is fetched.
• cc18c88 proxy mirror now builds its upstream clients the same way as serve, so it follows OCI Bearer challenges, uses configured upstream credentials and the SSRF guard, and artifact downloads have no overall time limit. Before this the CLI used a plain client and Docker Hub answered every request with 401.