Repository navigation
·
7 commits
to main
since this release
Immutable
release. Only release title and notes can be modified.
- Serve Git repositories and LFS objects from a local cache, FreeBSD repositories at
/freebsd/{name}/, additional Maven repositories at/maven/{name}/, the Go vulnerability database at/go/vulndb, and npm attestations. - Host generic files next to npm and Composer packages, and mirror PyPI, conda, Composer, Helm, Homebrew, Julia, CRAN and Swift packages.
proxy mirror --latestmirrors only the newest version. - Apply cooldown to Go modules and Maven and show the cooldown configuration in the web UI. When publish times are missing or unreadable, return an error instead of unfiltered metadata.
- Stream artifacts to the client while a cache miss is being stored, and cache more metadata rewrites. SQLite reads use a separate read-only pool, and the Go memory limit is set from the cgroup limit.
- Instances sharing a Postgres database fetch each cache miss once and share rate-limit holds.
- Match denylist entries using each ecosystem's name normalization. Verify Helm charts, mirrored files and sha512 OCI digests. Send registry tokens only to the token service's own registry and limit where upstream credentials are sent.
- Build with Go 1.27.3 and update registries to v0.13.0, enrichment to v0.7.5, purl to v0.2.1 and golang.org/x/net to v0.62.0.
Before upgrading:
- The proxy exits at startup if a generic upstream with credentials in
upstream.authhas noupstream.generic_paths(#581).config.example.yamllists the GitHub API paths that mise requests. - With cooldown on, the proxy withholds Go and Maven versions inside the window (#564, #573). When publish times are missing or unreadable, gem, Hex, PyPI and npm return 502 or 503 and compressed conda repodata returns 404 (#585, #605).
- Pushing a hosted file that already exists returns 409 for npm and Composer too; delete it first (#537).
- The proxy returns 502 for an uncached npm tarball when its packument is unreadable, for example when it exceeds
metadata_max_size(#582). Previously it fetched the tarball unverified. - Package paths with dot segments or
;return 400, andPOST /api/mirrorrequiresContent-Type: application/json(#554, #574). - Mirror jobs now list versions from the configured upstream rather than the public registry (#548). The proxy sends registry tokens only to the token service's own registry (#576) and checks forward-proxy destinations against the private address rules (#577).
- Denylist entries now match each ecosystem's spellings of a name and may block versions that were served before (#555). After the upgrade, the proxy refetches some cached conda, Conan, CRAN, RPM, Debian, Maven, gem and Go entries one time (#555, #557, #578, #642).
Changelog
- 1979fb0 Accept Swift source repository PURLs in the denylist and retention rules (#641)
- eb5ff1a Accept only relative paths in a conda shard index's base URLs
- ce9ab47 Add --latest to mirror only the newest version (#631)
- 1887b12 Add extraVolumes, extraVolumeMounts and extraInitContainers to Helm chart
- a06b20f Address Docker guide review points
- fc29959 Answer 404 for packages the registry doesn't have
- 23f48c5 Answer HEAD on a cached artifact without opening it
- fd0d9a5 Answer PyPI download cooldown from stored upload times
- e97d4aa Answer npm dist-tag ls from the served packument
- 4d74ad6 Apply cooldown to Go modules
- 74f8eb3 Apply cooldown to Maven
- 77bd66a Bound Git requests by their body and response progress
- aa0460a Bound Hex and gem cooldown reads by the metadata size limit
- aa3adbf Bound the registry token and challenge caches
- d28c0be Build with Go 1.27.3 everywhere
- 1b641b6 Bump enrichment to v0.7.4
- 9374437 Bump git-pkgs/registries to v0.12.0
- 359ba32 Bump github.com/git-pkgs/archives from 0.9.0 to 0.10.0
- 96bd0f4 Bump github.com/git-pkgs/magic from 0.5.0 to 0.6.0
- ce60611 Bump github.com/klauspost/compress from 1.20.1 to 1.20.3
- c08c25f Bump goreleaser/goreleaser-action from 7.2.3 to 7.3.0
- b31ee70 Bump modernc.org/sqlite from 1.60.1 to 1.61.0
- 2a4ce12 Bump purl to v0.1.24 and drop redundant Swift lowercasing
- c3d0448 Bump registries for streamed npm packuments
- 42584c3 Bump registries to v0.11.1
- 910dadc Bump registries to v0.11.1
- d154757 Bump zizmorcore/zizmor-action from 0.6.4 to 0.7.1
- 605e27d Cache Git LFS objects
- 10fc27b Cache Go modules that differ only in case apart (#540)
- 3afbf24 Cache Helm and PyPI metadata rewrites
- 0169214 Cache conda packages per subdir
- f4cab2d Cache cooldown metadata rewrites for a minute
- 58e87c2 Cache cooldown rewrites for a 24th of the shortest cooldown
- 8ea75a3 Check cached Swift documents against their digest and resolve source PURLs
- 16ae3de Check mirrored files against their own SHA-256
- d9d3bd2 Check unhashed FreeBSD packages against the denylist
- 9500c5b Check upstream destinations sent through a forward proxy
- c0320c7 Clearing a record and queueing its file for deletion now happen in one transaction, so a failed queue write keeps the record instead of leaving a file nothing points at. Eviction drops the queue entry again once it has deleted the file itself.
- 3fcdb0b Close the SQLite test databases in the lock tests
- d02ad9b Coordinate instances sharing a Postgres database
- 4094b0f Create breaking monthly releases as drafts
- d4addfe Cut a patch release on the 1st of each month
- cdd3bea Cut the relay connection on any upstream copy error
- 9babab7 Describe archived CRAN versions as fetched unchecked
- f81274b Describe how Maven publish times are read
- 2928b5c Detect Helm's cooldown check in the handler coverage test
- e6c4a3e Document Maven cooldown
- 0ed360b Document how pull requests are reviewed and merged
- 4402075 Document serving cached downloads when publish times can't be read
- f0548d1 Document that the OCI handler is pull-only
- bcda24e Document timeouts for large LFS objects
- 9037d93 Document using proxy in Docker builds
- b6d7575 Don't fetch npm tarballs whose packument can't be read
- f77d299 Don't hide upstream 404 and 401/403 behind stale metadata
- 16bd0e5 Drop FillVersionPublishedAt until it has a caller
- 6c9505f Drop cached metadata that fails its integrity check
- db8a60e Drop the Homebrew sibling formulae filter
- cb566d5 Expire cooldown rewrites when a withheld version becomes available (#638)
- 8ed0edb Extract portable npm metadata transformations
- a787913 Fail closed in Go cooldown and key stored times by escaped path
- 09e41d0 Fail closed in gem, Hex and PyPI cooldown
- 842fa21 Fetch archived CRAN versions unchecked and drop the version dedup
- 84111c3 Fetch each cache miss once across instances
- 2ea8915 Filter gzipped gem compact indexes under cooldown
- 8dcd194 Fix npm search routing and abbreviated packuments under cooldown
- 3fd79cc Fuzz the PACKAGES.rds decoder
- 371cefa Harden the Git LFS cache
- d2c6587 Hash path-based metadata cache keys
- fb346ae Hold an unrecorded object for a client reading its stored copy
- d13b8be Host generic files and share one push path across hosted ecosystems
- 1fbee10 Ignore path parameters when checking for dot segments
- 391c5b7 Index artifacts by filename and shorten Julia comments
- 9a5e5a2 Keep a slow client from holding back a streamed miss
- 64162c7 Keep detached lease waits alive and stop waiting on a failed fetch
- fd4f067 Keep legacy metadata whose blob matches its digest
- 10bc2f3 Keep rewritten metadata gzip-only and longer with cooldown
- b8a4cf7 Keep the long config test and dashboard list under the maintidx limit
- 0e25e46 Keep the stale fallback for 404 and 401/403, leave npm to #503
- 994a936 Keep the stale warning on filtered npm version metadata
- 743289c Let an interrupted mirror schedule run again
- f44ce1e List mirror versions from the configured upstream
- a2889d4 Log per-handler request lines at debug level
- ed77e40 Log rewritten npm tarball URLs once per document (#538)
- e0cb946 Make the git cache's RefreshWait test independent of timing
- 28ce5f4 Make the slow-client storage failure test deterministic
- 20c8878 Match denylist entries the way each ecosystem spells names
- e1650e2 Match more denylist spellings
- c72e18f Merge main into feat/mirror-helm
- a7fa07f Merge main into feat/mirror-julia
- 2334a51 Merge main into feat/mirror-julia
- 58e2258 Merge main into feat/mirror-swift
- 0fc8ae7 Merge main into fix/mirror-upstream-versions
- 847f4f3 Merge main into fix/ssrf-forward-proxy
- f8ee4c1 Merge main into perf/cooldown-metadata-cache
- 14a2534 Merge main into perf/cooldown-metadata-cache
- 5953af9 Merge main. Main now records UpsertArtifact and its queue entry in one transaction itself, so that part of this branch is gone; ClearArtifactCache, eviction and the unrecorded refetch remain, and an unrecorded refetch counts as failed for instances waiting on its lease.
- 6bd0cea Merge remote-tracking branch 'origin/main' into feat/mirror-julia
- 4de0848 Merge remote-tracking branch 'origin/main' into feat/stream-cache-miss
- cc4a9c4 Merge remote-tracking branch 'origin/main' into fix/mirror-upstream-versions
- c8be267 Merge remote-tracking branch 'origin/main' into mirror-swift
- f5251d3 Mirror CRAN source packages
- ae9cf36 Mirror Composer releases
- 17e4fc7 Mirror Helm charts
- 2a8ceb9 Mirror Homebrew bottles
- 3f83d9c Mirror Julia packages
- 607a55e Mirror PyPI packages
- 673a581 Mirror Swift registry releases
- 749934e Mirror conda packages
- baeccd4 Mirror gems and composer packages under the handler names
- e92afe9 Mirror packages from the configured upstreams
- 775b92f Mount an emptyDir at /tmp in the Helm chart (#590)
- e070618 Move abbreviated packument detection into internal/metadata/npm
- f4423ab Move the PURL's mirror name and qualifiers out of Enumerate
- 48916bc Never let a cache fill replace a pushed artifact
- faecb56 Polish Maven cooldown after review
- a29f9fc Preserve Docker registries and SBOM versions when mirroring
- 3eadb3e Preserve Swift and Go compatibility when upgrading PURL (#642)
- 91b37ec Proxy FreeBSD pkg repositories at /freebsd/{name}/
- 25f62da Queue only a row's own legacy metadata blob
- 3bf0cda Read SQLite through a separate read-only pool
- 78f614e Read a Maven version's time from its .pom download
- b09ac87 Read gem, Hex and conda cooldown metadata through the metadata cache
- a5868f1 Read metadata into a buffer of its known size
- 5c0120f Record pushes atomically and address review findings
- 7eed8bc Refer to govulncheck's -db flag in comments
- 5b1c53b Refresh the Julia name map only from the advertised registry
- d64bfc6 Refuse '?' and '#' in paths in more handlers
- fcac1d5 Refuse '?' and '#' through rejectQueryChars in APK, Debian and RPM
- 0e6ec29 Refuse dot segments in package paths and upstream auth matching
- 9a78895 Refuse framing of the web UI
- f27fe11 Refuse overlong gem filenames
- 0832c61 Refuse pushes of a version that holds a pushed file under another name
- 4b3bd6f Refuse unfiltered conda repodata while cooldown is on
- 25a5414 Release FreeBSD binaries
- 7e2bf15 Release a new minor version when breaking changes were merged
- 257d159 Release an unrecorded object's hold only once per reader
- 003c760 Require a JSON content type to start mirror jobs
- 112e9c8 Require generic_paths for generic upstreams with credentials
- fe4be3c Reuse batch download URLs and stored copies for LFS objects
- 0b76ca2 Revalidate stale OCI tags with HEAD and share the request
- f9d4da8 Route every Maven artifact file through the download path
- b907ef0 Run the Postgres tests in CI
- 2fed890 Send artifacts to the client while a cache miss is stored
- 351fe7c Send registry tokens only to the token service's own registry
- 626a8e8 Serve Git repositories from a local cache (#492)
- cc6069d Serve additional Maven repositories at /maven/{name}/ (#539)
- be50be2 Serve cached Composer releases without a metadata lookup
- 224bce8 Serve cached downloads when cooldown can't read publish times
- 96cc1ee Serve conda sharded repodata through cooldown and the cache
- 2b62037 Serve immutable Go and RPM metadata without revalidation
- 50e65a3 Serve npm attestations through the proxy
- 29ecb20 Serve the Go vulnerability database at /go/vulndb
- 6d39461 Set the Go memory limit from the cgroup limit
- 3935ccd Share upstream rate-limit holds across instances
- 2ea13b0 Show cooldown only for ecosystems that apply it
- b79cd4d Show the cooldown configuration in the web UI
- 7a3b79b Skip the monthly release while the last one is a draft
- 674528a Skip the pushed artifact lookup when hosting is disabled
- 4d0d4a2 Spill large Git request bodies into the cache's temporary directory
- 562faac Split Maven metadata fetch from its response
- 0acc414 Stop mirror jobs before a job store test cleans up
- 13cb62c Store metadata under a hash of its cache key
- f240d30 Stream metadata cache hits for ProxyCached
- 5f431a3 Test concurrent pushes of one file at the database level
- 844e2ae Test that Composer metadata doesn't hide upstream 404 and 401/403
- 6347735 Treat file::memory: URIs as in-memory and test more of the read pool
- 37ecc3d Trim BottleImage doc comment and rejoin the brew docs list
- e0e2129 Trim npm attestations doc comments
- 15789d4 Update golang.org/x/net to v0.62.0 (#647)
- 94668c9 Use git-pkgs/enrichment for the enrichment API
- de308e4 Use purl's ComparisonForm for denylist and NuGet retention keys
- 0b2fa83 Use registries' forward proxy check
- 486768b Use smaller upload buffers for S3 and Azure
- ce39583 Verify Helm charts before they are cached
- 77bec54 Verify cache hits against the content hash only
- ff10ab3 Verify sha512 OCI digests and refuse ones that can't be checked
- 35df59f Wait for background jobs before closing the database at shutdown
- 24acee2 When a refetched file can be neither recorded nor queued for deletion, it is still served to everyone sharing the fetch and deleted once the last of them has closed it, so it no longer stays in storage with nothing pointing at it. A record that committed despite the reported error keeps its file.