-
Notifications
You must be signed in to change notification settings - Fork 0
Introduction
Silo is a self-hosted package registry for RPM, Alpine APK, npm, pacman (Arch Linux), and Debian (apt). Publishing happens over gRPC; dnf, apk, npm, pacman and apt consume the results as real repositories over plain HTTP. Packages live in S3-compatible object storage, everything else — the package index, tokens, users, and the audit log — lives in Postgres.
GitLab's native RPM registry needs a paid tier. Pulp is heavier than this needs. Nexus dropped real yum/rpm support from its open-source line. Silo is a small alternative that keeps no local state, so replicas are interchangeable and horizontal scaling is just replicaCount.
crates/
silo-proto generated tonic/prost code from proto/silo/v1
silo-pkg the format seam: rpm | apk | npm | pacman | deb parsing, layout, index rendering
silo-db Postgres: package index, tokens, users, audit log, advisory locks
silo-core config, object storage, publish orchestration, signing, OIDC
silo-server gRPC (publish/read/auth/admin) + HTTP (package-manager-facing)
silo-cli `silo` — publishing and administration; never touches S3 or keys
Adding a format means adding one impl Format and one enum variant. Nothing else in the codebase matches on which format it's handling.
| storage layout | index unit ("group") | |
|---|---|---|
| rpm | {repo}/{ch}/Packages/{file} |
the whole channel |
| apk | {repo}/{ch}/apk/{arch}/{file} |
one architecture |
| pacman | {repo}/{ch}/pacman/{arch}/{file} |
one architecture |
| npm | {repo}/{ch}/npm/{name}/-/{file} |
one package name |
| deb | {repo}/{ch}/pool/{file} |
the whole channel |
The index group is what makes those the same shape: a publish invalidates exactly one group, and a group is the unit that gets locked. Two apk architectures, two pacman architectures, or two npm packages publish concurrently without ever contending.
apk and pacman share one wrinkle: an architecture-independent package (noarch for apk, any for pacman) belongs in every architecture's index, because neither client ever fetches anywhere but its own. That group is one other groups read from, and publishing into it rewrites them all — the one case where a single publish touches more than one index.
deb has an architecture-independent package too (Architecture: all), folded into every concrete architecture's Packages file the same way — but deb's index group is the whole channel, not one architecture, because apt's Release file enumerates every architecture's Packages* at once. Rendering it from only one architecture's rows would mean reading sibling architectures' already-written index objects back out of storage, which would make the index no longer a pure function of the database. See Usage-Deb for what that costs an all-only repo.
Every index is a pure function of the database. Whatever a format's index needs beyond the common columns — APKINDEX records, an npm package.json, the dependencies and file lists primary.xml carries — is extracted once at publish and stored on the row, so regenerating an index never reads a package back out of object storage. A publish is a constant number of object-storage operations, not one per package already in the repo.
RPM repodata (repomd.xml, primary, filelists, other) is generated in-process: silo does not shell out to createrepo_c, and the server image carries no package tooling.
Publishes take a Postgres transaction-scoped advisory lock (pg_advisory_xact_lock) keyed on the index group. Without it, two concurrent publishes would each regenerate the index from their own view of the bucket, and the loser's package would silently vanish from the index that won.
Transaction-scoped rather than session-scoped is deliberate: a session lock leaks if the holder is killed mid-publish or if a pooled connection is returned without an unlock. A transaction lock is released by the same machinery that rolls the transaction back, including when the backend notices the client is gone.
Bytes are written to object storage before the row commits, so a client following a fresh index never 404s. The cost is that a crash between the two leaves orphaned bytes that no row references; they're unreachable and the next publish of the same file overwrites them.
Index regeneration reads rows, never a bucket listing, so a publish never has to fetch every package already in the repo just to learn what is there. silo list never touches object storage at all.
Web UI; retention and dedup policies; mirroring or upstream proxying; per-package ACLs finer than repo scope.