Skip to content

Commit

Permalink
Storage Driver: Add an IPFS driver
Browse files Browse the repository at this point in the history
This adds an IPFS storage driver which stores and makes the registry objects
available on IPFS. Said otherwise, it enables a swarm of docker registries to
share an underlying, content-addressed storage layer out of the box.

Running `docker push ipfs-registry1/my-container` can be followed by `docker
pull ipfs-registry2/my-container`. `ipfs-registry2` will transparently fetch
the necessary objects from ipfs-registry1 (or from any other providers in the
network).

The driver runs an embedded IPFS-Lite daemon and is therefore ready-to-use
without additional external requirements (the ipfs daemon does NOT need to run
alongside).

---

It implements the filesystem by storing the [paths -> ipfs_object_identifier]
mappings in a replicated database using go-ds-crdt. Thus, when a registry
instance commits an object to the storage layer, every other registry instance
in the cluster will become aware of the new path/cid map (as long as the
pubsub messages arrive) and will be able to fetch the objects from the Network
when needed. Any registry instance willing to participate in the same cluster
will need to set the same value for the "pubsubtopic" parameter, to which all
registries subscribe.

The IPFS peer used for adding/getting/announcing is embedded in the driver
using IPFS-Lite, as are the libp2p host, DHT, pubsub and blockstore/datastore
facilities. Therefore, it is not required to run an additional IPFS daemon nor
there is any dependency between the regular IPFS daemon and this.

Each registry generates a private key for the libp2p host during the first run
and is then persisted to the datastore, though it can always be manually
provided via parameters. Instances may include custom bootstrappers and an
optional private network pre-shared key. In this case, the registry instances
will form an isolated IPFS private network. By default, instances will
bootstrap to the public bootstrappers and content made available in the public
IPFS network. In this case we disallow using the default pubsubtopic to avoid
accidents.

For reference, until driver documentation is created, it can be configured
this way (all keys are optional):

```
ipfs:
  # all registry instances should share pubsubtopic and protectorkey
  pubsubtopic: "ipfs-registry"
  # protectorkey is a 32-byte hex-encoded key. Change it!
  # not setting the protectorkey will run the registry on the public IPFS network.
  protectorkey: "e373e1babfa619e1544c88e0fd9e9f987d8f46443245274d21c6027f1477f45d"
  # comma-separated list of listen multiaddresses
  listenaddrs: /ip4/0.0.0.0/tcp/4619
  loglevel: info
  datastorepath: /var/lib/registry
  # The comma-separated multiaddresses of different ipfs-registries.
  # Needed specially when using a protectorkey.
  bootstrappers: "/ip4/x.x.x.x/tcp/4619/p2p/12D3KooWBfXWVX7nNUA1Sm64qEdCxGWQMM41FNE8fUXymRQQ7EZ4"
  # Private key will be autogenerated if not set
  privatekey: "base64-encoded ipfs private key"
```

The driver will print INFO log messages on startup detailing the peer
multiaddresses and other information.

Signed-off-by: Hector Sanjuan <code@hector.link>

Signed-off-by: Hector Sanjuan <code@hector.link>
  • Loading branch information
hsanjuan committed Apr 20, 2019
1 parent 3226863 commit 4502049
Show file tree
Hide file tree
Showing 3 changed files with 816 additions and 0 deletions.
1 change: 1 addition & 0 deletions cmd/registry/main.go
Expand Up @@ -12,6 +12,7 @@ import (
_ "github.com/docker/distribution/registry/storage/driver/filesystem"
_ "github.com/docker/distribution/registry/storage/driver/gcs"
_ "github.com/docker/distribution/registry/storage/driver/inmemory"
_ "github.com/docker/distribution/registry/storage/driver/ipfs"
_ "github.com/docker/distribution/registry/storage/driver/middleware/alicdn"
_ "github.com/docker/distribution/registry/storage/driver/middleware/cloudfront"
_ "github.com/docker/distribution/registry/storage/driver/middleware/redirect"
Expand Down

0 comments on commit 4502049

Please sign in to comment.