Skip to content
Fast docker image distribution plugin for containerd, based on CRFS/stargz
Go Shell Python Dockerfile Other
Branch: master
Clone or download
ktock Merge pull request #46 from ktock/benchmark
Enable to benchmark on CI
Latest commit aba96aa Feb 17, 2020
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
.github/workflows
cache Govern codes from CRFS project by the license of the project Feb 13, 2020
cmd Govern codes from CRFS project by the license of the project Feb 13, 2020
images Make snapshotter configurable and tighten the scope Jan 22, 2020
script
snapshot Make snapshotter configurable and tighten the scope Jan 22, 2020
stargz Govern codes from CRFS project by the license of the project Feb 13, 2020
task Fetch whole layer in background Feb 6, 2020
vendor Make snapshotter configurable and tighten the scope Jan 22, 2020
.gitignore Implement remote snapshotter as a proxy plugin Dec 12, 2019
.golangci.yml Implement remote snapshotter as a proxy plugin Dec 12, 2019
LICENSE Add LICENSE Oct 8, 2019
Makefile Enable to benchmark on CI Feb 17, 2020
NOTICE.md Enable to benchmark on CI Feb 17, 2020
README.md Rename snapshotter plugin Jan 22, 2020
overview.md Make snapshotter configurable and tighten the scope Jan 22, 2020
vendor.conf Fetch whole layer in background Feb 6, 2020

README.md

Stargz Snapshotter

Tests Status

Pulling image is one of the major performance bottlenecks in container workload. Research shows that time for pulling accounts for 76% of container startup time[FAST '16]. Remote snapshotter is a solution discussed in containerd community. Stargz Snapshotter is an implementation of the remote snapshotter which aims to standard-compatible remote snapshots leveraging stargz image format by CRFS.

Related discussion of the snapshotter in containerd community:

By using this snapshotter, images(even if they are huge) can be pulled in lightning speed because this skips pulling layers but fetches the contents on demand at runtime.

# time ctr-remote images rpull --plain-http registry2:5000/fedora:30 > /dev/null 
real	0m0.447s
user	0m0.081s
sys	0m0.019s
# time ctr-remote images rpull --plain-http registry2:5000/python:3.7 > /dev/null 
real	0m1.041s
user	0m0.073s
sys	0m0.028s
# time ctr-remote images rpull --plain-http registry2:5000/jenkins:2.60.3 > /dev/null 
real	0m1.231s
user	0m0.112s
sys	0m0.008s

Demo

You can test this snapshotter with the latest containerd. Though we still need patches on clients and we are working on, you can use a customized version of ctr command for a quick tasting. For an overview of the snapshotter, please check this doc.

NOTICE:

  • Put this repo on your GOPATH(${GOPATH}/src/github.com/ktock/stargz-snapshotter).

Build and run the environment

$ cd ${GOPATH}/src/github.com/ktock/stargz-snapshotter/script/demo
$ docker-compose build --build-arg HTTP_PROXY=$HTTP_PROXY \
                       --build-arg HTTPS_PROXY=$HTTP_PROXY \
                       --build-arg http_proxy=$HTTP_PROXY \
                       --build-arg https_proxy=$HTTP_PROXY \
                       containerd_demo
$ docker-compose up -d
$ docker exec -it containerd_demo /bin/bash
(inside container) # ./script/demo/run.sh

Prepare stargz-formatted image on a registry

To make and push a stargz image, you can use CRFS-official stargzify or our ctr-remote which has additional optimization functionality. In this example, we use ctr-remote to convert the image into stargz-formatted one as well as optimize the image for your workload. We optimize the image aming to speed up the execution of ls command on bash.

# ctr-remote image optimize --plain-http --entrypoint='[ "/bin/bash", "-c" ]' --args='[ "ls" ]' \
             ubuntu:18.04 http://registry2:5000/ubuntu:18.04

The converted image is still compatible with a normal docker image so you can still pull and run it with normal tools(e.g. docker).

Run the container with stargz snapshots

Layer downloads don't occur. So this "pull" operation ends soon.

# time ctr-remote images rpull --plain-http registry2:5000/ubuntu:18.04
fetching sha256:728332a6... application/vnd.docker.distribution.manifest.v2+json
fetching sha256:80026893... application/vnd.docker.container.image.v1+json

real	0m0.176s
user	0m0.025s
sys	0m0.005s
# ctr-remote run --rm -t --snapshotter=stargz registry2:5000/ubuntu:18.04 test /bin/bash
root@8dab301bd68d:/# ls
bin  boot  dev  etc  home  lib  lib64  media  mnt  opt  proc  root  run  sbin  srv  sys  tmp  usr  var

Authentication

We support private repository authentication powerd by go-containerregistry which supports ~/.docker/config.json-based credential management. You can authenticate yourself with normal operations (e.g. docker login command) using ~/.docker/config.json.

In the example showed above, you can pull images from your private repository on the DockerHub:

# docker login registry-1.docker.io
(Enter username and password)
# ctr-remote image rpull --user <username>:<password> docker.io/<your-repository>/ubuntu:18.04

The --user option is just for containerd's side which doesn't recognize ~/.docker/config.json. We doesn't use credentials specified by this option but uses ~/.docker/config.json instead. If you have no right to access the repository with credentials stored in ~/.docker/config.json, this pull optration fallbacks to the normal one(i.e. overlayfs).

Make your remote snapshotter

It is easy for you to implement your remote snapshotter using our general snapshotter package without considering the protocol between that and containerd. You can configure the remote snapshotter with your FileSystem structure which you want to use as a backend filesystem. Our snapshotter command is a good example for the integration.

TODO

General issues:

  • Completing necessary patches on the containerd.
    • Implement the protocol on metadata snapshotter: #3793
    • Skip downloading remote snapshot layers: #3846, #3870, #3911
    • Add handlers for image information propagation
    • Deal with ErrUnavailable error and try re-pull layers

Snapshotter specific issues:

  • Resiliency:
    • Ensure all mounts are available on every Prepare() and report erros when unavailable.
    • Deal with runtime problems(NW disconnection, authn failure and so on).
  • Authn: Implement fundamental private repository authentication using ~/.docker/config.json.
  • Performance: READ performance improvement
  • Documentation: Add overview docs.
You can’t perform that action at this time.