Skip to content
Fast docker image distribution plugin for containerd, based on CRFS/stargz
Go Shell Python Dockerfile Other
Branch: master
Clone or download
This branch is 14 commits behind containerd:master.

Latest commit

Fetching latest commit…
Cannot retrieve the latest commit at this time.


Type Name Latest commit message Commit time
Failed to load latest commit information.

Stargz Snapshotter

Tests Status Benchmarking

Pulling image is one of the time-consuming steps in the container lifecycle. Research shows that time to take for pull operation accounts for 76% of container startup time[FAST '16]. Stargz Snapshotter is an implementation of snapshotter which aims to solve this problem leveraging stargz image format by CRFS. The following histogram is the benchmarking result for startup time of several containers measured on Github Actions, using Docker Hub as a registry.

The benchmarking result on 288c338

legacy shows the startup performance when we use containerd's default snapshotter (overlayfs) with images copied from without optimization. For this configuration, containerd pulls entire image contents and pull operation takes accordingly. When we use stargz snapshotter with stargz images we are seeing performance improvement on the pull operation because containerd can start the container before the entire image contents locally available and fetches each file on-demand. But at the same time, we see the performance drawback for run operation because each access to files takes extra time for fetching them from the registry. When we use the further optimized version of images(estargz) we can mitigate the performance drawback observed in stargz images. This is because stargz snapshotter prefetches and caches some files which will be most likely accessed during container workload. Stargz snapshotter waits for the first container creation until the prefetch completes so create sometimes takes longer than other types of image. But this wait only occurs just after the pull completion until the prefetch completion and it's shorter than waiting for downloading all files of all layers.

The above histogram is the benchmarking result on the commit 288c338. We are constantly measuring the performance of this snapshotter so you can get the latest one through the badge shown top of this doc. Please note that we sometimes see dispersion among the results because of the NW condition on the internet and/or the location of the instance in the Github Actions, etc. Our benchmarking method is based on HelloBench.

Stargz Snapshotter is a non-core sub-project of containerd.


You can test this snapshotter with the latest containerd. Though we still need patches on clients and we are working on, you can use a customized version of ctr command for a quick tasting. For an overview of the snapshotter, please check this doc.


  • Put this repo on your GOPATH(${GOPATH}/src/

Build and run the environment

$ cd ${GOPATH}/src/
$ docker-compose build --build-arg HTTP_PROXY=$HTTP_PROXY \
                       --build-arg HTTPS_PROXY=$HTTP_PROXY \
                       --build-arg http_proxy=$HTTP_PROXY \
                       --build-arg https_proxy=$HTTP_PROXY \
$ docker-compose up -d
$ docker exec -it containerd_demo /bin/bash
(inside container) # ./script/demo/

Prepare stargz-formatted image on a registry

For making and pushing a stargz image, you can use CRFS-official stargzify command or our ctr-remote command which has further optimization functionality. In this example, we use ctr-remote. You can also try our pre-converted images listed in this doc.

We optimize the image for speeding up the execution of ls command on bash.

# ctr-remote image optimize --plain-http --entrypoint='[ "/bin/bash", "-c" ]' --args='[ "ls" ]' \
             ubuntu:18.04 http://registry2:5000/ubuntu:18.04

The converted image is still docker-compatible so you can push/pull/run it with existing tools (e.g. docker).

Run the container with stargz snapshots

Layer downloads don't occur. So this "pull" operation ends soon.

# time ctr-remote images rpull --plain-http registry2:5000/ubuntu:18.04
fetching sha256:728332a6... application/vnd.docker.distribution.manifest.v2+json
fetching sha256:80026893... application/vnd.docker.container.image.v1+json

real	0m0.176s
user	0m0.025s
sys	0m0.005s
# ctr-remote run --rm -t --snapshotter=stargz registry2:5000/ubuntu:18.04 test /bin/bash
root@8dab301bd68d:/# ls
bin  boot  dev  etc  home  lib  lib64  media  mnt  opt  proc  root  run  sbin  srv  sys  tmp  usr  var


We support private repository authentication powered by go-containerregistry which supports ~/.docker/config.json-based credential management. You can authenticate yourself to private registries with normal operations (e.g. docker login command) using ~/.docker/config.json.

# docker login
(Enter username and password)
# ctr-remote image rpull --user <username>:<password><your-repository>/ubuntu:18.04

The --user option is just for containerd's side which doesn't recognize ~/.docker/config.json. We don't use credentials specified by this option but uses ~/.docker/config.json instead. If you have no right to access the repository with credentials stored in ~/.docker/config.json, this pull operation fallbacks to the normal one(i.e. overlayfs).

Project details

Stargz Snapshotter is a containerd non-core sub-project, licensed under the Apache 2.0 license. As a containerd non-core sub-project, you will find the:

information in our containerd/project repository.

You can’t perform that action at this time.