Pre-release
Pre-release

@dmcgowan dmcgowan released this Dec 1, 2017 · 1273 commits to master since this release

Assets 3

Welcome to the release of containerd v1.0.0-rc.0!

This is a pre-release of containerd

The final 1.0 release stretch has been reached after 7 alphas and 4 betas full
of stability fixes, hardening, and API improvements. This release candidate
continues that pattern in including stability fixes, usability improvements and
performance optimizations.

As part of the preparation for supporting the release of Containerd 1.0, the
governance model has been updated. The same as in previous versions, the day to
day management of the project is taken care of by the maintainers. The roles
of chief maintainer and BDFL have been removed and a technical steering
committee has been added as an escalation point when consensus cannot be reached
among maintainers. See https://github.com/moby/tsc.

Please try out the release binaries and report any issues at
https://github.com/containerd/containerd/issues.

Shim Memory Improvements

The shim memory usage has been identified as a key component to optimize. The
shim gets run for every container making memory usage multiplied by the
number of containers on a machine, requiring lower memory usage in order to
increase density.

In profiling the memory usage of the shim, it was discovered that the HTTP2
and GRPC stacks were accounting for a majority of the usage. Since the RPC
requests to the shim are fairly simple and there is no need for handling many
clients, a simpler protocol is now used with much less overhead. In order to
avoid changing the client or server code, ttrpc is used to generate code
similar to GRPC using the same protocol definitions, but using the lighter
weight protocol. This had the effect of drastically reducing the memory
usage of the shim.

Memory improvements over beta 2...

  RSS CMD
 3644 containerd-shim -namespace default -workdir /var/lib/containerd/io.containerd.runtime.v1.linux/default/test10 -address /run/containerd/containerd.sock -containerd-binary /usr/local/bin/containerd -debug
15320 docker-containerd-shim --namespace moby --workdir /var/lib/docker/containerd/daemon/io.containerd.runtime.v1.linux/moby/ffa219303c1419b179393a1123f176bb4268337432b0ca7945bc7ac66db73cad --address /var/run/docker/containerd/docker-containerd.sock --runtime-root /var/run/docker/

See ttrpc repository for implementation.

Garbage Collector Improvements

Garbage collection is now scheduled by a background process which watches
changes and tracks the amount of time garbage collection is taking. It is able
to use that information to schedule garbage collection based on configuration
options. The default configuration option aims to limit the amount of time the
metadata store is locked for garbage collection to 20ms of every second.

Additionally image removal now has an option to synchronously cleanup all
referenced resources (such as snapshots). This option immediately schedules a
garbage collection and waits for it to complete before returning to the client.
Since the time to remove a snapshot depends on the size of the snapshot, the
time to synchronously remove an image cannot be consistent. For faster and
more consistent removals, images should be deleted asynchronously whenever
possible. Asynchronous remains the default.

Contributors

  • Akihiro Suda
  • Daniel Nephin
  • Daniel, Dao Quang Minh
  • Darren Stahl
  • Derek McGowan
  • Jess Valarezo
  • Kenfe-Mickaël Laventure
  • Kir Kolyshkin
  • Lantao Liu
  • Michael Crosby
  • Phil Estes
  • Stephen J Day
  • yanxuean

Changes

  • 08f1793 Merge pull request #1848 from dmcgowan/update-version-rc.0
  • 30ac46e Update version to v1.0.0-rc.0
  • 4bcd272 Merge pull request #1845 from dmcgowan/fix-rc.0-notes
  • 6952a61 Update previous version to beta not current rc
  • 7e4be46 Merge pull request #1834 from dmcgowan/v1.0.0-rc.0-notes
  • 07d4154 Merge pull request #1844 from Random-Liu/close-fifos-on-failure
  • a03fdab Close FIFOs on failure.
  • e64d179 Add release notes for v1.0.0-rc.0
  • 763d29f Merge pull request #1842 from dmcgowan/fix-snapshot-close-twice
  • 25b5592 Fix close twice test to retain snapshot
  • 2657e49 Merge pull request #1835 from stevvooe/we-use-ttrpc
  • 6c416fa shim: we use ttrpc in the shim now
  • c357934 Merge pull request #1841 from crosbymichael/parse-uid
  • a4a4c90 Use strconv.Atoi for uid/gid parsing
  • 8fbdd5c Merge pull request #1840 from kolyshkin/mkdir-all
  • acc6f4e MkdirAll: fix usage
  • 8114d2f Merge pull request #1838 from estesp/mention-kernel-vers
  • e974099 Merge pull request #1839 from crosbymichael/test-long-exec
  • a87062e Add runtime requirements section to README
  • 6bff39c Merge pull request #1837 from crosbymichael/bump-cgroups
  • 68bec1e Add test for large exec payload
  • c4b1114 Merge pull request #1831 from stevvooe/update-ttrpc
  • 4349df4 Update cgroups to 29da22c6171a4316169f9205ab6c49f5
  • 393cf8e vendor: update ttrpc with latest changes
  • 96ca973 Merge pull request #1797 from jessvalarezo/rename-packages
  • 04dab2a Merge pull request #1832 from crosbymichael/stress-kill
  • 1dd6f33 rename snapshot->snapshots proto pkg and service
  • 9885edf rename snapshot->snapshots pkg
  • 4363994 Fix stress test tool exec kill
  • 61c8fe2 ctr: snapshot->snapshots cmd, add aliases
  • 59bd196 Merge pull request #1830 from stevvooe/define-fieldpath-support-envelope
  • 16d3aeb Merge pull request #1829 from AkihiroSuda/ctr-envvar-snapshotter
  • d7c950e events: define fieldpath implement for envelope
  • a56e742 Merge pull request #1812 from darrenstahlmsft/revendor
  • d298f9a Merge pull request #1824 from kolyshkin/static-build
  • 3da99ed Merge pull request #1825 from mlaventure/use-defaults-pkg-everywhere
  • 7f95b9f ctr: add EnvVar CONTAINERD_SNAPSHOTTER for --snapshotter
  • 0d682e2 plugins: don't compile for static build
  • 6bf779c Use defaults pkg for all platforms
  • bc9cb25 Update tar to use PAXRecords instead of Xattrs
  • 16d0087 Add test for prefix header issue
  • 444e422 Revendor hcsshim and go-tar
  • 90a6b79 Merge pull request #1810 from dnephin/add-staticcheck-linter
  • a3955e1 Merge pull request #1822 from crosbymichael/subreaper
  • 6e9f24b Change default subreaper setting
  • 967caee Merge pull request #1821 from dnephin/remove-use-of-deprecated-grpc-funcs
  • dab3ba2 Replace calls to deprecated grpc functions
  • 118c0a2 Merge pull request #1820 from crosbymichael/shim-lockup
  • 74b3cb3 Fix exit event handling in shim
  • 723f37d Add exec support to stress test tool
  • a05e5fd restore deferred cleanup in rootfs.init
  • ee04cfa Add staticcheck linter
  • 2556c59 Merge pull request #1767 from stevvooe/ttrpc-shim
  • 9e04cff Merge pull request #1814 from stevvooe/panic-client-fix-oci-spec
  • ca39f76 Merge pull request #1798 from stevvooe/shim-gomaxprocs
  • d54961c vendor: add ttrpc as vendored dependency
  • 372cdfa Merge pull request #1638 from dmcgowan/gc-policy
  • f011bec Merge pull request #1800 from yanxuean/image-store-redandunt
  • 0d957b1 oci: allocate process if not already set
  • 3b670eb remove redundant check for imagestore create
  • 5764bf1 cmd/containerd-shim: set GOMAXPROCS to 2
  • abeb262 Merge pull request #1796 from dnephin/move-oci-pkg
  • f6e877e Proposed fix for image content store
  • a21a19a fix windows oci package
  • cdf62f6 Fix usage of oci in other packages.
  • 081f8c7 oci package passing tests
  • 70e0c84 Merge pull request #1809 from Random-Liu/fix-build
  • 33c95bb Fix build.
  • a7ade66 Merge pull request #1799 from yanxuean/provide-withLease
  • 9bf0a15 Merge pull request #1808 from crosbymichael/stress-race
  • ff6c12b Merge pull request #1803 from dnephin/warn-on-mem-not-supported
  • 1cb0e81 Fix race in stress test tool
  • 12e7c76 Merge pull request #1802 from yanxuean/unpack-lease
  • 153fcfa Merge pull request #1806 from dnephin/fix-ctr-events
  • 0103d14 Fix ctr events
  • 8fe12ad Warn if OOM monitoring is not available
  • 545f247 prevent snapshot from gc when unpack image
  • 4af5f65 Merge pull request #1793 from dmcgowan/gc-add-container-test
  • 908df29 Merge pull request #1792 from dmcgowan/schema1-resumeable
  • d56b49d provide client.WithLease to user
  • e8f52c3 linux/shim: reduce memory overhead by using ttrpc
  • 8e09b56 Merge pull request #1795 from crosbymichael/cgroup-lock
  • e830112 Add lock around task cgroup
  • 1a6e6a9 Merge pull request #1794 from yanxuean/some-typo
  • 8130779 some typo
  • 92d40b9 Add test for container as gc root
  • bae4782 Document defaults
  • b674c64 Support unknown size from schema 1 manifests
  • 2b1c915 Merge pull request #1787 from AkihiroSuda/diff-link
  • 633f4a6 Merge pull request #1751 from dnephin/add-unused-linters
  • f74862a Add structcheck, unused, and varcheck linters.
  • 96e2f30 Merge pull request #1759 from dnephin/single-integration-target
  • a3aeb39 fs/diff: support symlink to abspath with second-precision mtime
  • 374f04d Update gc policy configuration to use time duration
  • 3f1a61f Add synchronous image delete
  • 72fb8f8 Add gc labels to content tests
  • 56b6a5d Add root labels to snapshot test suite
  • 00596f4 Add gc policy plugin
  • 1bd39d3 Merge pull request #1788 from crosbymichael/plugin
  • 23dfe54 Add plugin stability guarantees to release.md
  • 9649a42 Merge pull request #1784 from stevvooe/remove-sync-write
  • 95cbe81 Merge pull request #1783 from crosbymichael/static-shim
  • 5113299 content/local: don't open file with sync
  • 67232e9 Merge pull request #1779 from AkihiroSuda/resolve-util-bin
  • 434f0e6 Compile shim without cgo for size and mem usage
  • 7ef4aa5 shim: support non-default binary name
  • 2fad782 Merge pull request #1782 from dnephin/move-io-pkg
  • 298dabc Move io.go into cio package
  • bc063f2 Merge pull request #1778 from stevvooe/weak-imports-on-plugins
  • 2edc475 Merge pull request #1776 from stevvooe/decouple-events-grpc
  • a496871 Merge pull request #1777 from jessvalarezo/migrate-client-code
  • 77cb722 Merge pull request #1775 from dmcgowan/fix-push-different-tags
  • 50532f2 api: use weak imports for plugin protobufs
  • 9cb91f8 Merge pull request #1765 from dmcgowan/rm-lease-resources
  • 2d05c4a events: decouple events package from grpc
  • 8c5dede client: move content store client to containerd package
  • aefaeea client: move diff client to containerd package
  • 8d23e5e client: move image store client to containerd package
  • 6cd9962 client: move namespaces client to containerd package
  • 56a35d5 Update docker pusher check tag
  • 5c6e789 Merge pull request #1769 from stevvooe/split-events-types
  • 5b64f70 client: move snapshot client to containerd package
  • 09b5ca1 api/events: split event types from events service
  • fc149f0 Merge pull request #1770 from crosbymichael/resume
  • 4192f99 Only compare times on non-windows
  • 3e5e2ec content/local: consistent handling of data and locks
  • f4fdb94 vendor: include require package from testify
  • 682151b remotes/docker: implement seekable http requests
  • a9308e1 content/local: ensure that resumption is properly supported
  • 368dc17 testutil: add Size to DumpDir output
  • 4df7075 Remove integration-parallel target
  • 05ec01e Fix .gitignore for coverage files
  • 7d64713 Merge pull request #1768 from stevvooe/consolidate-empty
  • c5022ad protobuf: use the gogo/types package for empty
  • cc7e593 Merge pull request #1760 from dnephin/run-unit-tests-on-windows
  • f49b1b7 Merge pull request #1764 from crosbymichael/exit
  • 96d2c1e Merge pull request #1766 from dnephin/fix-error-message
  • 2e7f731 Normalize 'already exists' errors
  • f3a63f5 Add support for removing leases on resource removal
  • a72279e Skip some tests on windows where the implementation is missing
  • fe5bb4a Update platform tests to use the defaultOS
  • 666d946 Implement diff.compareSysStat on windows
  • 5025b53 Re-enable unit tests on appveyor
  • a522a6c Add publish subcommand for publishing events
  • a542cf7 Merge pull request #1761 from stevvooe/update-protoc
  • 19168f9 Merge pull request #1754 from AkihiroSuda/fix-snapshot-commit-spec
  • 571c002 Merge pull request #1755 from AkihiroSuda/fix-ctr-snapshot-args
  • 343d2d3 Merge pull request #1758 from dnephin/speed-up-unit-test
  • 7354a7f README, travis: update protobuf compiler version
  • 5eff92d ctr: fix args
  • 5e8218a snapshot: fix specification of Commit
  • d8cf30e Reduce number of blobs in TestWalkBlobs
  • c78c156 Merge pull request #1728 from AkihiroSuda/snapshot-closable
  • d15cee0 Merge pull request #1737 from estesp/progress-line-fixup
  • f435ec0 Merge pull request #1757 from dnephin/fix-coverage
  • b28e1ee Fix coverage report
  • 37ee054 Merge pull request #1739 from crosbymichael/shim-redux
  • 4feb6f2 snapshot: add Close()
  • f9933e9 Merge pull request #1742 from jessvalarezo/ctr-task-kill
  • 8376b50 Add grpc methods to errdefs
  • c81788b Remove errdefs and shimapi types from proc package
  • 9abde39 Fix lint issues on init process
  • 33fe5c1 Add debug package for memory reporting
  • 66a70e7 Refactor runtime initialization
  • 36e5548 Remove namepsaces and id imports from shim
  • a7343b0 Move events from shim into linux runtime
  • 1fe5a25 Move Exec creation to init process
  • 6e25898 Move shim process code to package
  • 92ca22c Merge pull request #1735 from estesp/image-rm-feedback
  • a74148b Merge pull request #1741 from dnephin/fix-task-path-conflict
  • 1966f9f ctr: update task kill to take exec-id
  • 680101f Merge pull request #1750 from mlaventure/delete-exec-pidfile
  • 231c960 containerd-shim: Delete exec pidfile on exit
  • c2ab556 Merge pull request #1748 from crosbymichael/bdfl
  • 503b536 Add TSC [Rule]
  • a906693 Remove chief stuff as well
  • e36c106 Remove BDFL sections
  • 732b156 Merge pull request #1745 from AkihiroSuda/remove-lcow-sys-firmware-todo
  • 79e3b97 spec_unix.go: remove unneeded TODO (moby/moby#35442)
  • 0876075 Fix output on ctr images rm to show actual feedback
  • 17093c2 Merge pull request #1718 from jessvalarezo/ctr-task-ps
  • 0961807 rename runcopts to runctypes
  • 807f4d2 expose exec-id on ctr task ps
  • 49af59b Validate that root and state paths are different.
  • b4a65de Merge pull request #1740 from stevvooe/update-grpc
  • 08d1e72 api,linux,windows: update generated protobufs
  • 3569786 vendor: update protobuf, grpc and gogo
  • b1527e0 Handle progress line length properly for formatting

Breaking Changes

  • #1797 The snapshots GRPC service was renamed from snapshot.

  • #1767 The shim protocol has changed to use ttrpc. The
    new protocol does not use http2 and is not compatible with previous shim versions using grpc.

Dependency Changes

Previous release can be found at v1.0.0-beta.3

  • 2e2c51242e8993c50445dab7c03c8e7febddd0cf -> go1.10 github.com/dmcgowan/go-tar
  • d2e1ade2d719b78fe5b061b4c18a9f7111b5bdc8 -> v0.5 github.com/gogo/protobuf
  • 5a0f697c9ed9d68fef0116532c6e05cfeae00e55 -> 1643683e1b54a9e88ad26d98f81400c8c9d9f4f9 github.com/golang/protobuf
  • f7dd103d3e4e696aa67152f6b4ddd1779a3455a9 -> 29da22c6171a4316169f9205ab6c49f59b5b852f github.com/containerd/cgroups
  • v0.6.3 -> v0.6.7 github.com/Microsoft/hcsshim
  • v1.3.0 -> v1.7.2 google.golang.org/grpc
  • 8c92e22ce0c492875ccaac3ab06143a77d8ed0c1 github.com/stevvooe/ttrpc