Pre-release

@stevvooe stevvooe released this Oct 12, 2017 · 1488 commits to master since this release

Assets 3

Welcome to the release of containerd v1.0.0-beta.2!

This is a pre-release of containerd

This release rounds out much of the remaining feature set for the 1.0 release
time frame. In addition to a large number of bugfixes and utility additions,
services and methods have been added to the GRPC API to meet production use
cases. The highlight is garbage collection, along with database migrations,
plugin introspection and rich PID listing.

Please try out the release binaries and report any issues at
https://github.com/containerd/containerd/issues.

Garbage Collection

Full garbage collection support for cleaning up content, snapshots and metadata
for containerd resources. The implementation is based on well-known concurrent
mark and sweep and the approach allows for extensibility in collection
policies.

The GC is triggered Container.Delete and Image.Delete. Note that images may
need to be re-pulled for proper support.

Introspection API

The new introspection API allows for querying the state of containerd. The
Plugins method lists the state and exports of all initialized plugins in use
in a containerd instance.

$ ctr plugins
TYPE                            ID             PLATFORM       STATUS
io.containerd.content.v1        content        -              ok
io.containerd.metadata.v1       bolt           -              ok
io.containerd.differ.v1         walking        linux/amd64    ok
io.containerd.grpc.v1           containers     -              ok
io.containerd.grpc.v1           content        -              ok
io.containerd.grpc.v1           diff           -              ok
io.containerd.grpc.v1           events         -              ok
io.containerd.grpc.v1           healthcheck    -              ok
io.containerd.grpc.v1           images         -              ok
io.containerd.grpc.v1           namespaces     -              ok
io.containerd.snapshotter.v1    btrfs          linux/amd64    error
io.containerd.snapshotter.v1    overlayfs      linux/amd64    ok
io.containerd.grpc.v1           snapshots      -              ok
io.containerd.monitor.v1        cgroups        linux/amd64    ok
io.containerd.runtime.v1        linux          linux/amd64    ok
io.containerd.grpc.v1           tasks          -              ok
io.containerd.grpc.v1           version        -              ok

Rich ListPIDs

Listing PIDs through the task service can now provide runtime specific metadata
through the API. This is particularly interesting on the windows platform. The
following is an example of a Windows container ps output:

$ ctr tasks ps <container-id>
PID       INFO
3716      {ImageName:smss.exe CreatedAt:2017-10-10T17:51:45.1552607Z KernelTime_100Ns:156250 MemoryCommitBytes:348160 MemoryWorkingSetPrivateBytes:217088 MemoryWorkingSetSharedBytes:946176 ProcessID:3716 UserTime_100Ns:0}
5404      {ImageName:csrss.exe CreatedAt:2017-10-10T17:51:45.1848844Z KernelTime_100Ns:0 MemoryCommitBytes:606208 MemoryWorkingSetPrivateBytes:339968 MemoryWorkingSetSharedBytes:1560576 ProcessID:5404 UserTime_100Ns:0}
3628      {ImageName:wininit.exe CreatedAt:2017-10-10T17:51:45.1974386Z KernelTime_100Ns:156250 MemoryCommitBytes:983040 MemoryWorkingSetPrivateBytes:610304 MemoryWorkingSetSharedBytes:3448832 ProcessID:3628 UserTime_100Ns:0}
8576      {ImageName:services.exe CreatedAt:2017-10-10T17:51:45.2091635Z KernelTime_100Ns:468750 MemoryCommitBytes:2048000 MemoryWorkingSetPrivateBytes:1437696 MemoryWorkingSetSharedBytes:4182016 ProcessID:8576 UserTime_100Ns:0}
7892      {ImageName:lsass.exe CreatedAt:2017-10-10T17:51:45.2177712Z KernelTime_100Ns:937500 MemoryCommitBytes:2666496 MemoryWorkingSetPrivateBytes:1982464 MemoryWorkingSetSharedBytes:7561216 ProcessID:7892 UserTime_100Ns:312500}
6384      {ImageName:svchost.exe CreatedAt:2017-10-10T17:51:45.3676107Z KernelTime_100Ns:468750 MemoryCommitBytes:1896448 MemoryWorkingSetPrivateBytes:1277952 MemoryWorkingSetSharedBytes:4730880 ProcessID:6384 UserTime_100Ns:156250}
4904      {ImageName:svchost.exe CreatedAt:2017-10-10T17:51:45.5431743Z KernelTime_100Ns:312500 MemoryCommitBytes:1781760 MemoryWorkingSetPrivateBytes:1331200 MemoryWorkingSetSharedBytes:4952064 ProcessID:4904 UserTime_100Ns:0}
2092      {ImageName:svchost.exe CreatedAt:2017-10-10T17:51:45.5838844Z KernelTime_100Ns:156250 MemoryCommitBytes:2486272 MemoryWorkingSetPrivateBytes:2117632 MemoryWorkingSetSharedBytes:5668864 ProcessID:2092 UserTime_100Ns:312500}
6576      {ImageName:svchost.exe CreatedAt:2017-10-10T17:51:45.727607Z KernelTime_100Ns:156250 MemoryCommitBytes:1818624 MemoryWorkingSetPrivateBytes:1417216 MemoryWorkingSetSharedBytes:6684672 ProcessID:6576 UserTime_100Ns:156250}
2412      {ImageName:svchost.exe CreatedAt:2017-10-10T17:51:46.0427889Z KernelTime_100Ns:468750 MemoryCommitBytes:5570560 MemoryWorkingSetPrivateBytes:3915776 MemoryWorkingSetSharedBytes:5963776 ProcessID:2412 UserTime_100Ns:0}
5472      {ImageName:svchost.exe CreatedAt:2017-10-10T17:51:46.0827893Z KernelTime_100Ns:312500 MemoryCommitBytes:2625536 MemoryWorkingSetPrivateBytes:1908736 MemoryWorkingSetSharedBytes:7532544 ProcessID:5472 UserTime_100Ns:312500}
8756      {ImageName:svchost.exe CreatedAt:2017-10-10T17:51:46.1108672Z KernelTime_100Ns:312500 MemoryCommitBytes:1843200 MemoryWorkingSetPrivateBytes:1466368 MemoryWorkingSetSharedBytes:4612096 ProcessID:8756 UserTime_100Ns:0}
1508      {ImageName:svchost.exe CreatedAt:2017-10-10T17:51:46.1522136Z KernelTime_100Ns:2812500 MemoryCommitBytes:4816896 MemoryWorkingSetPrivateBytes:3600384 MemoryWorkingSetSharedBytes:9281536 ProcessID:1508 UserTime_100Ns:468750}
4612      {ImageName:svchost.exe CreatedAt:2017-10-10T17:51:46.2212148Z KernelTime_100Ns:937500 MemoryCommitBytes:3063808 MemoryWorkingSetPrivateBytes:2265088 MemoryWorkingSetSharedBytes:8667136 ProcessID:4612 UserTime_100Ns:156250}
5936      {ImageName:CExecSvc.exe CreatedAt:2017-10-10T17:51:46.2224031Z KernelTime_100Ns:0 MemoryCommitBytes:983040 MemoryWorkingSetPrivateBytes:737280 MemoryWorkingSetSharedBytes:3723264 ProcessID:5936 UserTime_100Ns:0}
4416      {ImageName:cmd.exe CreatedAt:2017-10-10T17:51:46.5943846Z KernelTime_100Ns:0 MemoryCommitBytes:1564672 MemoryWorkingSetPrivateBytes:356352 MemoryWorkingSetSharedBytes:2174976 ProcessID:4416 UserTime_100Ns:0}
8700      {ImageName:powershell.exe CreatedAt:2017-10-10T17:51:46.6120645Z KernelTime_100Ns:2343750 MemoryCommitBytes:24522752 MemoryWorkingSetPrivateBytes:19853312 MemoryWorkingSetSharedBytes:27156480 ProcessID:8700 UserTime_100Ns:10156250}

Migrations

A lightweight migration framework is now part of containerd. This allows us to
safely evolve the schema between releases with the groundwork laid early.

As part of this release, a migration will be run to add back references in
support of garbage collection.

Contributors

  • Ace-Tang
  • Akihiro Suda
  • Akim Demaille
  • Allen Sun
  • Daniel, Dao Quang Minh
  • Derek McGowan
  • Jess Valarezo
  • Kenfe-Mickaël Laventure
  • Lantao Liu
  • Mathieu Pasquet
  • Michael Crosby
  • Phil Estes
  • Stephen Day
  • Tobias Klauser
  • Yanqiang Miao

Changes

  • a543c93 Merge pull request #1630 from stevvooe/beta2
  • 683ed97 release: prepare 1.0.0-beta.2
  • 1fe31c7 Merge pull request #1628 from crosbymichael/release-tool
  • 769d9e1 add release tool to generate releases
  • 587f252 Merge pull request #1563 from dmcgowan/gc-alpha
  • 3cc7591 Merge pull request #1627 from jessvalarezo/task-remove-cast
  • 1e69de1 Remove ProcessInfo cast
  • b5152f2 Merge pull request #1626 from jessvalarezo/windows-changes
  • 830e0ea Change hcsshimopts to hcsshimtypes package
  • ffb03c4 Add checkpoint and userns gc labels
  • de7b281 Update services to call garbage collection
  • 17471d5 Metadata garbage collection
  • 7884707 Add reference labels to snapshots and content
  • 551579b Merge pull request #1621 from dmcgowan/refactor-diff
  • 64a96aa Abort newly created references on error
  • 69e7e7f Add error wrapping for client checkpoint errors
  • d9db1d1 Refactor differ into separate package
  • 60960e1 Merge pull request #1597 from stevvooe/introspection-service
  • 0e72ce5 services/introspection: implement plugin reporting
  • 4981464 plugin: allow declaring a plugin depends on all others
  • 8508e82 plugin: refactor plugin system to support initialization reporting
  • fe52d93 errdefs: use status package for mapping errors
  • d8bd6b7 plugin: allow querying plugin graph to be re-entrant
  • 363d692 api/services: define the introspection API
  • 12c79cc Merge pull request #1604 from jessvalarezo/windows-listpids
  • 061c719 ListPids returns process ID and other info
  • 96725a5 Merge pull request #1620 from mlaventure/runc-io-id
  • c807ba8 Allow setting the uid & gid of the io pipes
  • 1ea8ac4 Merge pull request #1617 from crosbymichael/2cgroup
  • d7864eb Use namespace in default cgroup path
  • 72bb45a Merge pull request #1603 from mlaventure/fix-windows-flaky-tests
  • cfa8756 windows: Create init process with task
  • ad52664 windows: Fix a few races
  • b2e3482 Merge pull request #1584 from miaoyq/fix-mount-lookup-err
  • d7c4611 Ensure getting the correct mountinfo corresponds to path
  • a8426ed Merge pull request #1609 from crosbymichael/attach-docs
  • 6c3f393 Merge pull request #1611 from stevvooe/address-cgroup-merge-conflicty
  • 77e5f65 metrics/cgroups: handle error on call to cgroup
  • bbb01ee Add docs around multiple attach
  • 987fcd1 Merge pull request #1598 from Random-Liu/fix-load-task
  • 67bbfc5 Merge pull request #1605 from dmcgowan/metadata-db-documentation
  • e4defbc Add documentation for metadata database
  • 6f351f0 Merge pull request #1586 from mathieui/postmortem-stats
  • d92f6ee Allow blocking and non-blocking metrics collection
  • ed519bb Collect cgroup stats one last time before exit
  • 8558b98 Merge pull request #1582 from dmcgowan/metadata-db-object
  • 7f657ce Add database migrations
  • 28ca8f0 Fix task load.
  • 7c4bca5 Merge pull request #1600 from AkihiroSuda/defaultspec
  • d7b0e52 spec: set MaskedPaths and ReadOnlyPaths by default
  • 72a3a01 Merge pull request #1595 from crosbymichael/container-lock
  • 28882fd Merge pull request #1589 from mlaventure/shim-no-newns
  • fa9e9bd Fetch current container info before operations
  • 8d892a6 Update metadata plugin initialization
  • 2ab70f2 Add content and snapshot store references
  • 447a0a9 Add children bucket to back reference snapshots
  • 56c1b79 Create metadata db object
  • acba0f5 Merge pull request #1594 from akimd/akim/clean
  • ec43dc2 Merge pull request #1592 from AkihiroSuda/workdir2
  • 26d4c2c Add an option to prevent putting the shim in a new mount namespace
  • 0ae9d8f Merge pull request #1567 from stevvooe/image-check-function
  • 8ceeeed style: move definition for consistency
  • b26e2e7 ctr run: add --cwd
  • f6d8892 Merge pull request #1583 from stevvooe/image-storage-validation
  • 698b6d1 metadata: validation and testing of image store
  • c555df5 images: support checking status of image content
  • 7c9b0ea Merge pull request #1590 from crosbymichael/container-update
  • 9e85035 Add container update method
  • 1c7a042 Merge pull request #1566 from stevvooe/split-children
  • 6b9aafd Merge pull request #1587 from Ace-Tang/remove_blank_line
  • 2231de3 Remove a blank line
  • 8ded4fe Merge pull request #1581 from crosbymichael/lint-2
  • 451421b Comment more packages to pass go lint
  • 33e974c Merge pull request #1577 from crosbymichael/lint-1
  • 2416559 Merge pull request #1580 from tklauser/unix-mkdev-major-minor
  • f43b7ac Update files based on go lint
  • 579093f Merge pull request #1579 from tklauser/vendor-ansiterm
  • f01b139 archive: use Mkdev, Major and Minor functions from golang.org/x/sys/unix
  • 727fd59 ctr: remove SIGUNUSED from signal map
  • e789135 vendor: update golang.org/x/sys
  • acc246e Remove stray entry for Azure/go-ansiterm from vendor.conf
  • ce6859f images: provide separated children function for images
  • 16bf823 Merge pull request #1570 from AkihiroSuda/oci-runtime-readonly
  • 27023c7 readonly: pass RW rootfs to runtime, and let the runtime remount it as RO
  • 70b353d Merge pull request #1551 from mlaventure/client-pull-set-labels
  • 9f84513 Merge pull request #1571 from allencloud/remove-defaultns-from-Client
  • fd44a3a remove field defaultns from struct Client
  • df82159 client: Allow setting image labels on Pull() and Import()
  • b24acea Merge pull request #1565 from allencloud/fix-potential-panic
  • 382dafe fix a potential panic when map is nil
  • cc9f58a Merge pull request #1559 from mlaventure/fix-oor-panic
  • cee13eb Merge pull request #1557 from crosbymichael/console-bump
  • cf2c460 Bump runc with console change for ONLCR
  • d026f81 Merge pull request #1562 from AkihiroSuda/ctr-images-q
  • ed81189 Update console dep to 84eeaae905fa414d03e07bcd6c8d
  • 2a64813 ctr: add ctr images ls --quiet
  • af2226c client: Prevent Out-Of-Range panic in task.Metrics()
  • bd1f89b Merge pull request #1555 from crosbymichael/client-lint
  • 51b9240 Update client to pass go lint
  • 1a9d939 Merge pull request #1558 from mlaventure/dialer-quick-fix
  • 357eff8 Merge pull request #1556 from crosbymichael/tests
  • c42f56b Move unix specific tidbits into dialer_unix.go
  • 76a2d1c State adding tests when submitting changes
  • 76bfff3 Merge pull request #1554 from mlaventure/list-images-filters
  • 8e44719 Merge pull request #1552 from allencloud/update-docs-and-comments
  • 4700968 update docs and comments
  • 8f1dd43 client: Allow specifying filters when listing images

Dependency Changes

Previous release can be found at v1.0.0-beta.1

  • 7ddbeae9ae08c6a06a59597f0c9edbc5ff2444ce -> 314a259e304ff91bd6985da2a7149bbf91237993 golang.org/x/sys
  • 5933ab4dc4f7caa3a73a1dc141bd11f42b5c9163 -> 9c238e632e80d94f71a067c3deb9b34b1886ef18 github.com/containerd/cgroups
  • 593914b8bd5448a93f7c3e4902a03408b6d5c0ce -> 0351df1c5a66838d0c392b4ac4cf9450de844e2d github.com/opencontainers/runc
  • b3c048c028ddd789c6f9510c597f8b9c62f25359 -> 633fd07c086ff591adaa2849278764771d791f6f github.com/containerd/go-runc
  • b28c739c79ce69d017e3691ad3664568d68e95c6 -> 84eeaae905fa414d03e07bcd6c8d3f19e7cf180e github.com/containerd/console