Integrate with CRI package and add CRI client to containerd client #2923

qiutongs · 2021-08-19T21:38:12Z

Signed-off-by: Qiutong Song songqt01@gmail.com

Overview

Integrating with CRI is ideal in cadvisor. It can get container status, pod metadata, filesystem metrics(#2872) in a more straightforward way. Today, there is a circular dependency issue blocking from integrating the cri module. So this PR is merged to a separate branch from master.

Testing

make build
sudo ./cadvisor

# unit tests pass
go test github.com/google/cadvisor/container/containerd

make test

Signed-off-by: Qiutong Song <songqt01@gmail.com>

k8s-ci-robot · 2021-08-19T21:38:21Z

Hi @qiutongs. Thanks for your PR.

I'm waiting for a google member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

bobbypage · 2021-08-20T00:44:29Z

/ok-to-test

bobbypage · 2021-08-20T02:08:24Z

Looks like CI is failing due to

I0820 00:47:58.100] >> checking file boilerplate
I0820 00:47:58.216] Boilerplate header is wrong for: /go/src/github.com/google/cadvisor/container/cri-api/pkg/apis/runtime/v1/api.pb.go
I0820 00:47:58.216] Boilerplate header is wrong for: /go/src/github.com/google/cadvisor/container/cri-api/pkg/apis/runtime/v1/constants.go
I0820 00:47:58.217] Boilerplate header is wrong for: /go/src/github.com/google/cadvisor/container/cri-api/pkg/apis/runtime/v1alpha2/api.pb.go
I0820 00:47:58.217] Boilerplate header is wrong for: /go/src/github.com/google/cadvisor/container/cri-api/pkg/apis/runtime/v1alpha2/constants.go
W0820 00:47:58.317] make: *** [Makefile:97: presubmit] Error 1

We might need to exclude the directory cri-api here: https://github.com/google/cadvisor/blob/master/build/boilerplate/boilerplate.py#L98

Signed-off-by: Qiutong Song <songqt01@gmail.com>

qiutongs · 2021-08-23T00:56:11Z

Looks like CI is failing due to

I0820 00:47:58.100] >> checking file boilerplate
I0820 00:47:58.216] Boilerplate header is wrong for: /go/src/github.com/google/cadvisor/container/cri-api/pkg/apis/runtime/v1/api.pb.go
I0820 00:47:58.216] Boilerplate header is wrong for: /go/src/github.com/google/cadvisor/container/cri-api/pkg/apis/runtime/v1/constants.go
I0820 00:47:58.217] Boilerplate header is wrong for: /go/src/github.com/google/cadvisor/container/cri-api/pkg/apis/runtime/v1alpha2/api.pb.go
I0820 00:47:58.217] Boilerplate header is wrong for: /go/src/github.com/google/cadvisor/container/cri-api/pkg/apis/runtime/v1alpha2/constants.go
W0820 00:47:58.317] make: *** [Makefile:97: presubmit] Error 1

We might need to exclude the directory cri-api here: https://github.com/google/cadvisor/blob/master/build/boilerplate/boilerplate.py#L98

As discussed, I changed to go module.

bobbypage · 2021-08-23T20:31:59Z

/retest

bobbypage · 2021-08-24T00:40:14Z

Looks like CI is failing, @qiutongs can you please take a look?

Looks like you need to run make tidy

I0823 20:34:59.271] go tidy changes are needed; please run make tidy
I0823 20:34:59.272] go.mod diff:
I0823 20:34:59.272] 
I0823 20:34:59.272] go.sum diff:
I0823 20:34:59.272] --- /tmp/tmp.SM0sAKDNfk	2021-08-23 20:34:58.809645028 +0000
I0823 20:34:59.272] +++ go.sum	2021-08-23 20:34:59.253644526 +0000
I0823 20:34:59.272] @@ -509,8 +509,8 @@
I0823 20:34:59.273]  github.com/kr/pretty v0.2.1/go.mod h1:ipq/a2n7PKx3OHsz4KJII5eveXtPO4qwEXGdVfWzfnI=
I0823 20:34:59.273]  github.com/kr/pty v1.1.1/go.mod h1:pFQYn66WHrOpPYNljwOMqo10TkYh1fy3cYio2l3bCsQ=
I0823 20:34:59.273]  github.com/kr/pty v1.1.5/go.mod h1:9r2w37qlBe7rQ6e1fg1S/9xpWHSnaqNdHD3WcMdbPDA=
I0823 20:34:59.273] -github.com/kr/text v0.1.0 h1:45sCR5RtlFHMR4UwH9sdQ5TC8v0qDQCHnXt+kaKSTVE=
I0823 20:34:59.273]  github.com/kr/text v0.1.0/go.mod h1:4Jbv+DJW3UT/LiOwJeYQe1efqtUx/iVham/4vfdArNI=
I0823 20:34:59.273] +github.com/kr/text v0.2.0 h1:5Nx0Ya0ZqY2ygV366QzturHI13Jq95ApcVaJBhpS+AY=
I0823 20:34:59.273]  github.com/kr/text v0.2.0/go.mod h1:eLer722TekiGuMkidMxC/pM04lWEeraHUUmBw8l2grE=
I0823 20:34:59.274]  github.com/lightstep/lightstep-tracer-common/golang/gogo v0.0.0-20190605223551-bc2310a04743/go.mod h1:qklhhLq1aX+mtWk9cPHPzaBjWImj5ULL6C7HFJtXQMM=
I0823 20:34:59.274]  github.com/lightstep/lightstep-tracer-go v0.18.1/go.mod h1:jlF1pusYV4pidLvZ+XD0UBX0ZE6WURAspgAczcDHrL4=
I0823 20:34:59.274] @@ -574,6 +574,7 @@
I0823 20:34:59.274]  github.com/nats-io/nkeys v0.1.3/go.mod h1:xpnFELMwJABBLVhffcfd1MZx6VsNRFpEugbxziKVo7w=
I0823 20:34:59.274]  github.com/nats-io/nuid v1.0.1/go.mod h1:19wcPz3Ph3q0Jbyiqsd0kePYG7A95tJPxeL+1OSON2c=
I0823 20:34:59.274]  github.com/ncw/swift v1.0.47/go.mod h1:23YIA4yWVnGwv2dQlN4bB7egfYX6YLn0Yo/S6zZO/ZM=
I0823 20:34:59.275] +github.com/niemeyer/pretty v0.0.0-20200227124842-a10e7caefd8e h1:fD57ERR4JtEqsWbfPhv4DMiApHyliiK5xCTNVSPiaAs=
I0823 20:34:59.275]  github.com/niemeyer/pretty v0.0.0-20200227124842-a10e7caefd8e/go.mod h1:zD1mROLANZcx1PVRCS0qkT7pwLkGfwJo4zjcN/Tysno=
I0823 20:34:59.275]  github.com/nxadm/tail v1.4.4 h1:DQuhQpB1tVlglWS2hLQ5OV6B5r8aGxSrPc5Qo6uTN78=
I0823 20:34:59.275]  github.com/nxadm/tail v1.4.4/go.mod h1:kenIhsEOeOJmVchQTgglprH7qJGnHDVpk1VPCcaMI8A=
I0823 20:34:59.275] @@ -1165,8 +1166,8 @@
I0823 20:34:59.276]  gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
I0823 20:34:59.276]  gopkg.in/check.v1 v1.0.0-20141024133853-64131543e789/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
I0823 20:34:59.276]  gopkg.in/check.v1 v1.0.0-20180628173108-788fd7840127/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
I0823 20:34:59.276] -gopkg.in/check.v1 v1.0.0-20190902080502-41f04d3bba15 h1:YR8cESwS4TdDjEe65xsg0ogRM/Nc3DYOhEAlW+xobZo=
I0823 20:34:59.276]  gopkg.in/check.v1 v1.0.0-20190902080502-41f04d3bba15/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
I0823 20:34:59.276] +gopkg.in/check.v1 v1.0.0-20200227125254-8fa46927fb4f h1:BLraFXnmrev5lT+xlilqcH8XK9/i0At2xKjWk4p6zsU=
I0823 20:34:59.277]  gopkg.in/check.v1 v1.0.0-20200227125254-8fa46927fb4f/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
I0823 20:34:59.277]  gopkg.in/cheggaaa/pb.v1 v1.0.25/go.mod h1:V/YB90LKu/1FcN3WVnfiiE5oMCibMjukxqG/qStrOgw=
I0823 20:34:59.277]  gopkg.in/errgo.v2 v2.1.0/go.mod h1:hNsd1EY+bozCKY1Ytp96fpM3vjJbqLJn88ws8XvfDNI=
W0823 20:34:59.378] make: *** [Makefile:95: presubmit] Error 1

and there is also some lint errors in github actions jobs.

>> running golangci-lint using configuration at .golangci.yml
Error: container/containerd/client_test.go:27:2: `status` is unused (structcheck)
	status    *criapi.ContainerStatus
	^
Error: make: *** [Makefile:101: lint] Error 1
Error: Process completed with exit code 2.

qiutongs · 2021-08-25T05:47:44Z

/retest

bobbypage · 2021-08-25T21:38:44Z

Looks like there is still tidy issue, cmd/go.sum needs to be updated:

$ gh pr checkout 2923
$ make tidy
$ git status
On branch cri-integration
Changes not staged for commit:
  (use "git add <file>..." to update what will be committed)
  (use "git restore <file>..." to discard changes in working directory)
        modified:   cmd/go.sum

no changes added to commit (use "git add" and/or "git commit -a")
$ git diff
diff --git a/cmd/go.sum b/cmd/go.sum
index 59c783b8..4f0fe057 100644
--- a/cmd/go.sum
+++ b/cmd/go.sum
@@ -509,7 +509,6 @@ github.com/kr/pretty v0.2.1 h1:Fmg33tUaq4/8ym9TJN1x7sLJnHVwhP33CNkpYV/7rwI=
 github.com/kr/pretty v0.2.1/go.mod h1:ipq/a2n7PKx3OHsz4KJII5eveXtPO4qwEXGdVfWzfnI=
 github.com/kr/pty v1.1.1/go.mod h1:pFQYn66WHrOpPYNljwOMqo10TkYh1fy3cYio2l3bCsQ=
 github.com/kr/pty v1.1.5/go.mod h1:9r2w37qlBe7rQ6e1fg1S/9xpWHSnaqNdHD3WcMdbPDA=
-github.com/kr/text v0.1.0 h1:45sCR5RtlFHMR4UwH9sdQ5TC8v0qDQCHnXt+kaKSTVE=
 github.com/kr/text v0.1.0/go.mod h1:4Jbv+DJW3UT/LiOwJeYQe1efqtUx/iVham/4vfdArNI=
 github.com/kr/text v0.2.0 h1:5Nx0Ya0ZqY2ygV366QzturHI13Jq95ApcVaJBhpS+AY=
 github.com/kr/text v0.2.0/go.mod h1:eLer722TekiGuMkidMxC/pM04lWEeraHUUmBw8l2grE=
@@ -1167,7 +1166,6 @@ gopkg.in/alecthomas/kingpin.v2 v2.2.6/go.mod h1:FMv+mEhP44yOT+4EoQTLFTRgOQ1FBLks
 gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
 gopkg.in/check.v1 v1.0.0-20141024133853-64131543e789/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
 gopkg.in/check.v1 v1.0.0-20180628173108-788fd7840127/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
-gopkg.in/check.v1 v1.0.0-20190902080502-41f04d3bba15 h1:YR8cESwS4TdDjEe65xsg0ogRM/Nc3DYOhEAlW+xobZo=
 gopkg.in/check.v1 v1.0.0-20190902080502-41f04d3bba15/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
 gopkg.in/check.v1 v1.0.0-20200227125254-8fa46927fb4f h1:BLraFXnmrev5lT+xlilqcH8XK9/i0At2xKjWk4p6zsU=
 gopkg.in/check.v1 v1.0.0-20200227125254-8fa46927fb4f/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=

For reference, make sure you're on go1.17:

$ go version
go version go1.17 linux/amd64

Signed-off-by: Qiutong Song <songqt01@gmail.com>

qiutongs · 2021-08-26T05:13:54Z

/retest

qiutongs · 2021-08-26T05:56:29Z

/retest

bobbypage · 2021-08-26T19:36:19Z

/retest

bobbypage · 2021-08-26T19:45:50Z

/retest

Integrate with CRI package and add CRI client to containerd client

e35a798

Signed-off-by: Qiutong Song <songqt01@gmail.com>

google-cla bot added the cla: yes label Aug 19, 2021

k8s-ci-robot added the needs-ok-to-test label Aug 19, 2021

qiutongs mentioned this pull request Aug 19, 2021

support collecting FsUsageMetrics for containerd #2872

Closed

k8s-ci-robot added ok-to-test and removed needs-ok-to-test labels Aug 20, 2021

Remove copied cri-api and import go module

e2a098a

Signed-off-by: Qiutong Song <songqt01@gmail.com>

make tidy and fix the lint error

dea607d

upgrade to go1.17 and make tidy

aac46a6

Signed-off-by: Qiutong Song <songqt01@gmail.com>

qiutongs force-pushed the cri-integration branch from fa892ea to aac46a6 Compare August 26, 2021 05:13

bobbypage approved these changes Aug 26, 2021

View reviewed changes

bobbypage merged commit c234fa0 into google:containerd-cri Aug 26, 2021

qiutongs deleted the cri-integration branch August 27, 2021 04:11

bobbypage mentioned this pull request Oct 11, 2021

Integrate CRI and containerd file system stats #2956

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Integrate with CRI package and add CRI client to containerd client #2923

Integrate with CRI package and add CRI client to containerd client #2923

qiutongs commented Aug 19, 2021 •

edited

k8s-ci-robot commented Aug 19, 2021

bobbypage commented Aug 20, 2021

bobbypage commented Aug 20, 2021

qiutongs commented Aug 23, 2021

bobbypage commented Aug 23, 2021

bobbypage commented Aug 24, 2021 •

edited

qiutongs commented Aug 25, 2021

bobbypage commented Aug 25, 2021

qiutongs commented Aug 26, 2021

qiutongs commented Aug 26, 2021

bobbypage commented Aug 26, 2021

bobbypage commented Aug 26, 2021

Integrate with CRI package and add CRI client to containerd client #2923

Integrate with CRI package and add CRI client to containerd client #2923

Conversation

qiutongs commented Aug 19, 2021 • edited

Overview

Testing

k8s-ci-robot commented Aug 19, 2021

bobbypage commented Aug 20, 2021

bobbypage commented Aug 20, 2021

qiutongs commented Aug 23, 2021

bobbypage commented Aug 23, 2021

bobbypage commented Aug 24, 2021 • edited

qiutongs commented Aug 25, 2021

bobbypage commented Aug 25, 2021

qiutongs commented Aug 26, 2021

qiutongs commented Aug 26, 2021

bobbypage commented Aug 26, 2021

bobbypage commented Aug 26, 2021

qiutongs commented Aug 19, 2021 •

edited

bobbypage commented Aug 24, 2021 •

edited