suggest adding github actions

[mikebrow]

In particular it would be beneficial to be testing the various cloud container runtimes against cri-test against k8s master and cri-tools master to ensure stability of the CRI api. ATM only dockershim is being tested in travis for cri-tools CI.

Desired Actions:

• containerd items

  • containerd master ubuntu 18.04 LTS runc
  • matrix over available platforms, releases of containerd, runtimes(crun, runc)

• crio items

  • cri-o master ubuntu 18.04 LTS runc
  • matrix over available platforms, releases of cri-o, runtimes(crun, runc)
  • remove SKIP for "runtime should support port mapping with host port and container port"

• dockershim items

  • dockershim master ubuntu 18.04 LTS runc
  • matrix over available platforms, releases of dockershim, runtimes(crun, runc), (Dockershim critest on Windows: wip Dockershim critest on Windows #634)
  • remove dockershim critest on travis, or fix it to run with a standalone dockershim, ...Update dockershim critest #611
  • need an owner/host for dockershim moby/kubernetes-sigs/_____ (not a scope fo cri-tools just tracking) (currently Run cri-dockerd using local-up-cluster + conformance dims/cri-dockerd#4)

• All Container Runtimes:

  • missing platforms blocked by github action runners (linux-ppc64le, ARM, s390x, ...)
  • VM isolated pods / containers with runc/crun hosting inside a VM for each pod/container using katacontainers, (hyper(?) for windows)

[saschagrunert]

Okay, how would the test look like?

We test the crictl master against a stable version of container runtimes? We could also add prow jobs to test-infra for that. But I'm also open to use GitHub actions.

[mikebrow]

just a simple install and run of container runtime (with it's dependencies)
followed by running cri-test against it's sock..

I would suggest running against the latest release of the container runtime with a must run and the container runtime's master branch with an "optional" run.

These would be fast checks... allowing for us to add new features and see if they are breaking changes against the container runtime.

[mikebrow]

https://github.com/containerd/cri/blob/master/.github/workflows/ci.yml#L75-L143

minus the unit and integration tests... and add a parameter to set the current master branch version of cri-tools .. I've been chatting it up a bit with @hickeyma for the potential of a containerd test here.. need to broach it with the cri-o team to add one for theirs.. similar to the e2e tests already added. I like the github actions mostly because they can be tested in the developers fork before pushing the PR.

[saschagrunert]

Yeah, sounds good to me. 👍 Are you planning to open-up a PR for that?

[mikebrow]

Yeah, sounds good to me. 👍 Are you planning to open-up a PR for that?

I or @hickeyma for containerd.. if that works out well we could do the crio one.. or someone could volunteer :-)

[mikebrow]

dockershim .. dunno.. I'd wait till it's back up working against k8s master again .. then maybe move it off travis as well..

[saschagrunert]

I can take the CRI-O part, for sure :)

[mikebrow]

I can take the CRI-O part, for sure :)

I think you'll like github actions.

[hickeyma]

I can do the containerd part if that's ok?

[dims]

fyi, i got the standalone dockershim working in a github action

• https://github.com/dims/cri-tools/pull/3/files

Here's a clean run:

• https://github.com/dims/cri-tools/runs/717714970?check_suite_focus=true

[mikebrow]

fyi, i got the standalone dockershim working in a github action

• https://github.com/dims/cri-tools/pull/3/files

Here's a clean run:

• https://github.com/dims/cri-tools/runs/717714970?check_suite_focus=true

Whoot! that's three CRIs!

[mikebrow]

now just need to decide where to put it.. kubernetes-sigs/dockershim?
or back into the kubernetes/kubernetes kublet tree as another binary target

will be an interesting discussion for sure at the next sig-node meeting

[dims]

@mikebrow @thaJeztah how about somewhere in moby github org?

[dims]

there was discussions already about this in code-organization and we did not like any of the options.

[hickeyma]

@dims Way to go. Excellent work. 👏

[dims]

fyi, i threw in a github action that uses cri-tool tests in the cri-dockerd repo and it turned out really well
latest run - https://github.com/dims/cri-dockerd/runs/720555441?check_suite_focus=true
github action itself - https://github.com/dims/cri-dockerd/blob/master/.github/workflows/main.yml

[hickeyma]

Nice work Dims! :)

[mikebrow]

added a work item list above

[saschagrunert]

Working on the CRI-O master part in #619

[saschagrunert]

• cri-o matrix over platforms, releases of containerd, runtimes(crun, runc, (kata?)

@mikebrow did you mean:

• cri-o matrix over platforms, releases of cri-o, runtimes(crun, runc, (kata?)

I guess we can add a test to run against the latest version of CRI-O. crun would be possible to if we add it to the static binary bundle of CRI-O.

[mikebrow]

• cri-o matrix over platforms, releases of containerd, runtimes(crun, runc, (kata?)

@mikebrow did you mean:

• cri-o matrix over platforms, releases of cri-o, runtimes(crun, runc, (kata?)

I guess we can add a test to run against the latest version of CRI-O. crun would be possible to if we add it to the static binary bundle of CRI-O.

Yes, cri-o, cut and paste fail :-)

Really it just meant we should add appropriate test variations (possibly tagged as optional).

For example, if you have a crio release candidate targeted to the latest release of kubernetes we should test that as well as your master. As for crun, I've been seeing subtle timing and other meta type issues with various tests when we switch from runc to crun. So probably good to run both possibly with crun as optional, so we can find those issues early with new or modified tests or dependencies (like :latest containers used in test).

[mikebrow]

updated status ^

[hickeyma]

@mikebrow I think the list could be updated following merging of PRs since last update. I would think all is complete except for:

• Owner for dockershim moby/kubernetes-sigs
• Dockershim critest on Windows: Dockershim critest on Windows #634
• Build on linux-ppc64le: Not yet supported by GH actions

[mikebrow]

@mikebrow I think the list could be updated following merging of PRs since last update. I would think all is complete except for:

• Owner for dockershim moby/kubernetes-sigs
• Dockershim critest on Windows: Dockershim critest on Windows #634
• Build on linux-ppc64le: Not yet supported by GH actions

updated

[fejta-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

[saschagrunert]

@mikebrow is there anything we would like to integrate before closing this issue? :)

[fejta-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-contributor-experience at kubernetes/community.
/lifecycle stale

[fejta-bot]

Stale issues rot after 30d of inactivity.
Mark the issue as fresh with /remove-lifecycle rotten.
Rotten issues close after an additional 30d of inactivity.

If this issue is safe to close now please do so with /close.

Send feedback to sig-contributor-experience at kubernetes/community.
/lifecycle rotten

[fejta-bot]

Rotten issues close after 30d of inactivity.
Reopen the issue with /reopen.
Mark the issue as fresh with /remove-lifecycle rotten.

Send feedback to sig-contributor-experience at kubernetes/community.
/close

[k8s-ci-robot]

@fejta-bot: Closing this issue.

In response to this:

Rotten issues close after 30d of inactivity.
Reopen the issue with /reopen.
Mark the issue as fresh with /remove-lifecycle rotten.

Send feedback to sig-contributor-experience at kubernetes/community.
/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.