Upstream or remove Kubernetes patches

[erikwilson]

Is your feature request related to a problem? Please describe.
Audit of our k8s patch set and work to reduce that.
Enumerates each change, what it does, and why it is needed.

Additional context
#548

Generated from:

cd ~/go/src/github.com/kubernetes/kubernetes
K8S=v1.22.2; git log --name-only --reverse --pretty=format:'* **[`%h`](https://github.com/k3s-io/kubernetes/commit/%H) %s**%n  by %an <%ae> on `%as`%n  %n%w(,2,2)%b' ${K8S}-k3s1~1...${K8S} | sed -E 's|^[[:alnum:]]+.*|    * `&`|'

Changes

• 0bc428a57b8 kubelet: new cgroup driver: "none"
  by Akihiro Suda akihiro.suda.cz@hco.ntt.co.jp on 2019-06-02

  The "none" driver is used for running "rootless" mode on a host that does not support cgroup v2.

  This commit is specific to Usernetes and isn't going to be proposed to the Kubernetes upstream.
  So, there is no FeatureFlag for gating this driver.

  Signed-off-by: Akihiro Suda akihiro.suda.cz@hco.ntt.co.jp

  • cmd/kubelet/app/options/options.go
  • cmd/kubelet/app/server.go
  • pkg/kubelet/apis/config/types.go
  • pkg/kubelet/cm/cgroup_manager_linux.go

• 018a084aaa8 Drop client-go cloud auth
  by Darren Shepherd darren@rancher.com on 2019-09-05

  • cmd/kubectl/kubectl.go
  • pkg/kubeapiserver/authenticator/config.go

• 094c8c4758c Drop storage plugins
  by Darren Shepherd darren@rancher.com on 2019-09-05

  • cmd/kube-controller-manager/app/plugins.go
  • cmd/kubelet/app/plugins.go

• 96b01267556 Drop credential providers
  by Darren Shepherd darren@rancher.com on 2019-09-05

  • cmd/kubelet/app/options/globalflags.go

• f5b3ff458c7 If you can't set hashsize on nf_conntrack don't fail
  by Darren Shepherd darren@rancher.com on 2018-10-05

  • cmd/kube-proxy/app/conntrack.go

• ac21553e5a5 only use the resolved name if port was zero
  by Darren Shepherd darren@rancher.com on 2018-12-31

  • pkg/kubelet/cri/streaming/server.go

• 7b0c45ae378 Make kubelet.sock path changable
  by Darren Shepherd darren@rancher.com on 2019-01-09

  • staging/src/k8s.io/kubelet/pkg/apis/deviceplugin/v1beta1/constants.go

• 039ffbb1cbf Wait for kube-apiserver for 2 minutes for slow (ARM) systems
  by Darren Shepherd darren@rancher.com on 2019-01-22

  • cmd/kube-controller-manager/app/controllermanager.go

• 91eb20a1943 Don't check for cpuset cgroup, not always required?
  by Darren Shepherd darren@rancher.com on 2019-01-22

  • pkg/kubelet/cm/container_manager_linux.go

• 3ec6171e307 Set all sources so node+agent in the same process doesn't get restricted
  by Darren Shepherd darren@rancher.com on 2019-02-07

  • cmd/kube-apiserver/app/server.go

• 38b8e7ff661 Hide deprecated warnings
  by Darren Shepherd darren@rancher.com on 2019-02-07

  • cmd/kubelet/app/options/options.go
  • staging/src/k8s.io/apiserver/pkg/server/options/deprecated_insecure_serving.go

• 5e319e3a223 Add ability to disable proxy hostname check
  by Darren Shepherd darren@rancher.com on 2019-02-07

  • pkg/proxy/util/utils.go

• f5172425969 Cache loopback cert in the certs dir if set
  by Darren Shepherd darren@rancher.com on 2019-02-07

  • staging/src/k8s.io/apiserver/pkg/server/options/serving_with_loopback.go

• cdb18d010cc Don't ever select the flannel bridge or cni bridge
  by Darren Shepherd darren@rancher.com on 2019-02-07

  • staging/src/k8s.io/apimachinery/pkg/util/net/interface.go

• 32b7b2050e1 Update kubernetes service on start for port changes
  by Erik Wilson Erik.E.Wilson@gmail.com on 2019-06-21

  • pkg/controlplane/controller.go

• ec39217a8e5 Add WrappedRoundTripper() to tokenSourceTransport
  by Darren Shepherd darren@rancher.com on 2019-02-27

  • staging/src/k8s.io/client-go/transport/token_source.go

• 29d1cbd8bb0 Add stopCh to apiserver & context to kublet commands
  by Darren Shepherd darren@rancher.com on 2019-08-26

  • cmd/genkubedocs/gen_kube_docs.go
  • cmd/genman/gen_kube_man.go
  • cmd/kube-apiserver/apiserver.go
  • cmd/kube-apiserver/app/server.go
  • cmd/kubelet/app/server.go
  • cmd/kubelet/kubelet.go

• 5cbb88fd4c8 Allow override of kubeconfig dialer
  by Darren Shepherd darren@rancher.com on 2018-10-09

  • cmd/kube-apiserver/app/server.go

• 22f3ec25de3 Notify startup to grab a hold of handler and authenticator
  by Darren Shepherd darren@rancher.com on 2019-09-18

  • cmd/kube-apiserver/app/server.go

• 4759b2b71cb Ignore proxy settings for kubelet client
  by Erik Wilson Erik.E.Wilson@gmail.com on 2019-04-15

  Proxy settings may interfere with a kubelet client communicating with
  the API server, so set the proxy to nil.

  • cmd/kube-apiserver/app/server.go
  • pkg/kubelet/client/kubelet_client.go

• cb65816a90f Allow override of "kubernetes" endpoint port
  by Darren Shepherd darren@rancher.com on 2018-10-08

  • staging/src/k8s.io/apiserver/pkg/server/config.go
  • staging/src/k8s.io/apiserver/pkg/server/options/serving.go

• f8fe3d4256b Don't check etcd if scheme is unix
  by Darren Shepherd darren@rancher.com on 2019-08-28

  • pkg/registry/core/rest/storage_core.go

• 39523fdeb00 Fix CSI initialization conflict
  by Darren Shepherd darren@rancher.com on 2019-08-30

  CSI is used by both the kubelet and kube-controller-manager. Both
  components will initialize the csiPlugin with different VolumeHost
  objects. The csiPlugin will then assign a global variable for
  the node info manager. It is then possible that the kubelet gets
  the credentials of the kube-controller-manager and that will cause
  CSI to fail.

  • pkg/volume/csi/csi_plugin.go

• df45a2cbc63 Fix inconsistent etcd read
  by Darren Shepherd darren@rancher.com on 2019-11-18

  • staging/src/k8s.io/apiserver/pkg/storage/etcd3/store.go

• 921e7df2305 Add openapi generator
  by Darren Shepherd darren@rancher.com on 2019-08-27

  • pkg/generated/openapi/gen/main.go

• cb4f3b09912 Add tag.sh script
  by Darren Shepherd darren@rancher.com on 2019-08-27

  • tag.sh

• c4ebe1924cb Add Vagrantfile
  by Erik Wilson Erik.E.Wilson@gmail.com on 2019-09-19

  • Vagrantfile

[olljanat]

I ended up here as searched about k3s + rooless stuff so FYI that there is already PR open to upstream rootless patches kubernetes/kubernetes#92863

[dims]

@erikwilson are there things here that could get into k8s 1.22? Is there a better place to look at how the upstreaming efforts are going on?

[dweomer]

@erikwilson are there things here that could get into k8s 1.22? Is there a better place to look at how the upstreaming efforts are going on?

This is probably as good a place to discuss any, prior to moving to a KEP or simple PR.

[cjellick]

Hey @dims - thanks for asking about this.

We'll work through this list in the next couple weeks to see what if anything can be moved forward in the 1.22 timeframe. Balanced against other obligations, that is somewhat tight, so we might be looking further out for bigger changes.

Is there anything in particular that you are looking for or just interested in this generally?

[dims]

@cjellick i just want the teams to start talking and doing things together :)

[AkihiroSuda]

232effd3d57 kubelet/cm: ignore sysctl error when running in userns
125fe9d9a5c kube-proxy: allow running in userns
c0d770b90bf kubelet: new feature gate: Rootless

Merged in v1.22 (feature gate was renamed to KubeletInUserNamespace)
https://github.com/kubernetes/kubernetes/pull/92863/commits

[brandond]

We'll be dropping a few patches in 1.22, as discussed over at #3596 (comment).

@AkihiroSuda for the CgroupNone patch, do you think we should drop that as well? I believe the effect of that would be to require delegated cgroupv2 under systemd for rootless, which means not being able to run rootless K3s from the command line?

[AkihiroSuda]

for the CgroupNone patch, do you think we should drop that as well?

Yes, not sure it should be in 1.22 or 1.23, though.

I believe the effect of that would be to require delegated cgroupv2 under systemd for rootless, which means not being able to run rootless K3s from the command line?

systemd-run --user -p Delegate=yes --tty k3s server --rootless can continue to work.

k3s/k3s-rootless.service

Line 28 in 238dc20

# If you really need to try it on a terminal, prepend `systemd-run --user -p Delegate=yes --tty` to create a systemd scope.

[brandond]

I'm hijacking the first comment to start running down the current (as of 1.22.2) list of patches we're carrying. @dims would you be interested in going over these with me at some point to discuss which ones you think we might be able to adapt for inclusion upstream?

[dims]

why yes of course @brandond . i can't speak for all the sigs for sure and we will need to go through regular process(es) here, but yes happy to chat!

[caroline-suse-rancher]

@brandond is this safe to close?

[brandond]

no, this is an ongoing effort. We still require a handful of the above-listed patches to embed Kubernetes in K3s.

[dims]

@brandond @caroline-suse-rancher what does 2023 confirmed label mean?

[brandond]

That we're going to continue working on this in 2023.