Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

kubeadm: kube-proxy needs to know the pod subnet CIDR #39440

Merged
merged 1 commit into from
Feb 2, 2017

Conversation

pires
Copy link
Contributor

@pires pires commented Jan 4, 2017

What this PR does / why we need it: kube-proxy 1.5 has a new flag cluster-cidr that isn't specified by kubeadm, thus resulting in bug kubernetes/kubeadm#102.

Which issue this PR fixes: fixes kubernetes/kubeadm#102

Special notes for your reviewer:
/cc @luxas @dmmcquay

@k8s-ci-robot k8s-ci-robot added the cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. label Jan 4, 2017
@k8s-reviewable
Copy link

This change is Reviewable

@k8s-github-robot k8s-github-robot added size/S Denotes a PR that changes 10-29 lines, ignoring generated files. release-note-label-needed labels Jan 4, 2017
@amacneil
Copy link
Contributor

amacneil commented Jan 5, 2017

This might be a noob question, but should the kube-proxy --cluster-cidr be set to match the kubeadm --pod-network-cidr or --service-cidr?

@luxas luxas added release-note-none Denotes a PR that doesn't merit a release note. and removed release-note-label-needed labels Jan 5, 2017
@luxas luxas assigned luxas and unassigned errordeveloper Jan 5, 2017
@pires
Copy link
Contributor Author

pires commented Jan 5, 2017

@amacneil the pod network. Let me remind you the service network is already known to the API server.

@pires
Copy link
Contributor Author

pires commented Jan 7, 2017

@luxas ping

@pires
Copy link
Contributor Author

pires commented Jan 9, 2017

ping @luxas

@luxas
Copy link
Member

luxas commented Jan 9, 2017

As I mentioned on Slack, this doesn't fix the issue in most cases.

kubeadm has no idea about the pod subnet except when using flannel in k8s api backed mode.
In all other cases, kubeadm don't know what the CIDR is gonna be, e.g. Weave has 10.32.0.0/16 IIRC, Calico and Canal has something else, flannel defaults to 10.244.0.0/16 etc.

We just don't know this and never will. I'm curious exactly what the issue with kube-proxy is when it does not know this, I've never encountered it.

@thockin (feel free to ping somebody else that knows) why does kube-proxy have this flag? At a first glance it seems like bad design for this particular flag, but on the other hand I don't have the full context on this.

TL;DR; This PR does not solve the issue, but we should discuss how we can solve the issue anyway.

@thockin
Copy link
Member

thockin commented Jan 10, 2017

cmd/kube-proxy/app/options/options.go

84: fs.StringVar(&s.ClusterCIDR, "cluster-cidr", s.ClusterCIDR, "The CIDR range of pods in the cluster. It is used to bridge traffic coming from outside of the cluster. If not provided, no off-cluster bridging will be performed.")

@pires
Copy link
Contributor Author

pires commented Feb 1, 2017

@luxas all checks are green, ping.

@luxas
Copy link
Member

luxas commented Feb 1, 2017

We discussed this during the sig meeting, and will do an other approach than we initially thought.

We'll encourage (and maybe even require) the user to setting this Pod CIDR value on kubeadm init time, and the cni daemonset yaml manifest as well. Even though it's suboptimal to start the subnet allocation feature of the controller-manager for network vendors like weave that uses something else, we chose to not expose that boolean because it would be a too detailed thing to expose to the normal user.

So we will encourage setting both the CNI network manifest and this Pod CIDR, which also allows us to do what this PR implements, namely pass that Pod CIDR to kube-proxy.

/lgtm
/approve

@k8s-ci-robot k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Feb 1, 2017
@k8s-github-robot
Copy link

[APPROVALNOTIFIER] This PR is APPROVED

The following people have approved this PR: luxas, pires

Needs approval from an approver in each of these OWNERS Files:

You can indicate your approval by writing /approve in a comment
You can cancel your approval by writing /approve cancel in a comment

@k8s-github-robot k8s-github-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Feb 1, 2017
@k8s-github-robot
Copy link

Automatic merge from submit-queue (batch tested with PRs 40574, 40806, 40308, 40771, 39440)

@k8s-github-robot k8s-github-robot merged commit 9dedf92 into kubernetes:master Feb 2, 2017
@pires pires deleted the kubeadm_102-fix_proxy branch February 2, 2017 07:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. lgtm "Looks good to me", indicates that a PR is ready to be merged. release-note-none Denotes a PR that doesn't merit a release note. size/S Denotes a PR that changes 10-29 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

nodes with multiple network interfaces can fail to talk to services
8 participants