Skip to content
This repository has been archived by the owner on Feb 5, 2020. It is now read-only.

ELB Integration failing when Cluster installed via Terraform #243

Closed
chrisricci opened this issue Apr 14, 2017 · 4 comments
Closed

ELB Integration failing when Cluster installed via Terraform #243

chrisricci opened this issue Apr 14, 2017 · 4 comments

Comments

@chrisricci
Copy link

Exposing a service of type='LoadBalancer' fails to provision the ELB in AWS.

Controller Logs shows the following:

E0414 18:44:13.332619       1 servicecontroller.go:760] Failed to process service. Retrying in 5m0s: Failed to create load balancer for service policy-demo/hello-world: Multiple tagged security groups found for instance i-041e7788960d67752; ensure only the k8s security group is tagged
W0414 18:49:13.802150       1 aws.go:2751] Error opening ingress rules for the load balancer to the instances: Multiple tagged security groups found for instance i-041e7788960d67752; ensure only the k8s security group is tagged
E0414 18:49:13.802323     
@bobstead
Copy link

bobstead commented Apr 14, 2017

It looks like this is a similar problem to kubernetes/kubernetes#26787.

The master and workers all have two security groups that are tagged KubernetesCluster=<value>. Removing that specific tag from the sg-cluster_default security group that the tectonic installer sets up, fixes the issue and kubernetes was able to successfully create the ELB and assign the correct instances.

Perhaps the cluster default security group should not be tagged with KubernetesCluster=??

A fix, assuming it's the right thing to do, would be to patch security-groups.tf so that the KubernetesCluster tag is not added.

Further analysis shows that PR 218 added these tags to the file linked above, but probably shouldn't have added the KubernetesCluster tag in there also. I'm unable to add PR's to this project at this time (work reasons), if someone can create a patch for that file, I believe it would fix this particular issue.

@sym3tri sym3tri added this to the AWS is stable milestone Apr 14, 2017
@alexsomesan
Copy link
Contributor

We are going to reorganize the security groups to explicitly open ports as needed. We'll keep this aspect in mind when implementing #248 and there should not be any other work necessary to fix this issue once that is complete.

s-urbaniak pushed a commit to s-urbaniak/tectonic-installer that referenced this issue Apr 19, 2017
Currently masters and workers share a pretty open security group.
Furthermore workers expose ingress traffic at critical k8s ports like
10250 and 10255.

This fixes it by removing the common cluster default security group and
specifying separate ingress/egress rules reflecting settings from the
current tectonic installer.

It also assigns only one security group for masters and workers.

Fixes coreos#248, coreos#243
s-urbaniak pushed a commit to s-urbaniak/tectonic-installer that referenced this issue Apr 19, 2017
Currently masters and workers share a pretty open security group.
Furthermore workers expose ingress traffic at critical k8s ports like
10250 and 10255.

This fixes it by removing the common cluster default security group and
specifying separate ingress/egress rules reflecting settings from the
current tectonic installer.

It also assigns only one security group for masters and workers.

Fixes coreos#248, coreos#243, coreos#227
@s-urbaniak
Copy link
Contributor

s-urbaniak commented Apr 19, 2017

#264 fixes this by assigning at most one security group for workers/masters.

alexsomesan pushed a commit that referenced this issue Apr 19, 2017
* modules/aws: tighten security groups

Currently masters and workers share a pretty open security group.
Furthermore workers expose ingress traffic at critical k8s ports like
10250 and 10255.

This fixes it by removing the common cluster default security group and
specifying separate ingress/egress rules reflecting settings from the
current tectonic installer.

It also assigns only one security group for masters and workers.

Fixes #248, #243, #227

* Documentation/generic-platform: change flannel port to 4789

... because that one is configured and recommended since it is the IANA
based one. Tools like tcpdump then decode vxlan packets natively.

The old port (8472) is retained as the default port in the kernel for
backwards compatibility purposes only, see [1].

Other projects also switched to the new IANA assigned port.

[1] http://lxr.free-electrons.com/source/drivers/net/vxlan.c#L43
@s-urbaniak
Copy link
Contributor

resolved by #264

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

No branches or pull requests

5 participants