Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add AWS Transit Gateway support #10948

Merged
merged 2 commits into from
Mar 1, 2021
Merged

Add AWS Transit Gateway support #10948

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
577df0a
Add support for defining a transit gateway for a default route
rifelpet Feb 28, 2021
2ebd448
Add integration test for transit gateway support
rifelpet Feb 28, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions cloudmock/aws/mockec2/routetable.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -164,6 +164,7 @@ func (m *MockEC2) CreateRoute(request *ec2.CreateRouteInput) (*ec2.CreateRouteOu
InstanceId: request.InstanceId,
NatGatewayId: request.NatGatewayId,
NetworkInterfaceId: request.NetworkInterfaceId,
TransitGatewayId: request.TransitGatewayId,
VpcPeeringConnectionId: request.VpcPeeringConnectionId,
}

Expand Down
12 changes: 12 additions & 0 deletions docs/cluster_spec.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -304,6 +304,18 @@ spec:
zone: us-east-1a
```

Specifying an existing AWS Transit gateways is also supported as of kOps 1.20.0:

```yaml
spec:
subnets:
- cidr: 10.20.64.0/21
name: us-east-1a
egress: tgw-0123456789abcdef0
type: Private
zone: us-east-1a
```

In the case that you don't use NAT gateways or internet gateways, kOps 1.12.0 introduced the "External" flag for egress to force kOps to ignore egress for the subnet. This can be useful when other tools are used to manage egress for the subnet such as virtual private gateways. Please note that your cluster may need to have access to the internet upon creation, so egress must be available upon initializing a cluster. This is intended for use when egress is managed external to kOps, typically with an existing cluster.

```yaml
Expand Down
4 changes: 3 additions & 1 deletion pkg/apis/kops/cluster.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -574,8 +574,10 @@ const (
EgressNatGateway = "nat"
// EgressElasticIP means that egress configuration is using a NAT Gateway with an existing Elastic IP
EgressElasticIP = "eipalloc"
// EgressElasticIP means that egress configuration is using an existing NAT Instance
// EgressNatInstance means that egress configuration is using an existing NAT Instance
EgressNatInstance = "i"
// EgressTransitGateway means that egress configuration is using a Transit Gateway
EgressTransitGateway = "tgw"
// EgressExternal means that egress configuration is done externally (preconfigured)
EgressExternal = "External"
)
Expand Down
4 changes: 2 additions & 2 deletions pkg/apis/kops/validation/validation.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -359,9 +359,9 @@ func validateSubnet(subnet *kops.ClusterSubnetSpec, fieldPath *field.Path) field

if subnet.Egress != "" {
egressType := strings.Split(subnet.Egress, "-")[0]
if egressType != kops.EgressNatGateway && egressType != kops.EgressElasticIP && egressType != kops.EgressNatInstance && egressType != kops.EgressExternal {
if egressType != kops.EgressNatGateway && egressType != kops.EgressElasticIP && egressType != kops.EgressNatInstance && egressType != kops.EgressExternal && egressType != kops.EgressTransitGateway {
allErrs = append(allErrs, field.Invalid(fieldPath.Child("egress"), subnet.Egress,
"egress must be of type NAT Gateway, NAT Gateway with existing ElasticIP, NAT EC2 Instance or External"))
"egress must be of type NAT Gateway, NAT Gateway with existing ElasticIP, NAT EC2 Instance, Transit Gateway, or External"))
}
if subnet.Egress != kops.EgressExternal && subnet.Type != "Private" {
allErrs = append(allErrs, field.Forbidden(fieldPath.Child("egress"), "egress can only be specified for private subnets"))
Expand Down
8 changes: 6 additions & 2 deletions pkg/model/network.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -303,6 +303,7 @@ func (b *NetworkModelBuilder) Build(c *fi.ModelBuilderContext) error {
}

var ngw *awstasks.NatGateway
var tgwID *string
var in *awstasks.Instance
if egress != "" {
if strings.HasPrefix(egress, "nat-") {
Expand Down Expand Up @@ -353,7 +354,8 @@ func (b *NetworkModelBuilder) Build(c *fi.ModelBuilderContext) error {
}

c.AddTask(in)

} else if strings.HasPrefix(egress, "tgw-") {
tgwID = &egress
} else if egress == "External" {
// Nothing to do here
} else {
Expand Down Expand Up @@ -439,7 +441,9 @@ func (b *NetworkModelBuilder) Build(c *fi.ModelBuilderContext) error {
Lifecycle: b.Lifecycle,
CIDR: s("0.0.0.0/0"),
RouteTable: rt,
NatGateway: ngw,
// Only one of these will be not nil
NatGateway: ngw,
TransitGatewayID: tgwID,
}
}
c.AddTask(r)
Expand Down
146 changes: 146 additions & 0 deletions tests/integration/update_cluster/complex/cloudformation.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -596,6 +596,50 @@
]
}
},
"AWSEC2RouteTableprivateustest1acomplexexamplecom": {
"Type": "AWS::EC2::RouteTable",
"Properties": {
"VpcId": {
"Ref": "AWSEC2VPCcomplexexamplecom"
},
"Tags": [
{
"Key": "KubernetesCluster",
"Value": "complex.example.com"
},
{
"Key": "Name",
"Value": "private-us-test-1a.complex.example.com"
},
{
"Key": "Owner",
"Value": "John Doe"
},
{
"Key": "foo/bar",
"Value": "fib+baz"
},
{
"Key": "kubernetes.io/cluster/complex.example.com",
"Value": "owned"
},
{
"Key": "kubernetes.io/kops/role",
"Value": "private-us-test-1a"
}
]
}
},
"AWSEC2Routeprivateustest1a00000": {
"Type": "AWS::EC2::Route",
"Properties": {
"RouteTableId": {
"Ref": "AWSEC2RouteTableprivateustest1acomplexexamplecom"
},
"DestinationCidrBlock": "0.0.0.0/0",
"TransitGatewayId": "tgw-123456"
}
},
"AWSEC2SecurityGroupEgressfrommasterscomplexexamplecomegressall0to000000": {
"Type": "AWS::EC2::SecurityGroupEgress",
"Properties": {
Expand Down Expand Up @@ -1018,6 +1062,28 @@
]
}
},
"AWSEC2SubnetRouteTableAssociationprivateuseast1aprivatecomplexexamplecom": {
"Type": "AWS::EC2::SubnetRouteTableAssociation",
"Properties": {
"SubnetId": {
"Ref": "AWSEC2Subnetuseast1aprivatecomplexexamplecom"
},
"RouteTableId": {
"Ref": "AWSEC2RouteTableprivateustest1acomplexexamplecom"
}
}
},
"AWSEC2SubnetRouteTableAssociationuseast1autilitycomplexexamplecom": {
"Type": "AWS::EC2::SubnetRouteTableAssociation",
"Properties": {
"SubnetId": {
"Ref": "AWSEC2Subnetuseast1autilitycomplexexamplecom"
},
"RouteTableId": {
"Ref": "AWSEC2RouteTablecomplexexamplecom"
}
}
},
"AWSEC2SubnetRouteTableAssociationustest1acomplexexamplecom": {
"Type": "AWS::EC2::SubnetRouteTableAssociation",
"Properties": {
Expand All @@ -1029,6 +1095,86 @@
}
}
},
"AWSEC2Subnetuseast1aprivatecomplexexamplecom": {
"Type": "AWS::EC2::Subnet",
"Properties": {
"VpcId": {
"Ref": "AWSEC2VPCcomplexexamplecom"
},
"CidrBlock": "172.20.64.0/19",
"AvailabilityZone": "us-test-1a",
"Tags": [
{
"Key": "KubernetesCluster",
"Value": "complex.example.com"
},
{
"Key": "Name",
"Value": "us-east-1a-private.complex.example.com"
},
{
"Key": "Owner",
"Value": "John Doe"
},
{
"Key": "SubnetType",
"Value": "Private"
},
{
"Key": "foo/bar",
"Value": "fib+baz"
},
{
"Key": "kubernetes.io/cluster/complex.example.com",
"Value": "owned"
},
{
"Key": "kubernetes.io/role/internal-elb",
"Value": "1"
}
]
}
},
"AWSEC2Subnetuseast1autilitycomplexexamplecom": {
"Type": "AWS::EC2::Subnet",
"Properties": {
"VpcId": {
"Ref": "AWSEC2VPCcomplexexamplecom"
},
"CidrBlock": "172.20.96.0/19",
"AvailabilityZone": "us-test-1a",
"Tags": [
{
"Key": "KubernetesCluster",
"Value": "complex.example.com"
},
{
"Key": "Name",
"Value": "us-east-1a-utility.complex.example.com"
},
{
"Key": "Owner",
"Value": "John Doe"
},
{
"Key": "SubnetType",
"Value": "Utility"
},
{
"Key": "foo/bar",
"Value": "fib+baz"
},
{
"Key": "kubernetes.io/cluster/complex.example.com",
"Value": "owned"
},
{
"Key": "kubernetes.io/role/elb",
"Value": "1"
}
]
}
},
"AWSEC2Subnetustest1acomplexexamplecom": {
"Type": "AWS::EC2::Subnet",
"Properties": {
Expand Down
9 changes: 9 additions & 0 deletions tests/integration/update_cluster/complex/in-legacy-v1alpha2.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -71,6 +71,15 @@ spec:
name: us-test-1a
type: Public
zone: us-test-1a
- cidr: 172.20.64.0/19
name: us-east-1a-private
type: Private
zone: us-test-1a
egress: tgw-123456
- cidr: 172.20.96.0/19
name: us-east-1a-utility
type: Utility
zone: us-test-1a

---

Expand Down
9 changes: 9 additions & 0 deletions tests/integration/update_cluster/complex/in-v1alpha2.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -71,6 +71,15 @@ spec:
name: us-test-1a
type: Public
zone: us-test-1a
- cidr: 172.20.64.0/19
name: us-east-1a-private
type: Private
zone: us-test-1a
egress: tgw-123456
- cidr: 172.20.96.0/19
name: us-east-1a-utility
type: Utility
zone: us-test-1a

---

Expand Down
Loading