Add initial support for configuring IPv6 with AWS #11442

hakman · 2021-05-09T19:25:44Z

Inspired by #11163.

At the moment, the IPv6 feature configures the following AWS components:

VPC:
- always requests the IPv6 CIDR from AWS
- cannot be disabled
Routes:
- always creates the default IPv6 route (for public topologies)
Security Group:
- always allows IPv6 egress
- adds / removes entries based on spec.kubernetesApiAccess and spec.sshAccess
Subnets:
- users can assign / remove an IPV6 CIDR (but cannot change it)
Launch Templates:
- IPv6 addresses are assigned to instances based on subnet setting, which allows marking instances as needing update

TODO:

docs, docs, docs, ... 😄
allow configuring Kubernetes IPv4/IPv6 dual-stack:
- https://kubernetes.io/docs/concepts/services-networking/dual-stack
set subnet CIDR based on VPC CIDR on cluster creation:
- https://www.terraform.io/docs/language/functions/cidrsubnet.html
- https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/intrinsic-function-reference-cidr.html
allow setting API NLB to dual-stack
create Egress-only Internet Gateway for private topologies
handle path MTU discovery
- https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/network_mtu.html#path_mtu_discovery
- AWS doesn't allow setting type and code for ICMPv6, which may be an issue

Ref: #8432

/cc @rifelpet @johngmyers @olemarkus

hakman · 2021-05-09T19:59:04Z

/retest

johngmyers

Things are changing out from under my review, so submitting this early. Will review more later.

cloudmock/aws/mockec2/vpcs.go

pkg/apis/kops/cluster.go

pkg/apis/kops/validation/validation.go

upup/pkg/fi/cloudup/awstasks/route.go

upup/pkg/fi/cloudup/awstasks/subnet.go

pkg/apis/kops/validation/validation.go

upup/pkg/fi/cloudup/awstasks/vpc.go

tests/integration/update_cluster/aws-lb-controller/kubernetes.tf

hakman · 2021-05-13T10:22:31Z

upup/pkg/fi/cloudup/awstasks/subnet_test.go

+			Name:     s("subnet1"),
+			VPC:      vpc1,
+			CIDR:     s("172.20.1.0/24"),
+			IPv6CIDR: s("cidrsubnet-8-1"),


@johngmyers @justinsb This is my proposed way of describing a subnet CIDR, using the network CIDR as a base. In this case, it is the equivalent of "2001:db8:0:1::/64.

Ref:

https://www.terraform.io/docs/language/functions/cidrsubnet.html

https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/intrinsic-function-reference-cidr.html

I also don't mind to split this in a separate PR and continue the discussion there.

I'd prefer bikeshedding a syntax that looks more like a function call. Perhaps something like cidrsubnet(8,1).

I think a separate PR would help as the rest is a bit more baked.

OK, separate PR it is. :)

upup/pkg/fi/utils/cidr.go

pkg/apis/kops/validation/validation.go

pkg/model/awsmodel/api_loadbalancer.go

pkg/model/awsmodel/autoscalinggroup.go

pkg/model/awsmodel/bastion.go

hakman · 2021-05-18T06:06:14Z

I will squash / re-stage some of the commits after addressing all review comments and getting the OK.
/hold

pkg/apis/kops/validation/validation_test.go

pkg/model/awsmodel/api_loadbalancer.go

tests/integration/update_cluster/complex/in-v1alpha2.yaml

upup/pkg/fi/cloudup/awstasks/subnet.go

upup/pkg/fi/cloudup/awstasks/vpc.go

upup/pkg/fi/cloudup/awstasks/vpcamazonipv6cidrblock.go

johngmyers · 2021-05-19T03:00:06Z

/lgtm

hakman · 2021-05-19T03:24:22Z

Thanks for the review @johngmyers. I restaged the commits to include the review feedback.
/hold cancel

k8s-ci-robot · 2021-05-19T03:32:42Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: johngmyers

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [johngmyers]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

k8s-ci-robot requested review from johngmyers, olemarkus and rifelpet May 9, 2021 19:25

k8s-ci-robot added cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. area/api area/provider/aws Issues or PRs related to aws provider labels May 9, 2021

hakman force-pushed the ipv6 branch 3 times, most recently from e943bc0 to ddfabbb Compare May 9, 2021 19:33

hakman force-pushed the ipv6 branch from ddfabbb to ea43146 Compare May 10, 2021 04:57

johngmyers requested changes May 10, 2021

View reviewed changes

k8s-ci-robot assigned johngmyers May 10, 2021

hakman force-pushed the ipv6 branch 4 times, most recently from 4894563 to 7bd4fbf Compare May 10, 2021 05:41

hakman mentioned this pull request May 10, 2021

[WIP] Adding IPV6 support for aws vpc and subnet #11163

Closed

rifelpet reviewed May 10, 2021

View reviewed changes

tests/integration/update_cluster/aws-lb-controller/kubernetes.tf Outdated Show resolved Hide resolved

hakman force-pushed the ipv6 branch 2 times, most recently from d9ea7f3 to fdfc533 Compare May 11, 2021 07:02

k8s-ci-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label May 11, 2021

hakman force-pushed the ipv6 branch from fdfc533 to c1d3249 Compare May 13, 2021 09:30

k8s-ci-robot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label May 13, 2021

hakman commented May 13, 2021

View reviewed changes

k8s-ci-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label May 14, 2021

hakman force-pushed the ipv6 branch from c1d3249 to 0e89f56 Compare May 14, 2021 04:20

k8s-ci-robot added needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. and removed needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. labels May 14, 2021

hakman force-pushed the ipv6 branch 2 times, most recently from 2733c93 to d768102 Compare May 17, 2021 06:06

hakman changed the title ~~[WIP] Add initial support for configuring IPv6 with AWS~~ Add initial support for configuring IPv6 with AWS May 17, 2021

k8s-ci-robot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label May 17, 2021

hakman requested review from rifelpet and johngmyers May 17, 2021 07:15

johngmyers requested changes May 17, 2021

View reviewed changes

Pre-add AWS IPv6 integration test

c08d0e2

hakman force-pushed the ipv6 branch from d768102 to 53ad8a6 Compare May 18, 2021 05:56

k8s-ci-robot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label May 18, 2021

hakman force-pushed the ipv6 branch from 53ad8a6 to 64acdca Compare May 18, 2021 14:54

johngmyers reviewed May 18, 2021

View reviewed changes

k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label May 19, 2021

hakman added 2 commits May 19, 2021 06:21

Add initial support for configuring IPv6 with AWS

cedbe1f

Run hack/update-expected.sh

57feaa6

hakman force-pushed the ipv6 branch from 1f167eb to 57feaa6 Compare May 19, 2021 03:21

k8s-ci-robot removed the lgtm "Looks good to me", indicates that a PR is ready to be merged. label May 19, 2021

k8s-ci-robot removed the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label May 19, 2021

johngmyers approved these changes May 19, 2021

View reviewed changes

k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label May 19, 2021

k8s-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label May 19, 2021

k8s-ci-robot merged commit fe7d6e5 into kubernetes:master May 19, 2021

k8s-ci-robot added this to the v1.22 milestone May 19, 2021

This was referenced May 19, 2021

IPv6 support #8432

Closed

Calculate IPv6 subnet CIDR based on cluster CIDR #11523

Merged

hakman deleted the ipv6 branch September 7, 2021 18:53

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add initial support for configuring IPv6 with AWS #11442

Add initial support for configuring IPv6 with AWS #11442

hakman commented May 9, 2021 •

edited

Loading

hakman commented May 9, 2021

johngmyers left a comment

hakman May 13, 2021 •

edited

Loading

hakman May 13, 2021 •

edited

Loading

hakman May 13, 2021

johngmyers May 13, 2021

hakman May 14, 2021

hakman commented May 18, 2021

johngmyers commented May 19, 2021

hakman commented May 19, 2021

k8s-ci-robot commented May 19, 2021

Add initial support for configuring IPv6 with AWS #11442

Add initial support for configuring IPv6 with AWS #11442

Conversation

hakman commented May 9, 2021 • edited Loading

hakman commented May 9, 2021

johngmyers left a comment

Choose a reason for hiding this comment

hakman May 13, 2021 • edited Loading

Choose a reason for hiding this comment

hakman May 13, 2021 • edited Loading

Choose a reason for hiding this comment

hakman May 13, 2021

Choose a reason for hiding this comment

johngmyers May 13, 2021

Choose a reason for hiding this comment

hakman May 14, 2021

Choose a reason for hiding this comment

hakman commented May 18, 2021

johngmyers commented May 19, 2021

hakman commented May 19, 2021

k8s-ci-robot commented May 19, 2021

hakman commented May 9, 2021 •

edited

Loading

hakman May 13, 2021 •

edited

Loading

hakman May 13, 2021 •

edited

Loading