Improvements in managed route tables

[mtulio]

/kind feature

Describe the solution you'd like
[A clear and concise description of what you want to happen.]

Currently the route tables for managed network (VPC) are created* in the following format:

• One route table by zone for private subnets
• Plus one route table by zone when public is enabled

*ignoring the default route table

Deploying in regions with a large amount of zones, like us-east1, it will generate 12 route tables when using full zones(6) for regular availability zones.

When thinking at scale, for example with Local Zones and Wavelength zones (proposal #4874), the number of route tables can grow significantly. When managing (creating subnets) in all zones available in Local Zones (LZ) and Wavelength (WL) on us-east-1, alongside the regular Availability Zones (AZ). There are a total of 37 zones available, it will generate around 74 route tables.

$ aws ec2 describe-availability-zones --region us-east-1 --all-availability-zones --query 'AvailabilityZones[].ZoneName' | jq '. | length'
37

I would like to propose optimization for the public route tables which uses the same default route, instead of creating one public route table by zone. Additionally, for the private route tables, it would be nice to allow more than one subnet to share the same route table to cover the scenarios where the gateway is not supported in the zone, for Example Local Zones (#4874), where the egress traffic from private subnets can use NAT Gateways created in the Region.

Allowing subnets to be associated with more flexibility in the route tables, could decrease the number of tables to be re-conciliated from 74 to 8, using the example above of us-east-1:

(1[Public subnets/AZ and LZ]) + (N*zones [Private subnet AZs]) + (1[Public subnet Carrier]) = 1 + 6 + 1 = 8

Where:

• 1[Public subnets/AZ and LZ]) is the single public route table managed by CAPA, where all public subnets will be associated, except WL [Create network resources for AWS Local Zones and Wavelength Zones #4874]; The default route entry is Internet Gateway (IGW)
• N*zones [Private subnet AZs] is the total private route tables for each zone with support of NAT Gateways. In us-east-1 the is 6 zones; (considering NAT Gateways is not globally available in LZ and WL)
• 1[Public subnet Carrier] is the single public carrier route table (Create network resources for AWS Local Zones and Wavelength Zones #4874) managed by CAPA, where all public subnets in Wavelength zone will be associated.

Anything else you would like to add:
[Miscellaneous information that will assist in solving the issue.]

Environment:

• Cluster-api-provider-aws version:
• Kubernetes version: (use kubectl version):
• OS (e.g. from /etc/os-release):

[k8s-ci-robot]

This issue is currently awaiting triage.

If CAPA/CAPI contributors determines this is a relevant issue, they will accept it by applying the triage/accepted label and provide further guidance.

The triage/accepted label can be added by org members by writing /triage accepted in a comment.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.