aws_route: Add support for "local" route target

[cmceniry]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Description

AWS started to allow you to change the routes for the VPC associated CIDRs[1]. Given the history, looks like tf allows you to import the automatically created local routes (that part feels like a separate bigger feature request) and resourceRouteUpdate works to point those elsewhere, but trying to flip back to the local route doesn't work.

(And just to be explicit, not Outpost related since "local" is used for both.)

Use case - being able to conditionally route to middleware boxes. Flip on and off via a flag to support rollouts and in the event of issues.

New or Affected Resource(s)

• aws_route
• aws_route_table

Potential Terraform Configuration

resource "aws_route" "associated_cidr" {
    route_table_id = ...
    destination_cidr_block = data.my_cidr....
    vpc_endpoint_id = var.enable_firewall ? var.firewall_vpce : null
    local = var.enabled_firewall
}

I'll admit that the syntax here isn't very parallel to the other target options. local doesn't have parameters, just a flag (just a boolean [2] - input.LocalTarget = aws.Bool(true)). But not really seeing any other good suggestion on it.

References

1. I think this is the first reference to it, but definitely "new but not too new" https://aws.amazon.com/blogs/networking-and-content-delivery/deployment-models-for-aws-network-firewall/
2. https://docs.aws.amazon.com/sdk-for-go/api/service/ec2/#ReplaceRouteInput
3. Erronious error when applying aws_route when destination_cidr_block == default route #13896

[paullemon]

The local route was previously non-configurable. Would it make sense to call it something different than "aws_route" because of this? Its destination was previously statically set to "Local". In Cisco terms it was a summary address of the full CIDR range from the VPC. Later as they allowed secondary CIDR ranges in the VPC's, it could be multiple summary CIDR blocks for each of those ranges. Would it make sense to call it something different like "aws_summary_route" or "aws_local_route" so its not confused with its completely different "aws_route" counterpart? Just a thought.

Additionally if you look at these settings in the GUI it allows other destination values besides the network firewall endpoint objects. Example ENI, VPCE (Both gateway load balancer and network firewall), Local etc.

There is also a way to configure routes inside of "aws_route_table". Could it also have a configuration block here too?

[ephenix]

AWS now supports configuring the local route to either an instance, network interface, or gateway load balancer endpoint.

This should be supported feature of the aws_route resource, as well as inline routes in the aws_route_table resource.

A supported network topology using AWS Network Firewall cannot be implemented using Terraform for inter-subnet packet inspection in a single VPC.

Rather than add a new resource, simply add a "replace_existing" flag, which defaults to false (existing functionality).

Changing this flag to true would override the default error which fails if a route exists, and instead calls the replace-route api command.

[cmceniry]

This should be supported feature of the aws_route resource, as well as inline routes in the aws_route_table resource.

👍 This shouldn't be any different than other route definitions. The CIDR routes are special cases only because they must exist and so are automatically created with the VPC/CIDR block addition (for now? Iirc, CFN has the same problem so not sure what will change to allow for a better experience there). From a declarative standpoint, they probably shouldn't be automatically created, or, when they are and that's a know behavior, the system should allow for this automagically. But I concede that that last part is a bit naive thinking.

Rather than add a new resource, simply add a "replace_existing" flag, which defaults to false (existing functionality).

I'm good with this as a compromise (from my previous "naive" comment). What happens if the route doesn't exist? Will it work or fail because it was expecting the route to exist?

[cmceniry]

What happens if the route doesn't exist? Will it work or fail because it was expecting the route to exist?

Looking at #22366, I see that it'll try to create and if that fails and replace=true, it'll then try to modify it. I'm good with that behavior.

[paullemon]

Any update on getting this implemented? This causes issues with both AWS Network Firewall and AWS Gateway Load Balancer routing.

[rhanielborromeo]

+1 on the earlier comment that this would be better to be part of the aws_route resource in the aws_route_table resource. That will simplify the update and it makes it more intuitive to create.

[ddericco]

If no one is working on this, I'll take a look and see what's required here.

[deutmeyerbrianpfg]

I'm also looking for the ability to update the "local" routes that come with a route table to point to a endpoint instead of being local. Ideally, we'd be able to supply the vpc_endpoint_id and that route will update. The input flag @ephenix mentioned is a good idea too.

[sgraup]

I'm currently running a lambda invocation to update the local route after table creation with the replace_route API. I would like to point local routes at a vpc_endpoint via terraform without this hack/workaround.

[bartpmika]

+1 - I really need this solved somehow in AWS. Now I know why Azure invalidates the default routes when an equal static route is added instead of replacing them. Way easier on IaC than what AWS does

[imcdo]

+1, would really help with firewall support.

[YakDriver]

We added support in aws_route for this previously.

#32794 will add support in aws_route_table. I was able to make it work without an additional flag such as replace_existing by making the assumption that if the route is described in the config, the practitioner wants to do what the config describes. In other words, default routes that have no basis in config, continue to be mostly ignored as previously. Now, however, if you have a local route in config, such as a result of importing or modifying a route, you can then update the route target. Let us know what you think about this approach in the comments below.

Our tests for this functionality, while thorough, are unlikely to cover every scenario for this very commonly used resource and routes. Please let us know if the enhancement in #32794 works for your situation.

[github-actions]

This functionality has been released in v5.11.0 of the Terraform AWS Provider. Please see the Terraform documentation on provider versioning or reach out if you need any assistance upgrading.

For further feature requests or bug reports with this functionality, please create a new GitHub issue following the template. Thank you!

[github-actions]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.