Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

aws_route: Add support for "local" route target #21350

Closed
cmceniry opened this issue Oct 18, 2021 · 14 comments · Fixed by #32794
Closed

aws_route: Add support for "local" route target #21350

cmceniry opened this issue Oct 18, 2021 · 14 comments · Fixed by #32794
Assignees
Labels
enhancement Requests to existing resources that expand the functionality or scope. prioritized Part of the maintainer teams immediate focus. To be addressed within the current quarter. service/ec2 Issues and PRs that pertain to the ec2 service.
Milestone

Comments

@cmceniry
Copy link

cmceniry commented Oct 18, 2021

Community Note

  • Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
  • Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request
  • If you are interested in working on this issue or have submitted a pull request, please leave a comment

Description

AWS started to allow you to change the routes for the VPC associated CIDRs[1]. Given the history, looks like tf allows you to import the automatically created local routes (that part feels like a separate bigger feature request) and resourceRouteUpdate works to point those elsewhere, but trying to flip back to the local route doesn't work.

(And just to be explicit, not Outpost related since "local" is used for both.)

Use case - being able to conditionally route to middleware boxes. Flip on and off via a flag to support rollouts and in the event of issues.

New or Affected Resource(s)

  • aws_route
  • aws_route_table

Potential Terraform Configuration

resource "aws_route" "associated_cidr" {
    route_table_id = ...
    destination_cidr_block = data.my_cidr....
    vpc_endpoint_id = var.enable_firewall ? var.firewall_vpce : null
    local = var.enabled_firewall
}

I'll admit that the syntax here isn't very parallel to the other target options. local doesn't have parameters, just a flag (just a boolean [2] - input.LocalTarget = aws.Bool(true)). But not really seeing any other good suggestion on it.

References

  1. I think this is the first reference to it, but definitely "new but not too new" https://aws.amazon.com/blogs/networking-and-content-delivery/deployment-models-for-aws-network-firewall/
  2. https://docs.aws.amazon.com/sdk-for-go/api/service/ec2/#ReplaceRouteInput
  3. Erronious error when applying aws_route when destination_cidr_block == default route #13896
@cmceniry cmceniry added the enhancement Requests to existing resources that expand the functionality or scope. label Oct 18, 2021
@github-actions github-actions bot added needs-triage Waiting for first response or review from a maintainer. service/ec2 Issues and PRs that pertain to the ec2 service. labels Oct 18, 2021
@ewbankkit ewbankkit removed the needs-triage Waiting for first response or review from a maintainer. label Oct 19, 2021
@paullemon
Copy link

paullemon commented Nov 22, 2021

The local route was previously non-configurable. Would it make sense to call it something different than "aws_route" because of this? Its destination was previously statically set to "Local". In Cisco terms it was a summary address of the full CIDR range from the VPC. Later as they allowed secondary CIDR ranges in the VPC's, it could be multiple summary CIDR blocks for each of those ranges. Would it make sense to call it something different like "aws_summary_route" or "aws_local_route" so its not confused with its completely different "aws_route" counterpart? Just a thought.

Additionally if you look at these settings in the GUI it allows other destination values besides the network firewall endpoint objects. Example ENI, VPCE (Both gateway load balancer and network firewall), Local etc.

There is also a way to configure routes inside of "aws_route_table". Could it also have a configuration block here too?

@ephenix
Copy link

ephenix commented Dec 29, 2021

AWS now supports configuring the local route to either an instance, network interface, or gateway load balancer endpoint.

This should be supported feature of the aws_route resource, as well as inline routes in the aws_route_table resource.

A supported network topology using AWS Network Firewall cannot be implemented using Terraform for inter-subnet packet inspection in a single VPC.

Rather than add a new resource, simply add a "replace_existing" flag, which defaults to false (existing functionality).

Changing this flag to true would override the default error which fails if a route exists, and instead calls the replace-route api command.

@cmceniry
Copy link
Author

cmceniry commented Jan 3, 2022

This should be supported feature of the aws_route resource, as well as inline routes in the aws_route_table resource.

👍 This shouldn't be any different than other route definitions. The CIDR routes are special cases only because they must exist and so are automatically created with the VPC/CIDR block addition (for now? Iirc, CFN has the same problem so not sure what will change to allow for a better experience there). From a declarative standpoint, they probably shouldn't be automatically created, or, when they are and that's a know behavior, the system should allow for this automagically. But I concede that that last part is a bit naive thinking.

Rather than add a new resource, simply add a "replace_existing" flag, which defaults to false (existing functionality).

I'm good with this as a compromise (from my previous "naive" comment). What happens if the route doesn't exist? Will it work or fail because it was expecting the route to exist?

@cmceniry
Copy link
Author

cmceniry commented Jan 3, 2022

What happens if the route doesn't exist? Will it work or fail because it was expecting the route to exist?

Looking at #22366, I see that it'll try to create and if that fails and replace=true, it'll then try to modify it. I'm good with that behavior.

@paullemon
Copy link

Any update on getting this implemented? This causes issues with both AWS Network Firewall and AWS Gateway Load Balancer routing.

@rhanielborromeo
Copy link

+1 on the earlier comment that this would be better to be part of the aws_route resource in the aws_route_table resource. That will simplify the update and it makes it more intuitive to create.

@ddericco
Copy link
Contributor

If no one is working on this, I'll take a look and see what's required here.

@deutmeyerbrianpfg
Copy link

I'm also looking for the ability to update the "local" routes that come with a route table to point to a endpoint instead of being local. Ideally, we'd be able to supply the vpc_endpoint_id and that route will update. The input flag @ephenix mentioned is a good idea too.

@sgraup
Copy link

sgraup commented Sep 27, 2022

I'm currently running a lambda invocation to update the local route after table creation with the replace_route API. I would like to point local routes at a vpc_endpoint via terraform without this hack/workaround.

@bartpmika
Copy link

+1 - I really need this solved somehow in AWS. Now I know why Azure invalidates the default routes when an equal static route is added instead of replacing them. Way easier on IaC than what AWS does

@imcdo
Copy link

imcdo commented Jun 22, 2023

+1, would really help with firewall support.

@YakDriver YakDriver self-assigned this Jul 19, 2023
@breathingdust breathingdust added the prioritized Part of the maintainer teams immediate focus. To be addressed within the current quarter. label Jul 21, 2023
@YakDriver
Copy link
Member

YakDriver commented Aug 2, 2023

We added support in aws_route for this previously.

#32794 will add support in aws_route_table. I was able to make it work without an additional flag such as replace_existing by making the assumption that if the route is described in the config, the practitioner wants to do what the config describes. In other words, default routes that have no basis in config, continue to be mostly ignored as previously. Now, however, if you have a local route in config, such as a result of importing or modifying a route, you can then update the route target. Let us know what you think about this approach in the comments below.

Our tests for this functionality, while thorough, are unlikely to cover every scenario for this very commonly used resource and routes. Please let us know if the enhancement in #32794 works for your situation.

@github-actions github-actions bot added this to the v5.11.0 milestone Aug 3, 2023
@github-actions
Copy link

github-actions bot commented Aug 3, 2023

This functionality has been released in v5.11.0 of the Terraform AWS Provider. Please see the Terraform documentation on provider versioning or reach out if you need any assistance upgrading.

For further feature requests or bug reports with this functionality, please create a new GitHub issue following the template. Thank you!

@github-actions
Copy link

github-actions bot commented Sep 3, 2023

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.

@github-actions github-actions bot locked as resolved and limited conversation to collaborators Sep 3, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
enhancement Requests to existing resources that expand the functionality or scope. prioritized Part of the maintainer teams immediate focus. To be addressed within the current quarter. service/ec2 Issues and PRs that pertain to the ec2 service.
Projects
None yet