[request]: Host / Path rewrite

[SleeperSmith]

Tell us about your request
Allow host / path rewrite
https://www.envoyproxy.io/docs/envoy/latest/api-v2/api/v2/route/route.proto#envoy-api-field-route-redirectaction-prefix-rewrite
https://www.envoyproxy.io/docs/envoy/latest/api-v2/api/v2/route/route.proto#envoy-api-field-route-routeaction-host-rewrite
https://www.envoyproxy.io/docs/envoy/latest/api-v2/api/v2/route/route.proto#envoy-api-field-route-routeaction-auto-host-rewrite

Which integration(s) is this request for?
I'm using ECS / with(out) Fargate / Service Discovery. But I don't think it matters here.

Tell us about the problem you're trying to solve. What are you trying to do, and why is it hard?
Host rewrite: We want to forward virtual service traffic website.com/feeds -> xmlfeed.com/feeds
xmlfeed.com itself only respond on the specific hostname.

Path rewrite: We want to forward, in a router:
website.com/sitemap.xml -> xmlfeed.com/feeds/sitemap

Are you currently working around this issue?
We are using nginx ingress to rewrite the rule. However, it's very clunky as it requires us to redeploy the config in nginx in tandem with the app mesh changes.

Additional context

Attachments

[casperc]

Any news on this feature?

[rajal-amzn]

At Virtual Gateway, we currently do auto rewrites where we rewrite hostname to the destination Virtual Service name and the matched path to “/”. But we must enable customers to configure rewriting hostname/path both in Virtual Node and Virtual Gateway.

[rabanian]

is there a way to disable the auto rewrite? we're counting on the host in our backend logic for several purposes. @rajal-amzn

[rajal-amzn]

@rabanian Unfortunately, the answer is no currently. We are working on a change to disable/make this configurable at the Virtual Gateway. So, please follow this thread for more updates on this.

[rabanian]

@rajal-amzn can I somehow get to the point when I can restore the original host through other headers? Maybe the ALB can add a duplicate of the host in a different name? Or in another tricky way? We're trying to find an alternative to waiting for this feature to be released and would appreciate any tip. Thanks

[rajal-amzn]

@rabanian The current behavior at Virtual Gateway doesnt add the original host to any headers. But we should be able to add it. Could you explain your usecase on how you are planning to use the original host received at the Virtual Gateway? That way, we can make sure we handle your usecase.

[rabanian]

@rajal-amzn
I came up with several use cases that are relevant to us -

1. On a regular flask app, a redirect / public asset path will use the host that the flask app sees.
2. If you're using datadog agent on k8s, it will see the internal host(virtual service), and that might be misleading unless all other services on datadog are internal.
3. We're sometimes routing based on sub-domains inside our flask app, so if it's completely over written by the gateway, it requires the app to be modified.

[rajal-amzn]

Hey all,
We are adding the following fields in Gateway Routes and Routes to support Hostname and Path rewrites.

Changes to Gateway route
We are making the following changes to Gateway Route in order to add support for rewriting hostname and path in Virtual Gateway

• Path match: This will be used to specify matching on full path in the incoming request to route to the target Virtual Service.
• Query parameters: This will be used to specify matching on query parameters in the incoming request to route to the target Virtual Service.
• Prefix rewrite: This will be used to specify rewriting only the matched prefix in the incoming request.
• Path rewrite: This will be used to specify rewriting the whole path in the incoming request.
• Auto_hostname rewrite: A boolean field used tp specify whether the original hostname need to be retained or whether the incoming hostname need be rewritten to the target Virtual Service name(current Virtual Gateway behavior).

Changes to Route

In addition, we are also adding capabilities in Route to match on additional parameters. Note that we are not adding support for rewrite behavior in Routes. Since routing rules are applied at both Virtual Gateway and at Virtual Service, enabling rewrites in both would lead to unintended rewrites happening one on top of the other. If you have a usecase where you would want to apply rewrite rules specifically at Virtual Service, please leave us feedback.

• Path match: This will be used to specify matching on full path in the incoming request to route to the target Virtual Node.
• Query parameters: This will be used to specify matching on query parameters in the incoming request to route to the target Virtual Node.

API Model
Following represents the updated API model for HTTP Gateway Routes. HTTP2 Gateway Routes would follow similar model changes.

kind: GatewayRoute
spec:
    httpRoute:
    match:
        prefix: "/" #Existing field.
        # (*NEW*) (OPTIONAL) Specifies matching on full path in the incoming request. Only one of path or prefix can be specified.
        path:
            # (*NEW*) (OPTIONAL) The incoming request must exactly match the given value.
            exact: "/exact/path"
            # (*NEW*) (OPTIONAL) The incoming request must match the given regular expression.
            regex: "regex_matching_path"
        # (*NEW*) (OPTIONAL) Specifies matching on list of query parameters in the incoming request. When multiple query parameters are specified, incoming request must match the **AND** of conditions.
        query_parameters:
               # (*NEW*) (OPTIONAL) Specifies name of query parameter to be matched.
            - name: "query_field"
               # (*NEW*) (OPTIONAL) Specifies value of query parameter to be matched. When not specified, presence of query parameter is considered a match.
               match:
                   # (*NEW*) (OPTIONAL) The value of the query parameters must exactly match the given value.
                   exact: "exact"
    action:
        # (*NEW*) (OPTIONAL) Specifies the rewrite rules of the matched request.
        rewrite:
            # (*NEW*) (OPTIONAL) Specifies the value to be rewritten on the matched prefix.
            prefix: "/rewrittenUri"
            # (*NEW*) (OPTIONAL) Specifies the rewritten value of the request path.
            path:
                # (*NEW*) (OPTIONAL) Specifies the exact path value to be rewritten.
                exact: "/rewrittenPath"
             # (*NEW*) (OPTIONAL) Specifies the rewrite rules for the hostname.
            hostname:
                # (*NEW*) (OPTIONAL) When set to `disabled`, the original hostname will not be modified. When set to `enabled`, the hostname would be rewritten to the target Virtual Service name.
                auto_hostname: "enabled"
        # Existing field
        target:
         ...

Following represents the updated API model for GRPC Gateway Routes.

kind: GatewayRoute
spec:
    grpcRoute:
    match:
        # No changes
        ....
    action:
        # (*NEW*) (OPTIONAL) Specifies the rewrite rules of the matched request.
        rewrite:
            hostname:
                # (*NEW*) (OPTIONAL) When set to `disabled`, the original hostname will not be modified. When set to `enabled`, the hostname would be rewritten to the target Virtual Service name.
                auto_hostname: "enabled"
        # Existing field
        target:
         ...

Following represents the updated API model for HTTP Routes. HTTP2 Routes would follow similar model changes.

kind: Route
spec:
    httpRoute:
    match:
        prefix: "/" #Existing field.
        # (*NEW*) (OPTIONAL) Specifies matching on full path in the incoming request. Only one of path or prefix can be specified.
        path:
            # (*NEW*) (OPTIONAL) The incoming request must exactly match the given value.
            exact: "/exact/path"
            # (*NEW*) (OPTIONAL) The incoming request must match the given regular expression.
            regex: "regex_matching_path"
        # (*NEW*) (OPTIONAL) Specifies matching on list of query parameters in the incoming request. When multiple query parameters are specified, incoming request must match the **AND** of conditions.
        query_parameters:
               # (*NEW*) (OPTIONAL) Specifies name of query parameter to be matched.
            - name: "query_field"
               # (*NEW*) (OPTIONAL) Specifies value of query parameter to be matched. When not specified, presence of query parameter is considered a match.
               match:
                   # (*NEW*) (OPTIONAL) The value of the query parameters must exactly match the given value.
                   exact: "exact"
    action:
         # No changes
         ...

No changes to the GRPC route.

Examples

Following example shows how to disable hostname rewrites in Virtual Gateway

kind: GatewayRoute
metadata:
 name: SampleGatewayRoute
 ...
spec:
  httpRoute:
    action:
      rewrite:
        hostname:
          auto_hostname: "disabled"
      target:
        ...

Following example shows how to change the request from example.com/location.php to example.com/contact/location.php

---
kind: GatewayRoute
metadata:
 name: SampleGatewayRoute
 ...
spec:
  httpRoute:
    match:
      path:
        exact: "/"
    action:
      rewrite:
        prefix: "/location/" #Add prefix
      target:
        ...

Following example shows how to change the request from example.com/pay/credit-card to example.com/credit-card

---
kind: GatewayRoute
metadata:
 name: SampleGatewayRoute
 ...
spec:
  httpRoute:
    match:
      path:
        exact: "/pay/"
    action:
      rewrite:
        prefix: "/" #Remove prefix
      target:
        ...

Following example shows how to change the request from example.com/instruments/violin to example.com/string-instruments/violin

---
kind: GatewayRoute
metadata:
 name: SampleGatewayRoute
 ...
spec:
  httpRoute:
    match:
      path:
        exact: "/instruments/violin/"
    action:
      rewrite:
        prefix: "/string-instruments/violin/" #Edit prefix
      target:
        ...

Following example shows how to change the request from example.com/index.php?tool=hammer to example.com/tools/hammer

---
kind: GatewayRoute
metadata:
 name: SampleGatewayRoute
 ...
spec:
  httpRoute:
    match:
      path:
        exact: "/index.php"
      query_parameters:
        - name: "tools"
          match:
            exact: "hammer"
    action:
      rewrite:
        path:
          exact: "/tools/hammer" #Edit path based on query parameters
      target:
        ...

We hope these changes fit your use cases. We'll be updating this issue once we have the feature enabled in our Preview Channel. Until then, we'd love to hear your feedback on this proposal in the comments.

[eddgrant]

Hi folks,

We're currently blocked by lack of rewrite capability whilst ingressing traffic in to the mesh. We'd like something which would use regexp capture groups to do rewrites. Our use case is as follows:

1. Traffic, from API Gateway enters the VPC, hitting an NLB. The requests are in the form:

   https://<microservice>.<environment>.example.com/

2. I then need to proxy the traffic to endpoints in the mesh, which live on a different endpoint format. I would like to use the following regular expression to capture the name in the incoming URL:

   ^https:\/\/(.*)\.\w*\.example\.com\/.*$

3. Having used a capture group to obtain the name in the incoming URL I would then like to proxy the request to the following endpoint, which is implemented as a Virtual Service in the mesh:

   https://<microservice>.some.internal.endpoint/

Having read through the proposal comment above I'm not 100% clear on whether my use case would be supported here. Would I need to use a prefix match? Do they support regular expressions? And are any capture group results in the action rewrite stages?

Most grateful if someone could shed some light on this 🙏 .

[rajal-amzn]

@eddgrant The above proposal would not allow supporting capture groups for hostname and use that in rewrite. Would like to understand whether the final hostname in this case https://<microservice>.some.internal.endpoint/ would be the same as the target Virtual Service from Virtual Gateway or will that be different? If it would be different, could you explain your usecase?

[rishijatia]

Hi everyone,

Host / Path Rewriting support is now available in the App Mesh Preview Channel.

Feel free to try out the example from our examples repository: howto-match-and-rewrite-at-ingress.

[rajal-amzn]

Hey all,
Thanks to everyone for your feedback. Routing enhancements are now generally available. Check out this blog post for more details: https://aws.amazon.com/about-aws/whats-new/2021/06/aws-app-mesh-introduces-enhanced-ingress-traffic-management-capabilities/

NOTE: Cloudformation support for this is yet to be launched. We will update this issue once it is done.

[rajal-amzn]

Cloudformation support are live now. Resolving this issue now.