EnvoyFilter RateLimit pattern regex not working , need help!

[wmashal]

Hi

I am trying to configure my Global Rate limit in istio using the EnvoyFilter resource to handle a a dynamic URL requests rate limit, my url contains a dynamic UUID in the path so I tried to use regex_value_rewrite pattern but its not working as expected, I expect to have a request count for each unique UUID, and not apply the rate limit on any other paths just if I have the UUID Regex

Example:

lets say we have limit 50 req/minute

/cloudautomation/v1/solutions//spfi/notify called 2 times --- > count1 is 2 out of 50
/cloudautomation/v1/solutions//spfi/notify called 10 times ---> count2 is 10 out of 50

My Configurations

apiVersion: networking.istio.io/v1alpha3
kind: EnvoyFilter
metadata:
  name: ratelimit-envoy-filter
  namespace: istio-system
spec:
  workloadSelector:
    labels:
      istio: ingressgateway
  configPatches:
    - applyTo: HTTP_FILTER
      match:
        context: GATEWAY
        listener:
          filterChain:
            filter:
              name: 'envoy.filters.network.http_connection_manager'
              subFilter:
                name: 'envoy.filters.http.router'
      patch:
        operation: INSERT_BEFORE
        value:
          name: envoy.filters.http.header_to_metadata
          typed_config:
            "@type": type.googleapis.com/envoy.extensions.filters.http.header_to_metadata.v3.Config
            request_rules:
            - header: ':path'
              on_header_present:
                # use an arbitary name for the namespace
                # will be used later to extract descriptor value
                metadata_namespace: spfi
                # use an arbitary key for the metadata
                # will be used later to extract descriptor value
                key: uri
                regex_value_rewrite:
                  pattern:
                    # regex matcher
                    google_re2: {}
                    # truncates parameters from path
                    regex: '^\/cloudautomation\/v1\/solutions\/(.+)\/spfi\/notify$'
                  substitution: '\1'
    - applyTo: HTTP_FILTER
      match:
        context: GATEWAY
        listener:
          filterChain:
            filter:
              name: "envoy.filters.network.http_connection_manager"
              subFilter:
                name: "envoy.filters.http.router"
      patch:
        operation: INSERT_BEFORE
        value:
          name: envoy.filters.http.ratelimit
          typed_config:
            "@type": type.googleapis.com/envoy.extensions.filters.http.ratelimit.v3.RateLimit
            domain: atom-istio-ratelimit
            failure_mode_deny: false
            timeout: 10s
            rate_limit_service:
              grpc_service:
                envoy_grpc:
                  cluster_name: outbound|8081||ratelimit.istio-system.svc.cluster.local
                  authority: ratelimit.istio-system.svc.cluster.local
              transport_api_version: V3

apiVersion: networking.istio.io/v1alpha3
kind: EnvoyFilter
metadata:
  name: uca-spfi-ratelimit-actions
  namespace: istio-system
spec:
  workloadSelector:
    labels:
      istio: ingressgateway
  configPatches:
  - applyTo: VIRTUAL_HOST
    match:
      context: GATEWAY
      routeConfiguration:
        vhost:
          name: {{ .Values.uca.ingress.appPrefix }}.ingress.{{ .Values.domain }}:443
          route:
            action: ANY
    patch:
      operation: MERGE
      value:
        rate_limits:
        - actions:
          - metadata:
              descriptor_key: SPFI
              metadata_key:
                key: spfi
                path:
                - key: uri

In the rate limit service config I configured

   - key: SPFI
     rate_limit:
       unit: MINUTE
       requests_per_unit: 50

Please Advice ?
Thanks

[ravenblackx]

I think substitution: '\1' is telling it to replace the pattern with itself, so making no change. For what you describe it looks like substitution: "" would do what you intended?
Or maybe substitution: "/cloudautomation/v1/solutions//spfi/notify"

[ravenblackx]

Oh, no, I misunderstood, you're aiming to do just the UUID, not to ignore the UUID, so your configuration looks right.

[ravenblackx]

@mattklein123 as ratelimit primary owner.

[wmashal]

@mattklein123 can help plz ?

[wmashal]

@ravenblackx u think its a bug ? or no support for such a scenario ?

[ravenblackx]

Not really my area, that's why I tried to ping Matt to look. Maybe @wbpcode might know more, as someone touching rate limit related stuff recently?

[wbpcode]

so, what is the current behavior? Seems the configuration is fine. It would be better to log the dynamic metadata or check the stats of rate limit server to find what actually happened.