PerRouteConfig of JWT with requirement_map can't clear cache

[YvesZHI]

I'm trying to make Envoy route requests with JWT. The payload of JWT contains a pair of KV: "role": "user" or "role": "grayUser".

The route rule is

if role == user
    goto endpointA
else if role == grayUser
    goto endpointB

Here is my config of Envoy:

static_resources:
  listeners:
  - name: listener_0
    address:
      socket_address:
        address: 0.0.0.0
        port_value: 18000
    filter_chains:
    - filters:
      - name: envoy.filters.network.http_connection_manager
        typed_config:
          "@type": type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager
          stat_prefix: ingress_http
          http_filters:
          - name: envoy.filters.http.jwt_authn
            typed_config:
              "@type": type.googleapis.com/envoy.extensions.filters.http.jwt_authn.v3.JwtAuthentication
              providers:
                sso_jwt_provider:
                  local_jwks:
                    filename: /conf/envoy/jwks.json
                  from_headers:
                  - name: X-Authorization
                  payload_in_metadata: jwt_payload
                  clear_route_cache: true    ######## NOT working with PerRouteConfig of JWT???
              rules:
              - match:
                  prefix: /
                requires:
                  provider_name: sso_jwt_provider
#              requirement_map:
#                general: {provider_name: sso_jwt_provider}
          - name: envoy.filters.http.router
            typed_config:
              "@type": type.googleapis.com/envoy.extensions.filters.http.router.v3.Router
          route_config:
            name: local_route
            virtual_hosts:
            - name: exception_service
              domains: ["*"]
              routes:
              - match:
                  prefix: "/"
                  dynamic_metadata:
                  - filter: envoy.filters.http.jwt_authn
                    path:
                    - key: jwt_payload
                    - key: role
                    value:
                      string_match:
                        exact: "loginUser"
                route:
                  cluster: service_addmachine
                  timeout: 5s
                  metadata_match:
                    filter_metadata:
                      envoy.lb:
                        canary: 0
#                typed_per_filter_config:
#                  envoy.filters.http.jwt_authn:
#                    "@type": type.googleapis.com/envoy.extensions.filters.http.jwt_authn.v3.PerRouteConfig
#                    "requirement_name": general
              - match:
                  prefix: "/"
                  dynamic_metadata:
                  - filter: envoy.filters.http.jwt_authn
                    path:
                    - key: jwt_payload
                    - key: role
                    value:
                      string_match:
                        exact: "grayLoginUser"
                route:
                  cluster: service_addmachine
                  timeout: 5s
                  metadata_match:
                    filter_metadata:
                      envoy.lb:
                        canary: 1
#                typed_per_filter_config:
#                  envoy.filters.http.jwt_authn:
#                    "@type": type.googleapis.com/envoy.extensions.filters.http.jwt_authn.v3.PerRouteConfig
#                    "requirement_name": general
  clusters:
  - name: service_addmachine
    connect_timeout: 30s
    type: STATIC
    lb_policy: ROUND_ROBIN
    load_assignment:
      cluster_name: service_addmachine
      endpoints:
      - lb_endpoints:
        - endpoint:
            address:
              socket_address:
                address: 127.0.0.1
                port_value: 20000
          metadata:
            filter_metadata:
              envoy.lb:
                canary: 0
        - endpoint:
            address:
              socket_address:
                address: 127.0.0.1
                port_value: 20001
          metadata:
            filter_metadata:
              envoy.lb:
                canary: 1

It works as expected. I could test it with the command curl -H 'X-Authorization: xxx' 'http://127.0.0.1:18000'.

However, if I use the per-route config of JWT. I will get the NR error, whch means "no route found".

    filter_chains:
    - filters:
      - name: envoy.filters.network.http_connection_manager
        typed_config:
          "@type": type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager
          stat_prefix: ingress_http
          http_filters:
          - name: envoy.filters.http.jwt_authn
            typed_config:
              "@type": type.googleapis.com/envoy.extensions.filters.http.jwt_authn.v3.JwtAuthentication
              providers:
                sso_jwt_provider:
                  local_jwks:
                    filename: /conf/envoy/jwks.json
                  from_headers:
                  - name: X-Authorization
                  payload_in_metadata: jwt_payload
                  clear_route_cache: true
#            rules:
#            - match:
#                 prefix: /
#               requires:
#                  provider_name: sso_jwt_provider
              requirement_map:
                general: {provider_name: sso_jwt_provider}
          - name: envoy.filters.http.router
            typed_config:
              "@type": type.googleapis.com/envoy.extensions.filters.http.router.v3.Router
          route_config:
            name: local_route
            virtual_hosts:
            - name: exception_service
              domains: ["*"]
              routes:
              - match:
                  prefix: "/"
                  dynamic_metadata:
                  - filter: envoy.filters.http.jwt_authn
                    path:
                    - key: jwt_payload
                    - key: role
                    value:
                      string_match:
                        exact: "loginUser"
                route:
                  cluster: service_addmachine
                  timeout: 5s
                  metadata_match:
                    filter_metadata:
                      envoy.lb:
                        canary: 0
                typed_per_filter_config:
                  envoy.filters.http.jwt_authn:
                    "@type": type.googleapis.com/envoy.extensions.filters.http.jwt_authn.v3.PerRouteConfig
                    "requirement_name": general
              - match:
                  prefix: "/"
                  dynamic_metadata:
                  - filter: envoy.filters.http.jwt_authn
                    path:
                    - key: jwt_payload
                    - key: role
                    value:
                      string_match:
                        exact: "grayLoginUser"
                route:
                  cluster: service_addmachine
                  timeout: 5s
                  metadata_match:
                    filter_metadata:
                      envoy.lb:
                        canary: 1
                typed_per_filter_config:
                  envoy.filters.http.jwt_authn:
                    "@type": type.googleapis.com/envoy.extensions.filters.http.jwt_authn.v3.PerRouteConfig
                    "requirement_name": general

As you see, I comment the rules part under JWT and decomment the requirement_map and all of typed_per_filter_config under the route_config

After reading this issue: #19910, I'm thinking if this is a bug about clear_route_cache?

[ravenblackx]

@TAOXUY ?