[Feature] Support for X-Forwarded-Port in AKS Application Routing Add-on

[torumakabe]

Certain applications such as Spring Boot, when running behind a gateway require the HTTP header X-Forwarded-Port. However, the current AKS Application Routing Add-on does not appear to set X-Forwarded-Port.

Below is the result of running httpbin, which returns the request’s HTTP headers behind the Application Routing Add-on. The X-Forwarded-Port header is not set.

The IP addresses in the text have been replaced with dummy ones.

curl http://203.0.113.1/headers
{
  "headers": {
    "Accept": "*/*", 
    "Host": "203.0.113.1", 
    "User-Agent": "curl/8.7.1", 
    "X-Forwarded-Host": "203.0.113.1", 
    "X-Forwarded-Scheme": "http", 
    "X-Scheme": "http"
  }
}

So I tried configuring it with a snippet of ingress. I also applied the following annotations to test whether custom headers can be used or variables can be read.

  annotations:
    nginx.ingress.kubernetes.io/configuration-snippet: |
      proxy_set_header X-Forwarded-Port "80";
      proxy_set_header X-Test-Pass-Port $pass_port;
      proxy_set_header X-Test-Server-Port $server_port;
      proxy_set_header X-Test-Service-Port $service_port;

However, it seems that X-Forwarded-Port is treated specially and isn’t being set.

curl http://203.0.113.1/headers
{
  "headers": {
    "Accept": "*/*", 
    "Host": "203.0.113.1", 
    "User-Agent": "curl/8.7.1", 
    "X-Forwarded-Host": "203.0.113.1", 
    "X-Forwarded-Scheme": "http", 
    "X-Scheme": "http", 
    "X-Test-Pass-Port": "8080", 
    "X-Test-Server-Port": "8080", 
    "X-Test-Service-Port": "80"
  }
}

The documentation’s list of limitations does not explicitly mention X-Forwarded-Port.

I would appreciate it if you could ensure this header is set for applications that require X-Forwarded-Port. You don’t need to configure it by default, support via an Ingress configuration snippet is sufficient. Thanks!

[jaiveerk]

Hey @torumakabe -

We can look into adding this in the future. In the meantime, feel free to add this setting to your nginx configmap that app routing creates. While this behavior isn't technically supported by the add-on, it is allowed. It will work as a work-around if/until we expose this as a feature through the NIC resource.

[torumakabe]

Thanks @jaiveerk . However, when I tried that setting use-forwarded-headers , it had no effect.

The configmap I applied is as follows

kind: ConfigMap
apiVersion: v1
metadata:
  name: nginx
  namespace: app-routing-system
  labels:
    app.kubernetes.io/component: ingress-controller
    app.kubernetes.io/managed-by: aks-app-routing-operator
    app.kubernetes.io/name: nginx
data:
  allow-snippet-annotations: "true"
  use-forwarded-headers: "true"
  annotation-value-word-blocklist: >-
    load_module,lua_package,_by_lua,location,root,proxy_pass,serviceaccount,{,},'

And my Ingress

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: httpbin
  namespace: default
  annotations:
    nginx.ingress.kubernetes.io/configuration-snippet: |
      proxy_set_header X-Forwarded-Port "80";
      proxy_set_header X-Test-Pass-Port $pass_port;
      proxy_set_header X-Test-Server-Port $server_port;
      proxy_set_header X-Test-Service-Port $service_port;
spec:
  ingressClassName: webapprouting.kubernetes.azure.com
  rules:
    - http:
        paths:
          - backend:
              service:
                name: httpbin
                port:
                  number: 80
            path: /
            pathType: Prefix

But, when I checked with httpbin which echoes headers, X-Forwarded-Port was not returned.

curl http://203.0.113.1/headers
{
  "headers": {
    "Accept": "*/*", 
    "Host": "203.0.113.1", 
    "User-Agent": "curl/8.7.1", 
    "X-Forwarded-Host": "203.0.113.1", 
    "X-Forwarded-Scheme": "http", 
    "X-Scheme": "http", 
    "X-Test-Pass-Port": "8080", 
    "X-Test-Server-Port": "8080", 
    "X-Test-Service-Port": "80"
  }
}

I went into the Ingress NGINX controller and inspected the nginx.conf, and use-forwarded-headers appear to have been injected.

[snip]
        init_by_lua_block {
                collectgarbage("collect")

                -- init modules
                local ok, res

                ok, res = pcall(require, "lua_ingress")
                if not ok then
                error("require failed: " .. tostring(res))
                else
                lua_ingress = res
                lua_ingress.set_config({
                        use_forwarded_headers = true,
[snip]

Is there anything I might be misunderstanding in my configuration or verification method?

[kev-andrews]

Hey @torumakabe -

We can look into adding this in the future. In the meantime, feel free to add this setting to your nginx configmap that app routing creates. While this behavior isn't technically supported by the add-on, it is allowed. It will work as a work-around if/until we expose this as a feature through the NIC resource.

Currently struggeling with this as well - we need some official support for setting things like use-forwarded-headers in the configmap - the documentation states that editing the configmap is not supported - is this the case or is it fine to change it?

[microsoft-github-policy-service]

This issue has been automatically marked as stale because it has not had any activity for 180 days. It will be closed if no further activity occurs within 7 days of this comment. @jaiveerk

[microsoft-github-policy-service]

This issue will now be closed because it hasn't had any activity for 7 days after stale. @torumakabe feel free to comment again on the next 7 days to reopen or open a new issue after that time if you still have a question/issue or suggestion.