Need option to specify location-specific auth-annotations in Ingress-nginx

[avishnu]

Is this a request for help? (If yes, you should use our troubleshooting guide and community support channels, see https://kubernetes.io/docs/tasks/debug-application-cluster/troubleshooting/.):

What keywords did you search in NGINX Ingress controller issues before filing this one? (If you have found any duplicates, you should instead reply there.):

---

Is this a BUG REPORT or FEATURE REQUEST? (choose one):
Feature request

NGINX Ingress controller version:
1.13.3

Kubernetes version (use kubectl version):
Server Version: version.Info{Major:"1", Minor:"7+", GitVersion:"v1.7.8-gke.0", GitCommit:"a7061d4b09b53ab4099e3b5ca3e80fb172e1b018", GitTreeState:"clean", BuildDate:"2017-10-10T18:48:45Z", GoVersion:"go1.8.3", Compiler:"gc", Platform:"linux/amd64"}

Environment:
Cloud

• Cloud provider or hardware configuration: GKE
• OS (e.g. from /etc/os-release): Ubuntu 16.04.3 LTS
• Kernel (e.g. uname -a): Linux nginx-ingress-controller-1998717890-gsllx 4.4.0-1027-gke Remove hardcoded health check from GenericController #27-Ubuntu SMP Thu Aug 10 13:13:08 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux
• Install tools:
• Others:

What happened:

What you expected to happen:
I've hosted 3 services (serviceA and serviceB) behind my ingress-nginx. Below is the yaml.

kind: Ingress
apiVersion: extensions/v1beta1
metadata:
annotations:
kubernetes.io/ingress.class: nginx
ingress.kubernetes.io/rewrite-target: /
kubernetes.io/tls-acme: true
name: my-nginx-ingress
spec:
tls:

• hosts:
  • myhost
    secretName: mysecret
    rules:
• host: myhost
  http:
  paths:
  • path: /
    backend:
    serviceName: serviceA
    servicePort: 8080
  • path: /serviceB/
    backend:
    serviceName: serviceB-service
    servicePort: 7070

Everything is working great without any issues.
I can access serviceA end-point with https://myhost/ and serviceB with https://myhost/serviceB/ perfectly well.

• serviceA has its own authentication mechanism.
• serviceB does not have any authentication.

I've been trying to enable ingress basic-auth on serviceB ONLY. By following the steps https://github.com/kubernetes/contrib/tree/master/ingress/controllers/nginx/examples/auth, I can successfully configure nginx basic-auth. However, the below 3 annotations are getting applied to all the available locations in the translated nginx.conf. I want the auth-specific annotations to get written only inside the location specific to serviceB, which is not happening.
echo "
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: ingress-with-auth
annotations:
# type of authentication
ingress.kubernetes.io/auth-type: basic
# name of the secret that contains the user/password definitions
ingress.kubernetes.io/auth-secret: basic-auth
# message to display with an appropiate context why the authentication is required
ingress.kubernetes.io/auth-realm: "Authentication Required - foo"
spec:
rules:

• host: foo.bar.com
  http:
  paths:
  • path: /
    backend:
    serviceName: echoheaders
    servicePort: 80
    " | kubectl create -f -

I've referred to another source https://github.com/nginxinc/kubernetes-ingress/blob/master/examples/customization/cafe-ingress-with-annotations.yaml which recommends to use nginx-scripting for making my use-case work. This has 2 issues:

1. My ingress/nginx does not seem to understand the nginx.org/location-snippets annotation.
2. Even then, I was able to work-around using the configuration-snippet annotation as follows:
   ingress.kubernetes.io/configuration-snippet: |
   if ($request_uri ~ "serviceB") {
   set $auth "auth req";
   }
   if ($request_uri !~ "serviceB") {
   set $auth off;
   }
   auth_basic $auth;
   auth_basic_user_file /etc/ingress-controller/auth/default.passwd;
   proxy_set_header Authorization "";
   proxy_set_header X-AUTH-USER $remote_user;

The above works but has some limitations. What I'm interested in is the following:

1. An ingress option (annotation or directive) to add configuration to a SPECIFIC LOCATION ONLY. This should reflect in the nginx-controller's nginx.conf as well.
2. The said configuration should not be added to other locations (http rule/path) served by the same ingress/nginx.

I've tried to search but have not found any solution to my requirement, hence posting a request here.

How to reproduce it (as minimally and precisely as possible):

Anything else we need to know:

[aledbf]

@avishnu you can create two Ingress and only add the annotations to the ingress with the path you want to protect. No need to use the configuration-snippet annotation

[avishnu]

Thanks a ton @aledbf.
Creating another ingress solved the problem !! For the sake of others, I'm pasting below the working modified yaml which has 2 ingress resources:

1. my-nginx-ingress contains the spec for the service(s) which don't require authentication through nginx eg. serviceA.
2. my-nginx-ingress-auth contains the spec for the service(s) which require authentication (basic in my case) through nginx eg. serviceB.

---

kind: Ingress
apiVersion: extensions/v1beta1
metadata:
annotations:
kubernetes.io/ingress.class: nginx
ingress.kubernetes.io/rewrite-target: /
kubernetes.io/tls-acme: true
name: my-nginx-ingress
spec:
tls:

• hosts:
  • myhost
    secretName: mysecret
    rules:
• host: myhost
  http:
  paths:
  • path: /
    backend:
    serviceName: serviceA
    servicePort: 8080

---

kind: Ingress
apiVersion: extensions/v1beta1
metadata:
annotations:
kubernetes.io/ingress.class: nginx
ingress.kubernetes.io/rewrite-target: /
kubernetes.io/tls-acme: true
# type of authentication
ingress.kubernetes.io/auth-type: basic
# name of the secret that contains the user/password definitions
ingress.kubernetes.io/auth-secret: basic-auth
# message to display with an appropiate context why the authentication is required
ingress.kubernetes.io/auth-realm: "Authentication Required - foo"
# Below configuration-snippet is to pass on the authenticated user-name to serviceB
ingress.kubernetes.io/configuration-snippet: |
proxy_set_header X-AUTH-USER $remote_user;
name: my-nginx-ingress-auth
spec:
tls:

• hosts:
  • myhost
    secretName: mysecret
    rules:
• host: myhost
  http:
  paths:
  • path: /serviceB/
    backend:
    serviceName: serviceB-service
    servicePort: 7070

[GimmeDaKitty]

@avishnu I am trying to do the same and have a couple of questions;

• Do you need to create a second deployment of an nginx ingress? (I install ingress using its helm chart) if so, how do you couple each ingress controller to the ingress deployment that is shown above?
• Don't you get issues as both ingresses are of the same class? It was my understanding that if you have multiple ingresses you have issues if you don't segregate the controllers by class

[QwertyJack]

@bdelsaz

1. No need to create a second ingress-controller.
2. No issues.

Because ingress controller will auto configure all matched ingress.

[sumanthjoel]

Hi,

Thanks for the clarification above. I have a single pod with multiple containers in it. These containers expose different endpoints for users and share a common filesystem.
Can I configure different authentication annotations for these different endpoints in my ingress? Since I have one pod, I am guessing I can only have one ingress object.

[sri-05]

Hi I am also having same problem
I need to allow only specific ips to /admin path and restrict all
I created two ingress

1. Without having any rules and only ssl termination which is working fine
2. Added server-snippet and location /admin block to allow and deny (not working )

For the second ingress expected behaviour is only accessible by allowed ips. But getting 403 for all the ips. And am using nginx ingress