Set custom ports

[pichouk]

Hi,
I'm using nginx ingress controller for my Kubernetes cluster, which can be reached by a load balancer.

I successfully set up Proxy Protocol between the LB and my ingress controller. But now, I can't reach my node with "traditional" HTTP request (seems logic because Nginx listen to proxy protocol).
In order to join my node with classic HTTP and through my LB, I try to setup the following :

• one ingress controller listening to port 80 and 443, without proxy protocol,
• one ingress controller listening to 40080 and 40443 with proxy protocol.

Here is my configuration file :

apiVersion: extensions/v1beta1
kind: DaemonSet
metadata:
  name: nginx-ingress-controller
spec:
  template:
    metadata:
      labels:
        app: nginx-ingress
        name: nginx-ingress
    spec:
      hostNetwork: true
      terminationGracePeriodSeconds: 60
      containers:
      - image: gcr.io/google_containers/nginx-ingress-controller:0.8.3
        name: nginx-ingress
        imagePullPolicy: Always
        livenessProbe:
          httpGet:
            path: /healthz
            port: 18080
            scheme: HTTP
          initialDelaySeconds: 10
          timeoutSeconds: 1
        ports:
        - containerPort: 80
          hostPort: 80
        - containerPort: 443
          hostPort: 443
        args:
        - /nginx-ingress-controller
        - --default-backend-service=default/default-http-backend
        - --nginx-configmap=default/nginx-ingress
      - image: gcr.io/google_containers/nginx-ingress-controller:0.8.3
        name: nginx-ingress-proxy-protocol
        imagePullPolicy: Always
        livenessProbe:
          httpGet:
            path: /healthz
            port: 18080
            scheme: HTTP
          initialDelaySeconds: 10
          timeoutSeconds: 1
        ports:
        - containerPort: 40080
          hostPort: 40080
        - containerPort: 40443
          hostPort: 40443
        args:
        - /nginx-ingress-controller
        - --default-backend-service=default/default-http-backend
        - --nginx-configmap=default/nginx-ingress-proxy-protocol
      nodeSelector:
         type: node

As you can see, I have no other choices than to map my containers ports to my node ports (hostNetwork: True) because the hostPort parameter is not working for now (see kubernetes/kubeadm#11 and kubernetes/kubernetes#31307 ). Because of this, my second nginx-ingress-controller (which should manage proxy protocol) have to listen to ports 40080 and 40443.

I only need to change the Nginx listen directive, I see 3 possibilities :

1. Mount a volume (as explain here https://github.com/kubernetes/ingress/blob/master/controllers/nginx/configuration.md#custom-nginx-template ) for this file
2. Use an annotation or ConfigMap argument to use custom ports
3. Build a custom nginx ingress controller with a custom template

I would prefer not to use solution 1 (don't want to add a volume only for that kind of thing).
Solution 2 is not implemented. Not sure how to do it, but I can try if that seems a good and clean solution ?
I tried solution 3, but I failed to build the container image as explain here https://github.com/kubernetes/ingress/blob/master/docs/dev/releases.md. I'm stuck to this error when make build :

pkg/cmd/controller/nginx.go:96: cannot convert &n (type *NGINXController) to type ingress.Controller:
	*NGINXController does not implement ingress.Controller (wrong type for OverrideFlags method)
		have OverrideFlags(*"github.com/spf13/pflag".FlagSet)
		want OverrideFlags(*"k8s.io/ingress/vendor/github.com/spf13/pflag".FlagSet)

(I'm not Go user, not sure of what this mean).

Any ideas to solve this issue or to successfully build a custom ingress controller ?

[cmluciano]

I think 3) may be related to #317

[pichouk]

I'm not sure. Changes are made in examples/custom-controller/server.go. The methods #317 add are already in controllers/nginx/pkg/cmd/controller/nginx.go.

EDIT : Ok it seems that I made a mistake when trying to solve Go dependencies before building. My bad, seems to build now. I'll try to set custom ports and will give some news.

[cmluciano]

👌🏽