Nginx: private ingress only

[james-masson]

Hi,

I've been trying to setup a GKE configuration that serves highly available internal-only services to clients over the GCE VPN.

I can't use any of the native GCE ingress / service modules, as they don't work over GCE VPN.

I've got close by using the Nginx ingress module, but it reports the external-IP of the node in the ingress definition. I need it to report the private-IP of the node, as it needs to be consumed for DNS registration with external-dns.

The official workaround seems to be, use --publish-service - but this won't work for me, as I can't use any of the GCE service modules to front Nginx.

Any ideas?

thanks

James M

[aledbf]

@james-masson you need to add the annotation service.beta.kubernetes.io/external-traffic: OnlyLocal to the nginx ingress service. This will report the correct IP address.

[james-masson]

Doesn't work - related to this? #1414

My config...

Name:		nginx-ingress-controller-2469740818-vg1t8
Namespace:	shared-services
Node:		gke-helm-dev-default-pool-3d748e00-m3r6/<node_ip>
Start Time:	Mon, 25 Sep 2017 09:25:25 +0100
Labels:		k8s-app=nginx-ingress-controller
		pod-template-hash=2469740818
Annotations:	kubernetes.io/created-by={"kind":"SerializedReference","apiVersion":"v1","reference":{"kind":"ReplicaSet","namespace":"shared-services","name":"nginx-ingress-controller-2469740818","uid":"144dcb08-a1c...
		prometheus.io/port=10254
		prometheus.io/scrape=true
		service.beta.kubernetes.io/external-traffic=OnlyLocal
Status:		Running
IP:		172.16.0.23
Created By:	ReplicaSet/nginx-ingress-controller-2469740818

JEG-CON-GEL0068:helm-nginx-ingress james.masson$ kubectl --namespace shared-services get ingress
NAME      HOSTS                         ADDRESS          PORTS     AGE
nginx     testconsul.<domain>   <public_ip>   80        2d

[aledbf]

@james-masson the ingress controller service is of type=LoadBalancer? If that's the case you just need to use a normal service instead to only make it available to the cluster.

[james-masson]

Ah - did you mean I have to deploy it on the ingress definition in my service behind nginx? Or on the nginx-ingress-controller deployment?

I deployed it on the nginx-ingress-controller