Istio ingress static IP support for GCP

[charlesverdad]

After applying this ingress resource in a GKE cluster,

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: static-ip
  # Assumes a global static ip with the same name exists.
  # You can acquire a static IP by running
  # gcloud compute addresses create test-ip --global
  annotations:
    kubernetes.io/ingress.global-static-ip-name: "test-ip"
    kubernetes.io/ingress.class: "istio"
spec:
  tls:
  # This assumes istio-ingress-certs exist with tls.key and tls.crt
  - secretName: istio-ingress-certs 
  backend:
    # This assumes http-svc exists and routes to healthy endpoints.
    serviceName: http-svc
    servicePort: 80

The address returned by kubectl get ing static-ip seem to depend on the ephemeral external ip of the istio-ingress service but is different from the gcloud provisioned test-ip. My expected behavior is for the ingress controller to use the global-static-ip-name test-ip.

[linsun]

@ayj this is a GCP issue, can you comment?

[ayj]

kubernetes.io/ingress.global-static-ip-name is not supported by Istio ingress controller.

[ldemailly]

do we plan on addressing this ? how do people expose GKE services ?

[vaikas]

Are there workarounds for this? Seems pretty limiting to not be able to use static IP with Istio ingress.

[ldemailly]

@vaikas-google the work around is to go in the cloud management UI and switch the ip from ephemeral to static after the ingress is up

for instance:

[vaikas]

Aha, ok, thanks!

…

On Thu, Feb 22, 2018 at 6:10 PM Laurent Demailly ***@***.***> wrote: @vaikas-google <https://github.com/vaikas-google> the work around is to go in the cloud management UI and switch the ip from ephemeral to static after the ingress is up for instance: [image: screen shot 2018-02-22 at 6 09 21 pm] <https://user-images.githubusercontent.com/3664595/36574666-a9693d4e-17fb-11e8-8a79-af87d8a8a19c.png> — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#1024 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AKwedOA1LN6HwBy-Odp6Hc4sL8tnVguvks5tXh4kgaJpZM4Pt3KL> .

[costinm]

Can we also add the kubernetes.io/ingress.global-static-ip-name ? ( in the helm templates, where it can be set in values.yaml ). Probably P2 since we have a workaround - which should be documented. On Fri, Feb 23, 2018 at 7:27 AM, Ville Aikas <notifications@github.com> wrote:

…

Aha, ok, thanks! On Thu, Feb 22, 2018 at 6:10 PM Laurent Demailly ***@***.*** > wrote: > @vaikas-google <https://github.com/vaikas-google> the work around is to > go in the cloud management UI and switch the ip from ephemeral to static > after the ingress is up > > for instance: > [image: screen shot 2018-02-22 at 6 09 21 pm] > <https://user-images.githubusercontent.com/3664595/ 36574666-a9693d4e-17fb-11e8-8a79-af87d8a8a19c.png> > > — > You are receiving this because you were mentioned. > Reply to this email directly, view it on GitHub > <#1024 (comment)>, or mute > the thread > <https://github.com/notifications/unsubscribe-auth/AKwedOA1LN6HwBy- Odp6Hc4sL8tnVguvks5tXh4kgaJpZM4Pt3KL> > . > — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#1024 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAFI6tSm68mDN8qGQXZX9r0RozLAG2Q1ks5tXtjRgaJpZM4Pt3KL> .

[ldemailly]

We need to support it before we add it, no?

[dennisseidel]

Am I correct that this workaround only allows to make the ip static but there is no way to attach this static ip address to another existing Istio cluster (e.g. starting a new cluster, attaching the the static ip of the ingress of the old cluster to the ingress of the new cluster, then deleting the old cluster)? Or does the workaround allow for this too?

[ilyasotkov]

@denseidel Not sure if that what you're asking, but if you just want to attach an existing (regional) static IP address to your istio-ingress, you can add spec.loadBalancerIP to your istio-ingress service. More info in Kubernetes docs: https://kubernetes.io/docs/concepts/services-networking/service/#type-loadbalancer

If you use the Helm chart to deploy Istio, these are the changes that are needed to be made (link to a commit in my fork of the Istio Helm chart): exekube/charts@5fc3353

@ldemailly should I submit a PR with the changes to the Helm chart? The feature is very useful for having an ephemeral cluster with a permanent static IP address (and permanent DNS records for it).

[ldemailly]

Yes please!

[ilyasotkov]

#4955 has been merged ✅

You can now reserve a regional static IP address (https://cloud.google.com/compute/docs/ip-addresses/reserve-static-external-ip-address), then bind it to Istio ingress service like this:

# values.yaml
ingress:
  enabled: true
  ...
  service:
    loadBalancerIP: 130.211.160.207 # (your static IP address)

[ldemailly]

that's great progress, thanks, any way to do it by the name of the reserved ip instead of the actual address ?

[rshriram]

the issue seems to have been resolved by #4955

[Bulat-Gumerov]

Can anyone give a link for example YAML on how to use istio ingress in GKE with static IP? I've already tried with regional static IP, no success.

[aaaaahaaaaa]

Google has documentation on how to change the IP of LB provisioned with Istio on GKE clusters using the addon: https://cloud.google.com/anthos/gke/docs/on-prem/archive/1.1/how-to/add-ons/istio#configure_an_external_ip_address

Which is basically patching the istio-ingressgateway:

kubectl patch service istio-ingressgateway --patch '{"spec":{"loadBalancerIP": "<STATIC_IP>"}}' --namespace istio-system

For those using Terraform, my workaround is to patch the gateway using the local-exec provisioner and kubectl. Not pretty but functional (based on hashicorp/terraform-provider-kubernetes#723 (comment)):

resource "null_resource" "istio-load-balancer-ip-patch" {
  provisioner "local-exec" {
    command = <<EOH
cat >/tmp/ca.crt <<EOF
${base64decode(google_container_cluster.my_cluster.master_auth[0].cluster_ca_certificate)}
EOF
  kubectl \
  --server="https://${google_container_cluster.my_cluster.endpoint}" \
  --token="${data.google_client_config.default.access_token}" \
  --certificate_authority=/tmp/ca.crt \
  patch service istio-ingressgateway --patch '{"spec":{"loadBalancerIP": "${var.istio_load_balancer_ip}"}}' --namespace istio-system
EOH
  }
}