Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

GKE ingress stuck in creating after deploying ingress 1.11.5.gke.5 #605

Closed
Arconapalus opened this issue Jan 11, 2019 · 18 comments
Closed

Comments

@Arconapalus
Copy link

Arconapalus commented Jan 11, 2019

I also am running into this issue on 1.11.5-gke.5. The creating ingress is stuck and I have deleted and recreated the ingress. Should I delete node and cluster and recreate at 1.10.7.gke.2?
https://cloud.google.com/kubernetes-engine/docs/tutorials/http-balancer
I followed this guide up to the point of apply -f basic-ingress.yml and ingress is stuck creating,
I waiting for an hour then deleted and re-deployed

@rramkumar1
Copy link
Contributor

Are there any events on your Ingress?

You can check by doing:

kubectl describe ing [NAME] -n [NAMESPACE]

@Arconapalus
Copy link
Author

Arconapalus commented Jan 11, 2019

events are blank
Would a VPC and subnet interfere?

@rramkumar1
Copy link
Contributor

Can you email me your project name, cluster name, and location so I can take a look?

@Arconapalus
Copy link
Author

sent

@rramkumar1
Copy link
Contributor

@Arconapalus Discussed offline but it appears that the L7 Load Balancing Add-On was not turned on for your cluster.

Please close this bug once you have turned on the add-on and verified everything works on your end.

@zvictor
Copy link

zvictor commented Feb 20, 2019

it appears that the L7 Load Balancing Add-On was not turned on for your cluster.

I was running into the same problem. I am part of the closed preview of knative in gcloud and the instructions provided in https://cloud.google.com/run/docs/gke/setup made me accidentally disable the load balancer.

I won't quote the precise command here because that page is tagged as Confidential Material 🙄, but just running the instructions provided there will replace the list of add-ons being added to your cluster instead of adding new ones, as some would have expected.

It is clearly stated in the documentation found at gcloud beta container clusters create --help:

 --addons=[ADDON,...]
    Default set of addons includes HttpLoadBalancing,
    HorizontalPodAutoscaling. Addons
    (https://cloud.google.com/kubernetes-engine/reference/rest/v1/projects.zones.clusters#AddonsConfig)
    are additional Kubernetes cluster components. Addons specified by this
    flag will be enabled. The others will be disabled. ADDON must be one
    of: HttpLoadBalancing, HorizontalPodAutoscaling, KubernetesDashboard,
    Istio, NetworkPolicy, CloudRun.

Even though I don't necessarily disagree with the chosen api, I can foresee many others experiencing the exact same problem and getting stuck with ingress while trying to deploy other features like knative.

I would suggest making improvement to detect the presence of the required add-on. Being in the dark with no events or errors thrown, with only this issues repository to help is not a comfortable situation.

@rramkumar1
Copy link
Contributor

rramkumar1 commented Feb 20, 2019

@zvictor Thanks for the investigation! I was wondering why we had so many issues related to the add-on not being on...

I would suggest making improvement to detect the presence of the required add-on. Being in the dark with no events or errors thrown, with only this issues repository to help is not a comfortable situation.

This is tricky because without the add-on being enabled, the controller cannot even be deployed and thus cannot emit events. However, I 100% agree that this is a bad UX. I'm going to file a separate issue in this repository for better tracking and update there once we have a solution.

In the meantime, I will also reach out to the owner of that documentation so we can get it fixed.

@rramkumar1
Copy link
Contributor

@zvictor Docs are fixed now.

@caquino
Copy link

caquino commented May 27, 2019

I think I have a different issue but with a similar outcome, I'm seeing my ingress stuck in create.

I'm running 1.13.6-gke.0 and I've checked addonsConfig on the cluster, and http load balancing is not set to "disabled".

I can't see any events other than "ADD" on my describe ingress.

Also when I check the load balancer on the console I see that it has no frontend attached to it, which I guess is waiting for the ingress to be provisioned.

My namespace + ingress name has 35 characters, which probably doesn't trigger the 55 chars bug

Is there anything else I can do to debug this issue?

@rramkumar1
Copy link
Contributor

@caquino Can you please file a separate issue in this repo?

@manast
Copy link

manast commented Jun 19, 2019

@caquino did you create a separate issue? I think I am running into exactly the same issue. Stuck Ingress and empty frontend on the loadbalancer, also no ephemeral IP assigned to the ingress.

I actually tried to delete it and create it a few times, for no use:
image

@manast
Copy link

manast commented Jun 19, 2019

as an extra piece of information, even trying to create new ingress gets stuck now. Seems like something broke in the cluster and it is not possible to create new ingress, If I have to guess I think it gets stuck in the creation of the ephemeral IP.

@rramkumar1
Copy link
Contributor

@manast Can you please file a separate issue?

@manast
Copy link

manast commented Jun 19, 2019

@rramkumar1 yes. I am doing some more tests before I fill the issue.

@manast
Copy link

manast commented Jun 19, 2019

I upgraded the cluster to version 1.13.6 and the issue seems to be resolved.

@salmansheikh5
Copy link

Hi everyone, I was having the same problem; but the solution I found was that I had to enable the "HTTP load balancing" Add-on in the cluster's add-ons settings.
kubernetes_1

When the HTTP Load Balancer was disabled it gave a warning that the Kubernetes ingress object may not work properly.

It took some time for the cluster of update, after that I deleted the stuck Ingress object and recreated it. And it worked. I hope my solution helps.

P.S
I also upgraded my cluster to 1.12.9-gke.7 (as stated above). But that solution didn't work for me.

@djsell
Copy link

djsell commented Jul 29, 2019

I wasted a long time trying to figure out why my Ingress wasn't working, and @muhammadsalmansheikh fix worked. This should really be brought to attention on the Services & Ingress page here https://console.cloud.google.com/kubernetes/discovery

It'll save a lot of people time if it just displays a warning like, "Ingress isn't available because you don't have the HTTP Load Balancer add-on enabled."

Thanks

@bowei
Copy link
Member

bowei commented Jul 29, 2019

It's an unfortunate aspect of the spec, but many people run their own Ingress provider (e.g. ingress-nginx). It's difficult to tell if disabling and having Ingress objects is around is intentional or not from the provider standpoint.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

8 participants