Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Test cluster is unhealthy no ready nodes #150

Closed
jlewi opened this issue Jan 25, 2018 · 4 comments
Closed

Test cluster is unhealthy no ready nodes #150

jlewi opened this issue Jan 25, 2018 · 4 comments
Labels
area/testing

Comments

@jlewi
Copy link
Contributor

jlewi commented Jan 25, 2018

Our test cluster is unhealthy. There are no ready nodes and this is causing workflows to no longer be sumbitted.
@jlewi
Copy link
Contributor Author

jlewi commented Jan 25, 2018

Kubelet seems to keep restarting

node.events.txt

@jlewi
Copy link
Contributor Author

jlewi commented Jan 25, 2018

Lets delete all the existing workflows

kubectl delete wf --all=true

@jlewi jlewi mentioned this issue Jan 25, 2018
Merged
@jlewi
Copy link
Contributor Author

jlewi commented Jan 25, 2018

I deleted the nodes and recreated them and now they are reporting as ready.

@jlewi
Copy link
Contributor Author

jlewi commented Jan 25, 2018

Tests are running and passing again.

@jlewi jlewi closed this as completed Feb 4, 2018
@richardsliu richardsliu mentioned this issue Jul 20, 2018
Merged
yanniszark pushed a commit to arrikto/kubeflow that referenced this issue Nov 1, 2019
@richardsliu @k8s-ci-robot
Partial Revert "Remove admission-webhook/bootstrap hard-coded selecto…
d0d71f6 
…r" (kubeflow#150)

* Revert "Remove admission-webhook/bootstrap hard-coded selector (kubeflow#134)"

This reverts commit fea53c8.

* Update stateful-set.yaml

* Update bootstrap-base_test.go

* Fix build

* Revert "Remove admission-webhook/bootstrap hard-coded selector (kubeflow#134)"

This reverts commit fea53c8.

Update stateful-set.yaml

Update bootstrap-base_test.go

Fix build
elenzio9 pushed a commit to arrikto/kubeflow that referenced this issue Oct 31, 2022
@johnugeorge @k8s-ci-robot
Update org.yaml (kubeflow#150)
fa8b6e5
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
area/testing
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@jlewi