Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

microk8s.enable kubeflow hangs at bootstrapping #1954

Closed
AdeloreSimiloluwa opened this issue Jan 30, 2021 · 6 comments
Closed

microk8s.enable kubeflow hangs at bootstrapping #1954

AdeloreSimiloluwa opened this issue Jan 30, 2021 · 6 comments
Assignees
Labels

Comments

@AdeloreSimiloluwa
Copy link

AdeloreSimiloluwa commented Jan 30, 2021

While enabling kubeflow on a vnc microk8s.enable kubeflow hangs at bootstrapping.

Running microk8s enable kubeflow --debug shows this:
image

microk8s inspect outputs :

Inspecting Certificates
Inspecting services
  Service snap.microk8s.daemon-cluster-agent is running
  Service snap.microk8s.daemon-containerd is running
  Service snap.microk8s.daemon-apiserver is running
  Service snap.microk8s.daemon-apiserver-kicker is running
  Service snap.microk8s.daemon-control-plane-kicker is running
  Service snap.microk8s.daemon-proxy is running
  Service snap.microk8s.daemon-kubelet is running
  Service snap.microk8s.daemon-scheduler is running
  Service snap.microk8s.daemon-controller-manager is running
  Copy service arguments to the final report tarball
Inspecting AppArmor configuration
Gathering system information
  Copy processes list to the final report tarball
  Copy snap list to the final report tarball
  Copy VM name (or none) to the final report tarball
  Copy disk usage information to the final report tarball
  Copy memory usage information to the final report tarball
  Copy server uptime to the final report tarball
  Copy current linux distribution to the final report tarball
  Copy openSSL information to the final report tarball
  Copy network configuration to the final report tarball
Inspecting kubernetes cluster
  Inspect kubernetes cluster
Inspecting juju
  Inspect Juju
Inspecting kubeflow
  Inspect Kubeflow

Building the report tarball
  Report tarball is at /var/snap/microk8s/1985/inspection-report-20210129_235025.tar.gz

The tarball is at /var/snap/microk8s/1985/inspection-report-20210129_235025.tar.gz

@AdeloreSimiloluwa
Copy link
Author

it finally stopped running and here is the error :

Enabling dns...
Enabling storage...
Enabling dashboard...
Enabling ingress...
Enabling metallb:10.64.140.43-10.64.140.49...
Waiting for DNS and storage plugins to finish setting up
DNS and storage setup complete. Checking connectivity...
Bootstrapping...
Kubeflow could not be enabled:
Creating Juju controller "uk8s" on microk8s/localhost
Fetching Juju Dashboard 0.3.0
Creating k8s resources for controller "controller-uk8s"
Downloading images
ERROR failed to bootstrap model: creating controller stack for controller: creating statefulset for controller: timed out waiting for controller pod: unschedulable: 0/1 nodes are available: 1 node(s) had taint {node.kubernetes.io/disk-pressure: }, that the pod didn't tolerate.
ERROR error cleaning up: cannot acquire lock file to read controller uk8s: unable to open /tmp/juju-store-lock-6465663065326438: permission denied
ERROR cannot destroy newly created controller "uk8s" details: cannot acquire lock file to remove controller uk8s: unable to open /tmp/juju-store-lock-6465663065326438: permission denied

Command '('microk8s-juju.wrapper', 'bootstrap', 'microk8s', 'uk8s')' returned non-zero exit status 1

i have also tried removing microk8s,reinstalling with --channel=latest/edge and enabling kubeflow with KUBEFLOW_CHANNEL=edge microk8s.enable kubeflow and i have the same issues.

@mavencode01
Copy link

Try setting export KUBEFLOW_DEBUG=true to see the logs

@knkski
Copy link
Contributor

knkski commented Mar 3, 2021

@AdeloreSimiloluwa: Are you still running into this issue?

@AdeloreSimiloluwa
Copy link
Author

AdeloreSimiloluwa commented Mar 4, 2021 via email

@maxtaq
Copy link

maxtaq commented Mar 5, 2021

@AdeloreSimiloluwa: Can you tell me about a solution you have done?

@stale
Copy link

stale bot commented Jan 31, 2022

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

@stale stale bot added the inactive label Jan 31, 2022
@stale stale bot closed this as completed Nov 23, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

4 participants