unable to attach aws ebs as persistent volume for kubernetes #1195

Open
srikumar-b opened this Issue Oct 19, 2017 · 7 comments

Comments

Projects
None yet
2 participants

srikumar-b commented Oct 19, 2017

Report

Thank you for trying conjure-up! Before reporting a bug please make sure you've gone through this checklist:

Please provide the output of the following commands

on MacOS

[~]$ juju version
2.2.4-highsierra-amd64
[~]$ which juju
/usr/local/bin/juju
[~]$ which conjure-up
/usr/local/bin/conjure-up
[~]$ conjure-up --version
conjure-up 2.3.1

Please attach tarball of ~/.cache/conjure-up:

2017-10-18 22:32:20,142 [DEBUG] conjure-up/kubernetes-core - events.py:53 - Setting RelationsAdded:easyrsa at conjureup/juju.py:708
2017-10-18 22:32:20,143 [DEBUG] conjure-up/kubernetes-core - events.py:53 - Clearing PendingRelations:kubernetes-master:kube-control <-> kubernetes-worker:kube-control at conjureup/juju.py:705
2017-10-18 22:32:20,145 [DEBUG] conjure-up/kubernetes-core - events.py:53 - Setting RelationsAdded:kubernetes-master:kube-control <-> kubernetes-worker:kube-control at conjureup/juju.py:706
2017-10-18 22:32:20,146 [DEBUG] conjure-up/kubernetes-core - events.py:53 - Setting RelationsAdded:kubernetes-worker at conjureup/juju.py:708
2017-10-18 22:32:20,147 [DEBUG] conjure-up/kubernetes-core - events.py:53 - Clearing PendingRelations:etcd:db <-> kubernetes-master:etcd at conjureup/juju.py:705
2017-10-18 22:32:20,148 [DEBUG] conjure-up/kubernetes-core - events.py:53 - Setting RelationsAdded:etcd:db <-> kubernetes-master:etcd at conjureup/juju.py:706
2017-10-18 22:32:20,150 [DEBUG] conjure-up/kubernetes-core - events.py:53 - Setting RelationsAdded:kubernetes-master at conjureup/juju.py:708
2017-10-18 22:32:20,154 [DEBUG] conjure-up/kubernetes-core - events.py:53 - Setting DeploymentComplete at conjureup/controllers/deploy/common.py:44
2017-10-18 22:32:20,155 [DEBUG] conjure-up/kubernetes-core - events.py:53 - Received DeploymentComplete at conjureup/controllers/deploy/common.py:59 in task _wait_for_applications at conjureup/controllers/deploy/gui.py:90
2017-10-18 22:32:20,155 [INFO] conjure-up/kubernetes-core - common.py:61 - Waiting for deployment to settle.
2017-10-18 22:32:20,155 [INFO] conjure-up/kubernetes-core - step.py:94 - Running step: 00_deploy-done.
2017-10-18 22:32:20,243 [DEBUG] conjure-up/kubernetes-core - step.py:134 - Storing environment
2017-10-18 22:32:20,249 [DEBUG] conjure-up/kubernetes-core - step.py:140 - Executing script: /Users/abcd/.cache/conjure-up/kubernetes-core/steps/00_deploy-done
2017-10-18 22:32:20,303 [DEBUG] conjure-up/kubernetes-core - events.py:53 - Setting ModelSettled at conjureup/controllers/deploy/common.py:69 in task _wait_for_applications at conjureup/controllers/deploy/gui.py:90
2017-10-18 22:32:20,303 [INFO] conjure-up/kubernetes-core - common.py:71 - Model settled.
2017-10-18 22:32:20,306 [INFO] conjure-up/kubernetes-core - step.py:94 - Running step: step-01_get-kubectl.
2017-10-18 22:32:20,421 [DEBUG] conjure-up/kubernetes-core - step.py:134 - Storing environment
2017-10-18 22:32:20,429 [DEBUG] conjure-up/kubernetes-core - step.py:140 - Executing script: /Users/abcd/.cache/conjure-up/kubernetes-core/steps/step-01_get-kubectl
2017-10-18 22:32:25,740 [DEBUG] conjure-up/kubernetes-core - events.py:53 - Setting Error at conjureup/events.py:147
2017-10-18 22:32:25,740 [ERROR] conjure-up/kubernetes-core - events.py:159 - Unhandled exception in <Task finished coro=<RunStepsController.run_steps() done, defined at /usr/local/Cellar/conjure-up/2.3.1_1/libexec/lib/python3.6/site-packages/conjureup/controllers/runsteps/gui.py:15> exception=Exception('Failure in step /Users/abcd/.cache/conjure-up/kubernetes-core/steps/step-01_get-kubectl',)>
Traceback (most recent call last):
  File "/usr/local/Cellar/conjure-up/2.3.1_1/libexec/lib/python3.6/site-packages/conjureup/controllers/runsteps/gui.py", line 19, in run_steps
    step.result = await step.run(app.ui.set_footer)
  File "/usr/local/Cellar/conjure-up/2.3.1_1/libexec/lib/python3.6/site-packages/conjureup/models/step.py", line 162, in run
    raise Exception("Failure in step {}".format(self.filename))
Exception: Failure in step /Users/abcd/.cache/conjure-up/kubernetes-core/steps/step-01_get-kubectl
2017-10-18 22:32:25,750 [DEBUG] conjure-up/kubernetes-core - __init__.py:21 - Showing dialog for exception: Failure in step /Users/abcd/.cache/conjure-up/kubernetes-core/steps/step-01_get-kubectl
2017-10-18 22:34:51,533 [DEBUG] conjure-up/kubernetes-core - events.py:53 - Setting Shutdown at conjureup/events.py:143
2017-10-18 22:34:51,534 [DEBUG] conjure-up/kubernetes-core - events.py:53 - Received Shutdown at conjureup/events.py:174
2017-10-18 22:34:51,534 [INFO] conjure-up/kubernetes-core - events.py:178 - Shutting down
2017-10-18 22:34:51,535 [INFO] conjure-up/kubernetes-core - app_config.py:187 - Storing conjure-up state
2017-10-18 22:34:51,639 [INFO] conjure-up/kubernetes-core - app_config.py:191 - State saved in model config
2017-10-18 22:34:51,642 [INFO] conjure-up/kubernetes-core - events.py:188 - Disconnecting model
2017-10-18 22:34:51,844 [INFO] conjure-up/kubernetes-core - events.py:190 - Disconnected
2017-10-18 22:34:51,844 [DEBUG] conjure-up/kubernetes-core - events.py:200 - Cancelling pending task: <Task finished coro=<WebSocketCommonProtocol.run() done, defined at /usr/local/Cellar/conjure-up/2.3.1_1/libexec/lib/python3.6/site-packages/websockets/protocol.py:428> result=None>```

## What provider (aws, maas, localhost, etc)?
aws


## Commands ran
conjure-up

## Additional Information
trying to create persistent volumes for kubernetes using amaon ebs but unable to create them

Warning FailedMount 1s kubelet, ip-172-31-4-138 MountVolume.SetUp failed for volume "postgres-pv" : mount failed: exit status 32
Mounting command: systemd-run
Mounting arguments: --description=Kubernetes transient mount for /var/lib/kubelet/pods/fa0f854f-b482-11e7-a5f0-0699a0199962/volumes/kubernetes.ioaws-ebs/postgres-pv --scope -- mount -o bind /var/lib/kubelet/plugins/kubernetes.io/aws-ebs/mounts/vol-0d61d5d0808a5bd71 /var/lib/kubelet/pods/fa0f854f-b482-11e7-a5f0-0699a0199962/volumes/kubernetes.ioaws-ebs/postgres-pv
Output: Running scope as unit run-raeb618ada174443893fbbe3044c92617.scope.
mount: special device /var/lib/kubelet/plugins/kubernetes.io/aws-ebs/mounts/vol-0d61d5d0808a5bd71 does not exist```

Contributor

battlemidget commented Oct 19, 2017

Can you at least attach a tarball of ~/.cache/conjure-up?

Thanks for quick followup.
Attached the folder that includes spells also
conjure-up_cache.tar.gz

Contributor

battlemidget commented Oct 19, 2017

Can you retry with:

CONJUREUP_REGISTRY_BRANCH=master conjure-up canonical-kubernetes

srikumar-b commented Oct 19, 2017

Tried on macOS and it did not work.

$ juju ssh kubernetes-master/0 sudo snap get kube-apiserver cloud-provider
error: snap "kube-apiserver" has no "cloud-provider" configuration option
Connection to 18.216.47.152 closed.
$ 
Contributor

battlemidget commented Oct 19, 2017

So the spells need to be updated from the master branch, so you needed to make sure you are running this https://github.com/conjure-up/spells/tree/master/canonical-kubernetes where the native stuff is enabled.

You could git clone that repo and run conjure-up spells/canonical-kubernetes --nosync

Valid spell files are copied to cache directory but conjure-up on macOS still does not run those spell files...
here is the command i used: conjure-up --nosync --spells-dir ~/spells/ kubernetes
attached new logs again...
conjure-up-cache.tar.gz
Not sure where it is failing to run the spell files

@battlemidget battlemidget self-assigned this Oct 20, 2017

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment