-
Notifications
You must be signed in to change notification settings - Fork 32
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
BUG: Can not deploy any type of Ceph cluster #700
Comments
This looks like the same thing we've hit intermittently inside sesdev CI, notably the most recent failure of #696 (output at http://see.prv.suse.net:8080/blue/organizations/jenkins/sesdev-integration/detail/PR-696/1/pipeline). The notes in #689 and #691 are also relevant. A couple of things stand out to me here in the above output:
That summary information is produced by ceph-salt at https://github.com/ceph/ceph-salt/blob/619351846592062c245e22555fea399d8f3d5c02/ceph_salt/execute.py#L1288. The counters indicate the number of minions on which Given that all the minions failed, surely it's incorrect for As for the subsequent failures running Honestly, it feels to me like what's happening here is ceph-salt invokes |
@votdev can you please try something for me? Remove any of my experimental patches applied to sesdev, then rerun
Does that give you a successful deployment? |
After looking through ceph-salt logs from previous failed Jenkins runs, and also at Volker's issue which seems to be the same thing (see #700), my current suspicion is that the ceph-salt executor is correctly starting `salt -G 'ceph-salt:member' state.apply ceph-salt` but is then failing to pick up some (or all) of the event notifications, which results in it returning too soon, while `cephadm bootstrap` is still running. Assuming I'm on the right track here, let's ask `sesdev` to run that salt command directly, to remove the ceph-salt executor from the picture entirely. Signed-off-by: Tim Serong <tserong@suse.com>
I can not deploy a multi or single-node cluster anymore, regardless if ses6, ses7, ses7p or pacific.
E.g. the following command(s) are used to deploy the cluster:
One of the following errors always appears and aborts the deployment.
The text was updated successfully, but these errors were encountered: