-
Notifications
You must be signed in to change notification settings - Fork 103
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
nexus: Assertion failure when unpublishing a faulted nexus #584
Conversation
bors try |
tryBuild failed: |
bors retry |
tryBuild succeeded: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
lgtm
The assertion occurs due to the nvmf subsystem transitioning from a paused to an inactive state, which is forbidden. Resume the nexus first, so that it is in an active state, before unpublishing it, which also covers destroying a nexus. A nexus in a faulted state already has no usable replicas so is itself already unusable. Repurpose the existing cargo test for a replica that is stopped, then continued, to test the above case with a single remote replica.
190818c
to
2beea26
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Although code wise it seems to be correct -- I wonder if we should fix it this way. The pause stated is not something that is allowed AFAIK, so what if we simply always resume with the note that when all replicas are gone, we simply have a controller without namespace.
I tried changing
If the delay is 10-12s, when faulting the child has started but not yet completed, then you get the original assertion back as the subsystem is in the paused state while child reconfiguration is taking place.
I'm inclined to close this and go back to the drawing board for a fix with wider applicability. |
This makes things worse, with the kernel initiator connected, once the child and nexus are in a faulted state, even |
Then I guess the best approach here is to simply not retire the last child, but have the initiator timeout/disconnect logic handle it? |
I guess so, I'll rework the PR. |
FWIW, returning early in
|
The assertion occurs due to the nvmf subsystem transitioning from a
paused to an inactive state, which is forbidden. Resume the nexus
first, so that it is in an active state, before unpublishing it, which
also covers destroying a nexus. A nexus in a faulted state already has
no usable replicas so is itself already unusable.
Fixes #549, CAS-549.