New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
HBASE-24361 Make RESTApiClusterManager
more resilient
#1701
HBASE-24361 Make RESTApiClusterManager
more resilient
#1701
Conversation
* sometimes API calls return with null/empty response bodies. thus, wrap all API calls in a retry loop. * calls that submit work in the form of "commands" now retrieve the commandId from successful command submission, and track completion of that command before returning control to calling context. * model CM's process state and use that model to guide state transitions more intelligently. this guards against, for example, the start command failing with an error message like "Role must be stopped". * improvements to logging levels, avoid spamming logs with the side-effects of retries at this and higher contexts. * include references to API documentation, such as it is.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Skimmed. +1 if it works.
// retaining the "Started" state but with the process marked as "unhealthy". | ||
// Instead of blindly issuing the START command, first send a STOP command | ||
// to ensure the START will be accepted. | ||
LOG.debug("Performing start of {} on {}:{}", service, hostname, port); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Good
+ " {} state. Restarting. {}, {}:{}", currentState, service, hostname, port); | ||
performClusterManagerCommand(service, hostname, RoleCommand.RESTART); | ||
return; | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Man
🎊 +1 overall
This message was automatically generated. |
🎊 +1 overall
This message was automatically generated. |
🎊 +1 overall
This message was automatically generated. |
* sometimes API calls return with null/empty response bodies. thus, wrap all API calls in a retry loop. * calls that submit work in the form of "commands" now retrieve the commandId from successful command submission, and track completion of that command before returning control to calling context. * model CM's process state and use that model to guide state transitions more intelligently. this guards against, for example, the start command failing with an error message like "Role must be stopped". * improvements to logging levels, avoid spamming logs with the side-effects of retries at this and higher contexts. * include references to API documentation, such as it is. Signed-off-by: stack <stack@apache.org>
* sometimes API calls return with null/empty response bodies. thus, wrap all API calls in a retry loop. * calls that submit work in the form of "commands" now retrieve the commandId from successful command submission, and track completion of that command before returning control to calling context. * model CM's process state and use that model to guide state transitions more intelligently. this guards against, for example, the start command failing with an error message like "Role must be stopped". * improvements to logging levels, avoid spamming logs with the side-effects of retries at this and higher contexts. * include references to API documentation, such as it is. Signed-off-by: stack <stack@apache.org>
* sometimes API calls return with null/empty response bodies. thus, wrap all API calls in a retry loop. * calls that submit work in the form of "commands" now retrieve the commandId from successful command submission, and track completion of that command before returning control to calling context. * model CM's process state and use that model to guide state transitions more intelligently. this guards against, for example, the start command failing with an error message like "Role must be stopped". * improvements to logging levels, avoid spamming logs with the side-effects of retries at this and higher contexts. * include references to API documentation, such as it is. Signed-off-by: stack <stack@apache.org>
* sometimes API calls return with null/empty response bodies. thus, wrap all API calls in a retry loop. * calls that submit work in the form of "commands" now retrieve the commandId from successful command submission, and track completion of that command before returning control to calling context. * model CM's process state and use that model to guide state transitions more intelligently. this guards against, for example, the start command failing with an error message like "Role must be stopped". * improvements to logging levels, avoid spamming logs with the side-effects of retries at this and higher contexts. * include references to API documentation, such as it is. Signed-off-by: stack <stack@apache.org>
* sometimes API calls return with null/empty response bodies. thus, wrap all API calls in a retry loop. * calls that submit work in the form of "commands" now retrieve the commandId from successful command submission, and track completion of that command before returning control to calling context. * model CM's process state and use that model to guide state transitions more intelligently. this guards against, for example, the start command failing with an error message like "Role must be stopped". * improvements to logging levels, avoid spamming logs with the side-effects of retries at this and higher contexts. * include references to API documentation, such as it is. Signed-off-by: stack <stack@apache.org>
wrap all API calls in a retry loop.
commandId from successful command submission, and track completion
of that command before returning control to calling context.
transitions more intelligently. this guards against, for example,
the start command failing with an error message like "Role must be
stopped".
side-effects of retries at this and higher contexts.