-
Notifications
You must be signed in to change notification settings - Fork 447
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Improve error handling and logging for Retryable IO Errors #2312
Comments
@VasuDevrani Looks good, you can go ahead to improve this. |
See, the code proposed as solution just prints ERROR in else block. |
I'm wondering if it's a good idea to do that. I prefer letting users determine whether to terminate themself instead of by default N times retry. @PragmaTwice @caipengbo @torwig What do you think? |
Yes, I also don't think it's necessary to set a number of retries. Alternatively, we could provide a |
Out-of curiousity, do we have ability to recovery from error if |
There might be some cases (disk no space?) where it needs to be done externally for the |
Thats a good point, so we should let user decide what to do during this case... |
@VasuDevrani As discussed above, improving the logging message is good, but don't expect to escalate the FATAL error after N time tries. What do you think? |
Yeah I've considered above conversation. I'm learning a bit more about Kvrocks, its internals. will update here with my thoughts after a while. Thanks for following up. |
@VasuDevrani Cool, don't hesitate to raise if you have any ideas. |
I agree with just improving the logging message for now. (should i make continue with a PR?)
Later, if need arise we can go with idea of having |
Sure, you can continue improving the logging message. |
Yes, RESUME is not a high priority. |
Search before asking
Motivation
In server.cc file, the code checks every minute if the storage is in a retryable IO error state and resumes the database if necessary.
Concerns:
Solution
example code to resolve this:
Are you willing to submit a PR?
The text was updated successfully, but these errors were encountered: