-
Notifications
You must be signed in to change notification settings - Fork 1.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Handle Out-Of-Disk gracefully #4108
Comments
/bounty $250 |
💎 $250 bounty • QdrantSteps to solve:
Additional opportunities:
Thank you for contributing to qdrant/qdrant! Add a bounty • Share on socials
|
@generall I would like to solve this issue. Could you please assign me? |
/attempt #4108 |
/attempt #4108 Options |
I think RocksDB employs a similar approach as you propsed, such as this Additionally, it appears that RocksDB allows users to set the disk buffer size as https://github.com/facebook/rocksdb/wiki/RocksDB-Tuning-Guide#flushing-options I wonder if we can use Either way, since RocksDB employs a strategy to maintain a maximum disk usage threshold, I think we should adopt a similar approach. I would love to proceed with this strategy. What do you think? |
💡 @kemkemG0 submitted a pull request that claims the bounty. You can visit your bounty board to reward. |
🎉🎈 @kemkemG0 has been awarded $250! 🎈🎊 |
Is your feature request related to a problem? Please describe.
Currently, there are situations, when Qdrant service can crush if it is not enough disk space to perform update operation.
This is sub-optimal behavior, as it should still be possible to respond to the search requests in this case.
Describe the solution you'd like
Add improve the handling of the situation, where qdrant faces out-of-disk problem. Instead of crashing, it should answer 500 to the user and still be able to process incoming search requests.
Describe alternatives you've considered
Block requests if the disk usage is above some threshold. This would require configuration of the arbitrary threshold and overall less desirable.
Additional context
We prepared an automated test scenario - #4105
Solution of this issue should include a PR into
test/low-disk-tests
branch, which makes the OOD test pass.The text was updated successfully, but these errors were encountered: