Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Collection remove times-out while optimization is in process #31

Closed
2 tasks
generall opened this issue Jun 5, 2021 · 1 comment
Closed
2 tasks

Collection remove times-out while optimization is in process #31

generall opened this issue Jun 5, 2021 · 1 comment
Assignees
@generall
Labels
bug Something isn't working

Comments

@generall
Copy link
Member

generall commented Jun 5, 2021 •
edited

Current Behavior

Collection update request awaits for optimizer finishes even if it's a remove request

Steps to Reproduce

  1. Crete collection
  2. Enable indexing or other long optimization process
  3. Try to re-create collection while optimization is still running
  4. Request is not finished until optimization runs

Expected Behavior

Collection removed immediately

Possible Solution

  • Process finished handles
    • Implement a JoinHandle wrapper, which could be checked if it is finished
    • Implement removing of finished handles before each new optimization scheduting
  • Handle optimization shutdown
    • Implement a stop-flag ( as AtomicBool) which could be flipped if it is required to stop optimization
    • Optimization thread should check for the flag value periodically, and if it is flipped - exit the optimization and restore segment states (unwrap proxy, release locks, e.t.c.)
    • Flip the flag on collection drop & on config update, gracefully wait for optimizations to finish
@generall generall added the bug Something isn't working label Jun 5, 2021
@generall generall assigned generall and unassigned generall Jun 5, 2021
@generall generall changed the title Collection remove times opt while optimization is in process Collection remove times-out while optimization is in process Jun 7, 2021
@anveq anveq self-assigned this Dec 9, 2021
@generall generall mentioned this issue Dec 26, 2021
Merged
5 tasks
@generall generall self-assigned this Dec 26, 2021
generall added a commit that referenced this issue Jan 18, 2022
@generall
[WIP] Force optimization stop #31 (#161)
0f91c9a 
* implement checking stop-flag in the optimization routine

* wip: optimization cancel test

* force optimization stop during the construction of vector index

* fix clippy
@generall generall mentioned this issue Jan 24, 2022
Closed
@generall
Copy link
Member Author

generall commented Feb 3, 2022

released in https://github.com/qdrant/qdrant/releases/tag/v0.5.0

@generall generall closed this as completed Feb 3, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@generall generall

Labels
bug Something isn't working
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@generall @anveq