New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat: improve reliability & usability of VFolder removal API #1884
Open
fregataa
wants to merge
71
commits into
main
Choose a base branch
from
feature/improve-vf-purge-using-bgtask
base: main
Could not load branches
Branch not found: {{ refName }}
Could not load tags
Nothing to show
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
github-actions
bot
added
comp:manager
Related to Manager component
size:L
100~500 LoC
labels
Feb 4, 2024
github-actions
bot
added
the
comp:storage-proxy
Related to Storage proxy component
label
Feb 6, 2024
Need to limit the concurrent deletion jobs for each vfolder to one and the maximum concurrent number of vfolder deletion jobs for each storage volume (configurable, defaults to 20) as a storage-proxy configuration. |
fregataa
changed the title
feature: improve VFolder purge using background task
feature: improve VFolder deletion using background task
Feb 24, 2024
fregataa
changed the title
feature: improve VFolder deletion using background task
enhance: VFolder deletion using background task
Feb 24, 2024
kyujin-cho
requested changes
Apr 16, 2024
…gtask' into feature/improve-vf-purge-using-bgtask
github-actions
bot
added
area:docs
Documentations
comp:common
Related to Common component
comp:cli
Related to CLI component
labels
Apr 29, 2024
fregataa
added
urgency:4
As soon as feasible, implementation is essential.
and removed
urgency:blocker
IT SHOULD BE RESOLVED BEFORE NEXT RELEASE!
labels
May 8, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Labels
area:docs
Documentations
comp:cli
Related to CLI component
comp:client
Related to Client component
comp:common
Related to Common component
comp:manager
Related to Manager component
comp:storage-proxy
Related to Storage proxy component
size:L
100~500 LoC
type:enhance
Enhance component, behavior, internals without user-facing features
urgency:4
As soon as feasible, implementation is essential.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
resolves #1797
blocked by #1892we do not need to back-port Graphene relay node part to 23.09.We are now able to backport relay node impl to 23.09Let's not backport this PR
asyncio.Semaphore
to limit too many concurrent deletion task and File-lock to limit concurrent deletion on the same directoryAddmove_to_trash_vfolder
,restore_vfolder
,delete_forever_vfolder
andpurge_vfolder
GraphQL endpointsAdding GQL APIs will be handled in another PR
Checklist: (if applicable)
ai.backend.test
📚 Documentation preview 📚: https://sorna--1884.org.readthedocs.build/en/1884/
📚 Documentation preview 📚: https://sorna-ko--1884.org.readthedocs.build/ko/1884/