Add delete API for torchstore distributed storage #39
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Summary: Implement delete functionality across the torchstore distributed storage system to allow removal of stored tensors and objects. This adds the delete API to all layers of the storage system: - API layer: Added delete() function with proper documentation - Client layer: Implemented distributed delete across storage volumes with asyncio.gather - Controller layer: Added notify_delete() to maintain key index consistency - Storage layer: Added delete() method to storage implementations with proper error handling The delete operation ensures data consistency by removing entries from all storage volumes where the key exists and updating the controller's key index accordingly. Also includes comprehensive test coverage that verifies: - Sequential deletion of multiple tensors - Verification that deleted keys no longer exist - Confirmation that remaining keys are unaffected during incremental deletion - Proper error handling when attempting to access deleted keys Test Plan: Added test_delete() function that: - Tests deletion across multiple storage volumes and processes - Verifies each deletion incrementally (delete one, verify it's gone, verify others remain) - Confirms deleted tensors cannot be retrieved (proper error handling) - Runs with different transport and strategy configurations like existing tests
meta-cla
bot
added
the
CLA Signed
LucasLLC
requested changes
Sep 22, 2025
LucasLLC
approved these changes
Sep 25, 2025
|
Wait for tests to pass please! |
|
@kaiyuan-li |
|
Feel free to merge and I'll fix the CI tests. |
kaiyuan-li
approved these changes
Sep 26, 2025
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Implement delete functionality across the torchstore distributed storage system to allow removal of stored tensors and objects.
Current problems
actor_mesh.slice()seems to accept different arguments. Work around is to usechooseinstead of explicitly choosing a slice.put().See P1955672937 for error logs.
@LucasLLC @kaiyuan-li what is your monarch version?