Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

gc: add dry option and log num removed objs #9952

Merged
merged 4 commits into from
Sep 30, 2023
Merged

gc: add dry option and log num removed objs #9952

merged 4 commits into from
Sep 30, 2023

Conversation

dberenbaum
Copy link
Collaborator

@dberenbaum dberenbaum commented Sep 15, 2023

Requires iterative/dvc-data#441 (and need to add a version bump here).

Closes #9330

Adds gc --dry option and reports the number of files removed for each cache.

Before this PR (a file is removed here but it look like nothing happened):

$ dvc gc -w
WARNING: This will remove all cache except items used in the workspace of the current repo.
Are you sure you want to proceed? [y/n]: y
No unused 'local' cache to remove.
No unused 'legacy' cache to remove.

After this PR:

$ dvc gc -w
WARNING: This will remove all cache except items used in the workspace of the current repo.
Are you sure you want to proceed? [y/n]: y
Removed 1 objects from repo cache.
No unused 'local' cache to remove.
No unused 'legacy' cache to remove.

assert set(odb.oids_exist([*good_cache, *bad_cache])) == good_cache
assert set(odb.all()) == good_cache
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This assertion was passing even before running gc. It wasn't actually retrieving oids from bad_cache because it was checking if they exist in the fs, and here they only exist in the odb.

@codecov
Copy link

codecov bot commented Sep 28, 2023

Codecov Report

All modified lines are covered by tests ✅

Files Coverage Δ
dvc/commands/gc.py 83.87% <100.00%> (+0.26%) ⬆️
dvc/repo/gc.py 92.10% <100.00%> (+0.21%) ⬆️
tests/func/test_gc.py 100.00% <100.00%> (ø)
tests/unit/command/test_gc.py 100.00% <ø> (ø)

📢 Thoughts on this report? Let us know!.

@dberenbaum dberenbaum marked this pull request as ready for review September 28, 2023 21:50
@dberenbaum dberenbaum requested a review from a team September 28, 2023 21:50
dvc/repo/gc.py Outdated Show resolved Hide resolved
@efiop efiop enabled auto-merge (squash) September 30, 2023 01:56
@efiop efiop merged commit bc1a700 into main Sep 30, 2023
21 checks passed
@efiop efiop deleted the gc-ui branch September 30, 2023 02:33
@sjawhar
Copy link
Contributor

sjawhar commented Oct 5, 2023

Oh my goodness, thank you for this! 🙏

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Dry run option for dvc gc
3 participants