-
Notifications
You must be signed in to change notification settings - Fork 1.3k
Description
If the dvc.yaml files are corrupted in certain revisions, dvc just fails and there's no way to skip that.
@courentin raised #3885 for this and again on #5037 to specifically address this.
For most parts of DVC, it might be okay and we can also teach params/metrics/plots to cope with failures in certain revisions.
But, for other commands such as push/pull/gc, dvc might just fail, even if we skip those failing commits, it'd be better not to skip and ask users to make their intentions clearer to provide stronger "data versioning" requirements.
And, for gc, it's even dangerous, and for push/pull/gc`, it might just be confusing.
So, coming to my proposal, it'd be better if we could provide a way for users to tell DVC to skip certain commits.
One of the ways would be to provide --ignore-revs or similarly named flags in push/pull/gc, etc.
But, it might not be always easier to use those flags and might be cumbersome especially in CI.
To solve this, we can provide a config (eg: ignore_revsFile) where you could provide a file with list of revisions to ignore:
# .dvc/config
[core]
ignore_revsFile = .dvc-ignore-revs
# .dvc-ignore-revs
# list of revisions for the DVC to ignore
a12ef
12345
The file will always be read from the latest checked out commit.
If the file is cumbersome, we could instead provide a way of adding a list of revs in .dvc/config as they are always read from the workspace.