Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

dvc add ignore unprotected files that have not been modified #1655

Closed
rmcavoy opened this issue Feb 22, 2019 · 5 comments

Comments

2 participants
@rmcavoy
Copy link

commented Feb 22, 2019

DVC version 0.29.0 installed via pip on Linux

Following the sequence of commands
dvc unprotect data.dat
(modify data.dat)
dvc add data.dat

Correctly replaces dvc.dat with the correct kind of file link
but
dvc unprotect data.dat
(not modifying)
dvc add data.dat

Causes add to skip data.dat because it is already cached and leaves the file on disk without linking. Presumably the ideal behavior is to have dvc add check what kind of link the cache should use and update the file to that link even if the file was not modified. This way modified and unmodified data have the same behavior.

Currently the workaround is to rm the data file and checkout which is somewhat awkward.

@efiop efiop added the enhancement label Feb 22, 2019

@efiop

This comment has been minimized.

Copy link
Member

commented Feb 22, 2019

Looks like it is a side effect of https://github.com/iterative/dvc/blob/0.29.0/dvc/stage.py#L423 . We should probably call save() on that stage if we see that it is already cached, just to ensure that all the cache is optimized.

@rmcavoy

This comment has been minimized.

Copy link
Author

commented Feb 25, 2019

@efiop I am having problems with the work around of rm'ing the file and checking it out. If I have more than one such file, "dvc checkout" will only checkout file1.dat: and then quit with an error message "Error: unexpected error - [errno 1] Operation not permitted: file2.dat". If I call dvc checkout n'times where n is the number of files I can get all of them but this is very very tedious.

@efiop

This comment has been minimized.

Copy link
Member

commented Apr 4, 2019

@rmcavoy So sorry for the delay! Notification got lost in the flow 🙁 . Does that issue still persist?

@efiop efiop self-assigned this Apr 10, 2019

@efiop efiop added this to To do in Weekly tasks via automation Apr 10, 2019

@efiop efiop removed this from To do in Weekly tasks Apr 10, 2019

@efiop efiop assigned efiop and unassigned efiop Apr 10, 2019

@efiop efiop added this to To do in Weekly tasks via automation Apr 10, 2019

@efiop efiop moved this from To do to In progress in Weekly tasks Apr 10, 2019

efiop added a commit to efiop/dvc that referenced this issue Apr 16, 2019

remote: generalize file/dir handling
Fixes iterative#1654
Fixes iterative#1655

Signed-off-by: Ruslan Kuprieiev <ruslan@iterative.ai>

efiop added a commit to efiop/dvc that referenced this issue Apr 16, 2019

remote: generalize file/dir handling
Fixes iterative#1654
Fixes iterative#1655

Signed-off-by: Ruslan Kuprieiev <ruslan@iterative.ai>

efiop added a commit to efiop/dvc that referenced this issue Apr 17, 2019

remote: generalize file/dir handling
Fixes iterative#1654
Fixes iterative#1655

Signed-off-by: Ruslan Kuprieiev <ruslan@iterative.ai>

efiop added a commit to efiop/dvc that referenced this issue Apr 19, 2019

remote: generalize file/dir handling
Fixes iterative#1654
Fixes iterative#1655

Signed-off-by: Ruslan Kuprieiev <ruslan@iterative.ai>

efiop added a commit to efiop/dvc that referenced this issue Apr 19, 2019

remote: generalize file/dir handling
Fixes iterative#1654
Fixes iterative#1655

Signed-off-by: Ruslan Kuprieiev <ruslan@iterative.ai>

efiop added a commit to efiop/dvc that referenced this issue Apr 20, 2019

remote: generalize file/dir handling
Fixes iterative#1654
Fixes iterative#1655

Signed-off-by: Ruslan Kuprieiev <ruslan@iterative.ai>

efiop added a commit to efiop/dvc that referenced this issue Apr 20, 2019

remote: generalize file/dir handling
Fixes iterative#1654
Fixes iterative#1655

Signed-off-by: Ruslan Kuprieiev <ruslan@iterative.ai>

efiop added a commit to efiop/dvc that referenced this issue Apr 22, 2019

remote: generalize file/dir handling
Fixes iterative#1654
Fixes iterative#1655

Signed-off-by: Ruslan Kuprieiev <ruslan@iterative.ai>

efiop added a commit to efiop/dvc that referenced this issue Apr 24, 2019

remote: generalize file/dir handling
Fixes iterative#1654
Fixes iterative#1655

Signed-off-by: Ruslan Kuprieiev <ruslan@iterative.ai>

efiop added a commit to efiop/dvc that referenced this issue Apr 24, 2019

remote: generalize file/dir handling
Fixes iterative#1654
Fixes iterative#1655

Signed-off-by: Ruslan Kuprieiev <ruslan@iterative.ai>

@efiop efiop closed this in 00cfb04 Apr 24, 2019

Weekly tasks automation moved this from In progress to Done Apr 24, 2019

@efiop

This comment has been minimized.

Copy link
Member

commented Apr 24, 2019

Closed accidentally.

@efiop efiop reopened this Apr 24, 2019

Weekly tasks automation moved this from Done to In progress Apr 24, 2019

efiop added a commit to efiop/dvc that referenced this issue Apr 24, 2019

stage: commit cached stage to avoid data duplication
Fixes iterative#1655

Signed-off-by: Ruslan Kuprieiev <ruslan@iterative.ai>

@efiop efiop closed this in #1925 Apr 24, 2019

Weekly tasks automation moved this from In progress to Done Apr 24, 2019

@efiop

This comment has been minimized.

Copy link
Member

commented May 12, 2019

@rmcavoy Fix for this has been released in 0.40.1. Please feel free to upgrade 🙂 Thanks for the feedback!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.