-
Notifications
You must be signed in to change notification settings - Fork 1.3k
Closed
Labels
Description
Example from experiment using tensorflow ModelCheckpoint files
2020-08-24 12:56:57,888 ERROR: failed to apply experiment changes.
------------------------------------------------------------
Traceback (most recent call last):
File "/Users/pmrowla/git/dvc/dvc/repo/experiments/__init__.py", line 529, in checkout_exp
self.repo.scm.repo.git.apply(tmp, reverse=True)
File "/Users/pmrowla/.virtualenvs/dvc/lib/python3.8/site-packages/git/cmd.py", line 542, in <lambda>
return lambda *args, **kwargs: self._call_process(name, *args, **kwargs)
File "/Users/pmrowla/.virtualenvs/dvc/lib/python3.8/site-packages/git/cmd.py", line 1005, in _call_process
return self.execute(call, **exec_kwargs)
File "/Users/pmrowla/.virtualenvs/dvc/lib/python3.8/site-packages/git/cmd.py", line 822, in execute
raise GitCommandError(command, status, stderr_value, stdout_value)
git.exc.GitCommandError: Cmd('git') failed due to: exit code(1)
cmdline: git apply --reverse /var/folders/s_/j8cg24s945dcvlh77c677f080000gn/T/tmptqvdsgr5
stderr: 'error: missing binary patch data for 'model.hdf5'
error: binary patch does not apply to 'model.hdf5'
error: model.hdf5: patch does not apply'
In this case, model.hdf5 is untracked by both git and dvc. Unless a repro artifact is explicitly gitignored, we should be including it with our experiment/executor output, so we should be using git diff --full-index --binary and git stash --include-untracked when generating our patches
dmpetrov