Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

experiments: experiment git patching will fail if patch contains binary files #4451

Closed
pmrowla opened this issue Aug 24, 2020 · 0 comments · Fixed by #4452
Closed

experiments: experiment git patching will fail if patch contains binary files #4451

pmrowla opened this issue Aug 24, 2020 · 0 comments · Fixed by #4452
Assignees
Labels
A: experiments Related to dvc exp bug Did we break something?

Comments

@pmrowla
Copy link
Contributor

pmrowla commented Aug 24, 2020

Example from experiment using tensorflow ModelCheckpoint files

2020-08-24 12:56:57,888 ERROR: failed to apply experiment changes.
------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/pmrowla/git/dvc/dvc/repo/experiments/__init__.py", line 529, in checkout_exp
    self.repo.scm.repo.git.apply(tmp, reverse=True)
  File "/Users/pmrowla/.virtualenvs/dvc/lib/python3.8/site-packages/git/cmd.py", line 542, in <lambda>
    return lambda *args, **kwargs: self._call_process(name, *args, **kwargs)
  File "/Users/pmrowla/.virtualenvs/dvc/lib/python3.8/site-packages/git/cmd.py", line 1005, in _call_process
    return self.execute(call, **exec_kwargs)
  File "/Users/pmrowla/.virtualenvs/dvc/lib/python3.8/site-packages/git/cmd.py", line 822, in execute
    raise GitCommandError(command, status, stderr_value, stdout_value)
git.exc.GitCommandError: Cmd('git') failed due to: exit code(1)
  cmdline: git apply --reverse /var/folders/s_/j8cg24s945dcvlh77c677f080000gn/T/tmptqvdsgr5
  stderr: 'error: missing binary patch data for 'model.hdf5'
error: binary patch does not apply to 'model.hdf5'
error: model.hdf5: patch does not apply'

In this case, model.hdf5 is untracked by both git and dvc. Unless a repro artifact is explicitly gitignored, we should be including it with our experiment/executor output, so we should be using git diff --full-index --binary and git stash --include-untracked when generating our patches

@pmrowla pmrowla self-assigned this Aug 24, 2020
@pmrowla pmrowla added bug Did we break something? A: experiments Related to dvc exp labels Aug 24, 2020
@pmrowla pmrowla added this to To do in DVC 11 - 25 August 2020 via automation Aug 24, 2020
@pmrowla pmrowla moved this from To do to In progress in DVC 11 - 25 August 2020 Aug 24, 2020
@pmrowla pmrowla moved this from In progress to Review in progress in DVC 11 - 25 August 2020 Aug 24, 2020
DVC 11 - 25 August 2020 automation moved this from Review in progress to Done Aug 24, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A: experiments Related to dvc exp bug Did we break something?
Projects
No open projects
Development

Successfully merging a pull request may close this issue.

1 participant