Skip to content

experiments: experiment git patching will fail if patch contains binary files #4451

@pmrowla

Description

@pmrowla

Example from experiment using tensorflow ModelCheckpoint files

2020-08-24 12:56:57,888 ERROR: failed to apply experiment changes.
------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/pmrowla/git/dvc/dvc/repo/experiments/__init__.py", line 529, in checkout_exp
    self.repo.scm.repo.git.apply(tmp, reverse=True)
  File "/Users/pmrowla/.virtualenvs/dvc/lib/python3.8/site-packages/git/cmd.py", line 542, in <lambda>
    return lambda *args, **kwargs: self._call_process(name, *args, **kwargs)
  File "/Users/pmrowla/.virtualenvs/dvc/lib/python3.8/site-packages/git/cmd.py", line 1005, in _call_process
    return self.execute(call, **exec_kwargs)
  File "/Users/pmrowla/.virtualenvs/dvc/lib/python3.8/site-packages/git/cmd.py", line 822, in execute
    raise GitCommandError(command, status, stderr_value, stdout_value)
git.exc.GitCommandError: Cmd('git') failed due to: exit code(1)
  cmdline: git apply --reverse /var/folders/s_/j8cg24s945dcvlh77c677f080000gn/T/tmptqvdsgr5
  stderr: 'error: missing binary patch data for 'model.hdf5'
error: binary patch does not apply to 'model.hdf5'
error: model.hdf5: patch does not apply'

In this case, model.hdf5 is untracked by both git and dvc. Unless a repro artifact is explicitly gitignored, we should be including it with our experiment/executor output, so we should be using git diff --full-index --binary and git stash --include-untracked when generating our patches

Metadata

Metadata

Assignees

Labels

A: experimentsRelated to dvc expbugDid we break something?

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions