dvc repro -e does not work with files which are not tracked by git

[yogi81]

Bug Report

dvc repro -e does not work if the change consists of files which are still not tracked by git (for example new class was created as part of the change)

Also adding the new file to cache with git add did not solve the issue.

In this case experiment starts but fails because the new file is then not found

Please provide information about your setup

Output of dvc version:
1.7.3

$ dvc version

Additional Information (if any):

If applicable, please also provide a --verbose output of the command, eg: dvc add --verbose.

[pmrowla]

@yogi81 can you try running this again but using the -N/--intent-to-add flag with git add and see if that allows you to run the experiment as expected?

$ `git add -N <new_files>` 
$ dvc repro -e ...

This is kind of a known issue due to how we use git diff under the hood when generating experiment patches (git diff normally excludes untracked files).

[yogi81]

Unfortunately this did not help:

python test.py 
hello world
a
(tensorflow) roman@romanbook:~/dvc_test$ dvc repro -e 
What's next?                                                          
------------
- Check out the documentation: https://dvc.org/doc
- Get help and share ideas: https://dvc.org/chat
- Star us on GitHub: https://github.com/iterative/dvc
Running callback stage 'run' with command:                                                                                                                    
        bash -c "python test.py > res.txt" -o res.txt                                                                                                         
Traceback (most recent call last):
  File "test.py", line 1, in <module>
    from data import something
ModuleNotFoundError: No module named 'data'
ERROR: Failed to reproduce experiment 'fc4207a2caf22957f576aba8f4d2ad47c801c60b'
Data and pipelines are up to date.
(tensorflow) roman@romanbook:~/dvc_test$ ls
data.py  dvc.lock  dvc.yaml  __pycache__  res.txt  test.py
(tensorflow) roman@romanbook:~/dvc_test$ **git add -N data.py**
(tensorflow) roman@romanbook:~/dvc_test$ dvc repro -e 
ERROR: unexpected error - Cmd('git') failed due to: exit code(1)      
  cmdline: git apply /tmp/tmpd9gi9l61
  stderr: 'error: data.py: No such file or directory'

Having any troubles? Hit us up at https://dvc.org/support, we are always happy to help!
DVC version: 1.7.3 (pip)
---------------------------------
Platform: Python 3.6.9 on Linux-5.6.16-xanmod1-x86_64-with-debian-buster-sid
Supports: http, https, ssh
Cache types: <https://error.dvc.org/no-dvc-cache>
Workspace directory: ext4 on /dev/nvme0n1p3
Repo: dvc, git
(tensorflow) roman@romanbook:~/dvc_test$ git status
On branch master
Changes not staged for commit:
  (use "git add <file>..." to update what will be committed)
  (use "git checkout -- <file>..." to discard changes in working directory)

        new file:   data.py
        modified:   test.py

Untracked files:
  (use "git add <file>..." to include in what will be committed)

        __pycache__/

(tensorflow) roman@romanbook:~/dvc_test$ cat dvc.yaml 
stages:
  run:
    cmd: bash -c "python test.py > res.txt" -o res.txt

[yogi81]

@pmrowla

[pmrowla]

As mentioned before, this is a known issue due to how we use git diff internally. In the meantime, if you need to make code changes which add entire new files, the suggested workaround would be to commit the code changes in git (you may want to consider using a temporary branch for this).