gitrepo: Direct files to git with annex.gitaddtoannex

[kyleam]

• Update GitRepo.add_ to use annex.gitaddtoannex instead of annex.largefiles to direct files to Git. See the message of second commit for the background.

• Update the associated regression test for a change in git-annex's behavior.

[codecov]

Codecov Report

Merging #5545 (a9a674c) into master (f2f95b3) will increase coverage by 0.06%.
The diff coverage is 100.00%.

❗ Current head a9a674c differs from pull request most recent head 7527b72. Consider uploading reports for the commit 7527b72 to get more accurate results

@@            Coverage Diff             @@
##           master    #5545      +/-   ##
==========================================
+ Coverage   88.52%   88.58%   +0.06%     
==========================================
  Files         296      296              
  Lines       42467    42485      +18     
==========================================
+ Hits        37592    37637      +45     
+ Misses       4875     4848      -27     

Impacted Files	Coverage Δ
datalad/support/gitrepo.py	91.61% <ø> (ø)
datalad/support/tests/test_gitrepo.py	99.80% <100.00%> (ø)
datalad/customremotes/tests/test_archives.py	89.26% <0.00%> (-0.61%)	⬇️
datalad/__init__.py	80.40% <0.00%> (-0.42%)	⬇️
datalad/plugin/export_to_figshare.py	26.54% <0.00%> (-0.17%)	⬇️
datalad/support/digests.py	100.00% <0.00%> (ø)
datalad/interface/tests/test_ls_webui.py	100.00% <0.00%> (ø)
datalad/interface/add_archive_content.py	90.09% <0.00%> (+0.04%)	⬆️
datalad/support/tests/test_annexrepo.py	97.58% <0.00%> (+0.07%)	⬆️
datalad/customremotes/base.py	82.15% <0.00%> (+0.14%)	⬆️
... and 9 more

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update f2f95b3...7527b72. Read the comment docs.

[bpoldrack]

LGTM.

[yarikoptic]

Just to make sure I grasp situation correctly: with this change we will avoid possible race condition in git-annex, but we will change behavior of GitRepo.add if repository was not initiated with e.g. -c text2git or some other procedure which would setup the annex.largefiles in .gitattributes. Given that it was only a single dedicated test which behavior has changed, no other changes in typical use cases (through datalad.save) should be effected, and thus it should not come as a surprise in change in behavior to typical users (unless they use GitRepo.add specifically). Correct?

[kyleam]

Thanks, @bpoldrack and @yarikoptic, for taking a look.

@yarikoptic:

we will change behavior of GitRepo.add if repository was not initiated with e.g. -c text2git or some other procedure which would setup the annex.largefiles in .gitattributes.

No change in behavior is intended. Both the old setting (annex.largefiles=nothing) and new one (annex.gitaddtoannex=false) are used to make GitRepo.add_ always send files to Git regardless of whether the repository has git-annex initialized or has largefiles configured.

Given that it was only a single dedicated test which behavior has changed, no other changes in typical use cases (through datalad.save) should be effected

Right, it's a weird state to get into: an annex repo needs to be instantiated as a GitRepo. This isn't relevant in the typical situation where AnnexRepo is used.

This original change came in ad0158c (BF(v7): gitrepo: Avoid adding files to annex, 2019-09-23), part of gh-3648. This issue was discovered when troubleshooting a crawler test failure, though it looks like it ended up not being the reason for it (or at least not the sole reason).

and thus it should not come as a surprise in change in behavior to typical users (unless they use GitRepo.add specifically). Correct?

As mentioned above, this shouldn't change the behavior of GitRepo.add_.

The change in behavior noted in the PR description is about git-annex 7.20191024 changing its effect on git add. That resulted in the test from ad0158c no longer being an effective regression test; test_gitrepo_add_to_git_with_annex_v7 still passes if you drop the -c annex.largefiles=nothing bit added in ad0158c.

[mih]

Thx for this change!

[yarikoptic]

Thank you @kyleam for the details, and @mih for blessing, let's proceed then!