`diff` reports annexed file to be in git

[mih]

Here is the test failure:

https://github.com/mih/datalad/commit/396864502cedebc214d9dd0a48e44d0797d82d10/checks#step:8:275

It essentially compares the output of

ds.diff(recursive=True, annex='all')

{'action': 'diff',
 'backend': 'MD5E',
 'bytesize': 5,
 'gitshasum': 'e90d4b712a53d0af1707d83a8a727d610cf4b41c',
 'has_content': True,
 'humansize': '5 B',
 'key': 'MD5E-s5--275876e34cf609db118f3d84b799a790.txt',
 'keyname': '275876e34cf609db118f3d84b799a790.txt',
 'mtime': 'unknown',
 'objloc': 'C:\\Users\\RUNNER~1\\AppData\\Local\\Temp\\datalad_temp_pi1jh5dh\\.git\\annex\\objects\\f33\\94b\\MD5E-s5--275876e34cf609db118f3d84b799a790.txt\\MD5E-s5--275876e34cf609db118f3d84b799a790.txt',
 'parentds': 'C:\\Users\\RUNNER~1\\AppData\\Local\\Temp\\datalad_temp_pi1jh5dh',
 'path': 'C:\\Users\\RUNNER~1\\AppData\\Local\\Temp\\datalad_temp_pi1jh5dh\\subdir\\annexed_file.txt',
 'prev_backend': 'MD5E',
 'prev_bytesize': '5',
 'prev_gitshasum': 'e90d4b712a53d0af1707d83a8a727d610cf4b41c',
 'prev_hashdirlower': 'f33\\94b\\',
 'prev_hashdirmixed': '7p\\gp\\',
 'prev_humansize': '5 B',
 'prev_key': 'MD5E-s5--275876e34cf609db118f3d84b799a790.txt',
 'prev_keyname': '275876e34cf609db118f3d84b799a790.txt',
 'prev_mtime': 'unknown',
 'refds': 'C:\\Users\\RUNNER~1\\AppData\\Local\\Temp\\datalad_temp_pi1jh5dh',
 'state': 'clean',
 'status': 'ok',
 'type': 'file'}

with that of ds.diff(path='.', recursive=True, annex='all')

{'action': 'diff',
 'bytesize': 61,
 'gitshasum': 'e90d4b712a53d0af1707d83a8a727d610cf4b41c',
 'parentds': 'C:\\Users\\RUNNER~1\\AppData\\Local\\Temp\\datalad_temp_pi1jh5dh',
 'path': 'C:\\Users\\RUNNER~1\\AppData\\Local\\Temp\\datalad_temp_pi1jh5dh\\subdir\\annexed_file.txt',
 'prev_backend': 'MD5E',
 'prev_bytesize': '5',
 'prev_gitshasum': 'e90d4b712a53d0af1707d83a8a727d610cf4b41c',
 'prev_hashdirlower': 'f33\\94b\\',
 'prev_hashdirmixed': '7p\\gp\\',
 'prev_humansize': '5 B',
 'prev_key': 'MD5E-s5--275876e34cf609db118f3d84b799a790.txt',
 'prev_keyname': '275876e34cf609db118f3d84b799a790.txt',
 'prev_mtime': 'unknown',
 'refds': 'C:\\Users\\RUNNER~1\\AppData\\Local\\Temp\\datalad_temp_pi1jh5dh',
 'state': 'clean',
 'status': 'ok',
 'type': 'file'}

the latter reports that the present state of the file is to be checked into git. Which is nonsense, as the two diff commands run right after each other. Also:

>>> len('/annex/objects/MD5E-s5--275876e34cf609db118f3d84b799a790.txt\n')
61

makes me think that it misinterprets an annex pointer file.

The whole thing is happening on windows, but I cannot replicate it on a real win10 box.

[mih]

This things is a bit confusing. On another windows box I see this:

(base) C:\Users\mih\AppData\Local\Temp\datalad_temp_56wrmww0>datalad -f json_pp status subdir\annexed_file.txt --annex
{
  "action": "status",
  "backend": "MD5E",
  "bytesize": 5,
  "gitshasum": "e90d4b712a53d0af1707d83a8a727d610cf4b41c",
  "hashdirlower": "f33\\94b\\",
  "hashdirmixed": "7p\\gp\\",
  "humansize": "5 B",
  "key": "MD5E-s5--275876e34cf609db118f3d84b799a790.txt",
  "keyname": "275876e34cf609db118f3d84b799a790.txt",
  "mtime": "unknown",
  "parentds": "C:\\Users\\mih\\AppData\\Local\\Temp\\datalad_temp_56wrmww0",
  "path": "C:\\Users\\mih\\AppData\\Local\\Temp\\datalad_temp_56wrmww0\\subdir\\annexed_file.txt",
  "prev_gitshasum": "e90d4b712a53d0af1707d83a8a727d610cf4b41c",
  "refds": "C:\\Users\\mih\\AppData\\Local\\Temp\\datalad_temp_56wrmww0",
  "state": "clean",
  "status": "ok",
  "type": "file"
}

(base) C:\Users\mih\AppData\Local\Temp\datalad_temp_56wrmww0>datalad -f json_pp status subdir\annexed_file.txt
{
  "action": "status",
  "bytesize": 61,
  "gitshasum": "e90d4b712a53d0af1707d83a8a727d610cf4b41c",
  "parentds": "C:\\Users\\mih\\AppData\\Local\\Temp\\datalad_temp_56wrmww0",
  "path": "C:\\Users\\mih\\AppData\\Local\\Temp\\datalad_temp_56wrmww0\\subdir\\annexed_file.txt",
  "prev_gitshasum": "e90d4b712a53d0af1707d83a8a727d610cf4b41c",
  "refds": "C:\\Users\\mih\\AppData\\Local\\Temp\\datalad_temp_56wrmww0",
  "state": "clean",
  "status": "ok",
  "type": "file"
}

And it clearly should not state 61 bytes, because:

(base) C:\Users\mih\AppData\Local\Temp\datalad_temp_56wrmww0>type subdir\annexed_file.txt
dummy

I suspect something gets confused by cat-fileing from the wrong branch:

(base) C:\Users\mih\AppData\Local\Temp\datalad_temp_56wrmww0>git branch
* adjusted/master(unlocked)
  git-annex
  master

[mih]

So if we consider the bytesize report to be a problem, it is not windows-specific:

(datalad3-dev) mih@meiner /tmp/datalad_temp_test_path_diffrhab1czt (git)-[master] % datalad -f json_pp status subdir/annexed_file.txt
{
  "action": "status",
  "bytesize": 119,
  "gitshasum": "a79f797e1ce00f414131f7e84f47049c4c5c5f1a",
  "parentds": "/tmp/datalad_temp_test_path_diffrhab1czt",
  "path": "/tmp/datalad_temp_test_path_diffrhab1czt/subdir/annexed_file.txt",
  "prev_gitshasum": "a79f797e1ce00f414131f7e84f47049c4c5c5f1a",
  "refds": "/tmp/datalad_temp_test_path_diffrhab1czt",
  "state": "clean",
  "status": "ok",
  "type": "file"
}
(datalad3-dev) mih@meiner /tmp/datalad_temp_test_path_diffrhab1czt (git)-[master] % datalad -f json_pp status subdir/annexed_file.txt --annex
{
  "action": "status",
  "backend": "MD5E",
  "bytesize": 5,
  "gitshasum": "a79f797e1ce00f414131f7e84f47049c4c5c5f1a",
  "hashdirlower": "f33/94b/",
  "hashdirmixed": "7p/gp/",
  "humansize": "5 B",
  "key": "MD5E-s5--275876e34cf609db118f3d84b799a790.txt",
  "keyname": "275876e34cf609db118f3d84b799a790.txt",
  "mtime": "unknown",
  "parentds": "/tmp/datalad_temp_test_path_diffrhab1czt",
  "path": "/tmp/datalad_temp_test_path_diffrhab1czt/subdir/annexed_file.txt",
  "prev_gitshasum": "a79f797e1ce00f414131f7e84f47049c4c5c5f1a",
  "refds": "/tmp/datalad_temp_test_path_diffrhab1czt",
  "state": "clean",
  "status": "ok",
  "type": "file"
}
% cat /tmp/datalad_temp_test_path_diffrhab1czt/subdir/annexed_file.txt
dummy%                                                                                       

[adswa]

does anyone remember which test that was? The link to the workflow run is dead.

[adswa]

I think I found it: test_path_diff. And the test seems to pass again! #7191