ENH: enable recursive glob (**) for run

[kimsin98]

Closes #6241

[adswa]

Awesome! Thanks a lot! I just quickly glanced at the test failures for now, I believe they stem from a misspelled key in the examples - left a quick suggestion for that.

[adswa]

added semver-patch for the semver test, please change or say something if someone disagrees :)

[codecov]

Codecov Report

Merging #6262 (00eee07) into master (b7ed22e) will decrease coverage by 3.40%.
The diff coverage is 100.00%.

❗ Current head 00eee07 differs from pull request most recent head 22bf7fa. Consider uploading reports for the commit 22bf7fa to get more accurate results

@@            Coverage Diff             @@
##           master    #6262      +/-   ##
==========================================
- Coverage   89.62%   86.21%   -3.41%     
==========================================
  Files         323      323              
  Lines       41984    41979       -5     
==========================================
- Hits        37629    36194    -1435     
- Misses       4355     5785    +1430     

Impacted Files	Coverage Δ
datalad/core/local/run.py	97.78% <ø> (ø)
datalad/support/globbedpaths.py	100.00% <100.00%> (ø)
datalad/support/tests/test_globbedpaths.py	95.06% <100.00%> (-4.94%)	⬇️
datalad/version.py	0.00% <0.00%> (-100.00%)	⬇️
datalad/plugin/wtf.py	0.00% <0.00%> (-100.00%)	⬇️
datalad/plugin/addurls.py	0.00% <0.00%> (-100.00%)	⬇️
datalad/interface/clean.py	0.00% <0.00%> (-100.00%)	⬇️
datalad/interface/rerun.py	0.00% <0.00%> (-100.00%)	⬇️
datalad/plugin/no_annex.py	0.00% <0.00%> (-100.00%)	⬇️
datalad/interface/unlock.py	0.00% <0.00%> (-100.00%)	⬇️
... and 166 more

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update b7ed22e...22bf7fa. Read the comment docs.

[adswa]

This works nicely for me! Thanks a lot! I will wait a bit to give others a chance to spot things I may have missed, though :)

[mih]

Great, thx!

[yarikoptic]

Looks great! My only "fear"s are

• that it might change reproducibility of prior runs. E.g. if someone used ** just by doubletyping *, which resulted in non-recursive match, now behavior would change, so rerun might find more. edit: but I think it is Ok as long as we clearly document such a change in CHANGELOG for the release. Nothing should stop the progress! (a complimentary -- there could be yet another config option to disable recursive=True)
• in presence of subdatasets, something like sub/**/*.nii.gz could find even git annex keys themselves within sub/.git/annex/objects . But it seems to be not the case, in that ** I guess does not include dot dirs:

$> python3 -c "from glob import glob; print([x for x in glob('ds*/.git/**/*.nii.gz', recursive=True) if 'annex' in x])"
['ds003027/.git/annex/objects/ZJ/k5/MD5E-s89045275--1625ff12d9c2d4e35f50df32601a7bb2.nii.gz', 'ds003027/.git/annex/objects/ZJ/k5/MD5E-s89045275--1625ff12d9c2d4e35f50df32601a7bb2.nii.gz/MD5E-s89045275--1625ff12d9c2d4e35f50df32601a7bb2.nii.gz']

$> python3 -c "from glob import glob; print([x for x in glob('ds*/**/*.nii.gz', recursive=True) if 'annex' in x])"     
[]

so I guess we should be safe on that end.

[mih]

OK, let's proceed! A changelog will come.

[adswa]

Its mentioned in the handbook now as well 👍

[mih]

Great, thx!