BF: run_procedure: do not hardcode python executable (fixes gh-3623)

[mih]

Stay within the same Python version to avoid crash or undesired behavior
when PY2 is called from PY3 with missing or outdated PY2 versions of
datalad.

[kyleam]

I suppose we should shlex_quote() sys.executable to guard against poorly named virtual environments with spaces.

This is outside the scope of this PR, but the quoting in these templates looks problematic. I think we should drop the quotes around the placeholders and pass each value through shlex_quote().

[mih]

Good point, thx!

[adswa]

@kyleam I think I took care of this. I will open a new issue about

This is outside the scope of this PR, but the quoting in these templates looks problematic. I think we should drop the quotes around the placeholders and pass each value through shlex_quote().

and will try to PR a fix.

[mih]

Windows is not happy. There is no shlex for windows it seems, and https://stackoverflow.com/questions/33560364/python-windows-parsing-command-lines-with-shlex suggests that doing it correctly is a project on its own. I'd say

from datalad.utils import on_windows
sys.executable if on_windows else shlex_quote(sys.executable)

[kyleam]

Windows is not happy. There is no shlex for windows it seems

Eh, that means we need to deal with this in other spots that use shlex_quote.

Thanks for the updates, @adswa. This would not be a straightforward test for the test suite (and I'm not suggesting you add one), but I thought I should at least see that it works locally if I create a virtual environment with spaces. But I couldn't even get to that point because other things upstream (some outside of DataLad) were failing.

Anyway, outside of windows, shlex_quote at least gives us a better chance of working with weird names and doesn't hurt for non-weird names, so it seems worth keeping.

I'm going to rebase this branch onto 0.11.x because the sys.executable change is worth having there.

[kyleam]

I'm going to rebase this branch onto 0.11.x because the sys.executable change is worth having there.

Done, with a minor spacing tweak to the last commit:

1:  97acf5f04 = 1:  dacb63257 BF: run_procedure: do not hardcode python executable (fixes gh-3623)
2:  94391e1f6 = 2:  b48290564 shlex quote sys.executable
3:  dc3628311 ! 3:  ece364ecc no shlexquote on windows
    @@ datalad/interface/run_procedure.py: def _guess_exec(script_file):
                      'state': state}
          else:
              return {'type': None, 'template': None, 'state': None}
    - 
    --
    - @build_doc
    - class RunProcedure(Interface):
    -     """Run prepared procedures (DataLad scripts) on a dataset

[codecov]

Codecov Report

Merging #3624 into 0.11.x will decrease coverage by 10.79%.
The diff coverage is 74.28%.

@@            Coverage Diff             @@
##           0.11.x    #3624      +/-   ##
==========================================
- Coverage   81.23%   70.44%   -10.8%     
==========================================
  Files         256      273      +17     
  Lines       33819    35511    +1692     
==========================================
- Hits        27473    25015    -2458     
- Misses       6346    10496    +4150

Impacted Files	Coverage Δ
datalad/tests/test_utils.py	96.92% <ø> (+0.4%)	⬆️
datalad/plugin/check_dates.py	40% <ø> (-28.58%)	⬇️
datalad/distribution/add.py	28.19% <ø> (-60.64%)	⬇️
datalad/support/tests/test_path.py	92.3% <ø> (+4.3%)	⬆️
datalad/tests/utils.py	89.29% <ø> (-0.29%)	⬇️
datalad/support/tests/test_repo_save.py	100% <ø> (ø)
datalad/tests/test_direct_mode.py	100% <ø> (+1.85%)	⬆️
datalad/support/tests/test_gitrepo.py	99.88% <ø> (ø)	⬆️
datalad/interface/__init__.py	100% <ø> (ø)	⬆️
datalad/support/tests/test_fileinfo.py	100% <ø> (ø)
... and 218 more

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update d4a3e25...ece364e. Read the comment docs.

[mih]

Thx @adswa and welcome!

[mih]

Hey, @kyleam this fix is needed for a 0.12 deployment tomorrow. Would be good to be able to pull that from its final resting place in master.

[kyleam]

Hey, @kyleam this fix is needed for a 0.12 deployment tomorrow. Would be good to be able to pull that from its final resting place in master.

You'll be able to. I'll merge 0.11.x into master.

[mih]

Perfect, thx!