New helper tools/adhoc-httpd to facilitate manual HTTP debugging

[mih]

This helper exposes serve_path_via_http via a cmdline utility that is
capable of deploying an ad-hoc instance of the same HTTP server that we
are using for internal testing (with SSL and auth, if desired).

This lowers the threshold for debugging real usage, by avoiding the need
to develop a dedicated test for a scenario. It also makes it possibly to
bypass to additional implications of a datalad test implementation, like
the fake homedir (see #6164, #6160).

[bpoldrack]

Nice!

[codecov]

Codecov Report

Merging #6169 (84c6519) into master (981aa95) will increase coverage by 0.00%.
The diff coverage is 87.50%.

@@           Coverage Diff           @@
##           master    #6169   +/-   ##
=======================================
  Coverage   89.70%   89.71%           
=======================================
  Files         318      318           
  Lines       41896    41900    +4     
=======================================
+ Hits        37584    37589    +5     
+ Misses       4312     4311    -1     

Impacted Files	Coverage Δ
datalad/tests/utils.py	88.92% <87.50%> (-0.06%)	⬇️
datalad/support/tests/test_annexrepo.py	97.93% <0.00%> (+0.14%)	⬆️

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 981aa95...84c6519. Read the comment docs.

[yarikoptic]

sweet -- will come handy.
Note though that PR is not just an addition of a helper but a tune up to the HTTPPath itself with not explained reasoning for the changes there ( I thought that auth was already implemented, so not clear why these extra changes).

[mih]

Note though that PR is not just an addition of a helper but a tune up to the HTTPPath itself with not explained reasoning for the changes there ( I thought that auth was already implemented, so not clear why these extra changes).

yes, good catch. I should have added this to the message. I discovered that the check for proper function of an SSL enabled webserver that also had basic auth enabled would not pass, because the check itself did not perform authentication properly. This is inconsequential for datalad as of now, because I am only adding such tests in a local branch that improves the RIA/ORA testing. But given that the helper can actually trigger the problem, I thought to include the fix here. More comprehensive machinery on this will come.