BF+ENH: fixes on the way to get --existing=reconfigure functioning as it should

[yarikoptic]

it is never enough of tests, but new ones revealing bugs are nice ;-)

[codecov-io]

Codecov Report

Merging #774 into master will decrease coverage by 1.97%.
The diff coverage is 83.16%.

@@            Coverage Diff            @@
##           master    #774      +/-   ##
=========================================
- Coverage   89.07%   87.1%   -1.98%     
=========================================
  Files         214     216       +2     
  Lines       19200   19415     +215     
=========================================
- Hits        17102   16911     -191     
- Misses       2098    2504     +406

Impacted Files	Coverage Δ
datalad/distribution/add_sibling.py	85.86% <100%> (-0.31%)	⬇️
datalad/utils.py	90.22% <100%> (+0.02%)	⬆️
datalad/support/sshconnector.py	94.5% <100%> (+0.06%)	⬆️
datalad/tests/utils.py	89.9% <71.79%> (-1.52%)	⬇️
...ribution/create_publication_target_sshwebserver.py	78.68% <80%> (+1.24%)	⬆️
datalad/tests/test_tests_utils.py	96.19% <85.71%> (-0.54%)	⬇️
datalad/distribution/tests/test_target_ssh.py	99.17% <96.42%> (-0.83%)	⬇️
datalad/crawler/nodes/s3.py	32.82% <0%> (-57.26%)	⬇️
datalad/tests/test_s3.py	43.47% <0%> (-56.53%)	⬇️
datalad/downloaders/s3.py	41.73% <0%> (-51.31%)	⬇️
... and 19 more

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update fa6e6af...500f938. Read the comment docs.

[yarikoptic]

@debanjum please review, and test with --existing=reconfigure settings to see if it actually works ok before I run it "in production". Theoretically (since still has TODO failing tests etc) now if I rerun with reconfigure it should also call the hooks so should regenerate all the meta-data etc. So -- you can even look into enhancing that test we were working on there

[coveralls]

Coverage decreased (-4.1%) to 84.959% when pulling 08ca396 on yarikoptic:bf-reconfigure into fa6e6af on datalad:master.

[coveralls]

Coverage decreased (-4.1%) to 84.949% when pulling a19846d on yarikoptic:bf-reconfigure into fa6e6af on datalad:master.

[mih]

LGTM, feel free to merge. Wasn't clear to me whether @debanjum already approved.

[yarikoptic]

Well need to fix-up the test a bit first

[debanjum]

this should be datasets[current_dataset].repo instead of dataset.repo.
Also no need to pass datasets[current_dataset] to create_postupdate_hook function now

[yarikoptic]

On Fri, 09 Sep 2016, Debanjum wrote:

In datalad/distribution/create_publication_target_sshwebserver.py:

@@ -297,6 +302,11 @@ def call(sshurl, target=None, target_dir=None,
lgr.error("Failed to add json creation command to post update hook.\n"
"Error: %s" % exc_str(e))

•        if not only_reconfigure:
  

•            # Initialize annex repo on remote copy if current_dataset is an AnnexRepo
  

•            if isinstance(dataset.repo, AnnexRepo):
  

this should be datasets[current_dataset].repo instead of dataset.repo

great catch! thanks!

[debanjum]

Can't trigger post-update hook on falkor due to the updated permissions on the virtualenv at datalad.org

[yarikoptic]

On Fri, 09 Sep 2016, Debanjum wrote:

Can't trigger post-update hook on falkor due to the updated permissions on
the virtualenv at datalad.org

ok -- now it is available for you as well. but again, for testing we
should just make it happen on smaug. I will create some Dockerfile for
that I guess some time.

[yarikoptic]

I am confused with codecov report -- it looks like previous tests somehow did run s3 and some other downloading related tests and then now they didn't... the same situation reported in other PRs so I don't think it is anything to do with us -- might try to figure it out some time. meanwhile merging this one