Overhaul ORA remote's HTTP access to RIA stores #5459
Conversation
Codecov Report
@@ Coverage Diff @@
## master #5459 +/- ##
==========================================
- Coverage 90.63% 88.63% -2.01%
==========================================
Files 309 307 -2
Lines 42444 42454 +10
==========================================
- Hits 38471 37628 -843
- Misses 3973 4826 +853
Continue to review full report at Codecov.
|
|
Changed the approach and updated PR description accordingly. |
There was a problem hiding this comment.
Thanks for the changes. I left a few minor comments.
There should be a test though. Given that the original issue is about not failing, in case of an invallid URL such a test requires no particular infrastructure, just a random URL with no RIA store behind it.
Given the more substantial changes in this PR, the title should also be updated.
Thx!
`HTTPRemoteIO` now behaves largely just like SSH and local access, except for being read-only. Remove special casing throughout `RIARemote`. This means, that RIA stores are validated by the ORA special remote over HTTP the same way as over SSH or local access. Therefore providing the ria-layout-version files is now mandatory for HTTP stores as well. This patch should streamline things a bit and closes datalad#5451. Thus making the reconfiguration business in `datalad clone` consistent across protocols. Furthermore, since the version files are now requested and used to construct the URLs, we can simplify server-side scripts dealing wiht archives, etc. There's also hope that this way we can allow for a version file to define a layout directly, instead of declaring a label that needs to be known to released datalad, because all the version is about is locations associated with the .git, annex/objects and archives of datasets in the store. In theory these locations could be declared in a JSON-record or alike in that version file, allowing store maintainers to define their thing freely.
bpoldrack
changed the title
Restructure ORA remote's
HTTPRemoteIOto remove special cases in ORA's code, where HTTP is treated somewhat differrent than SSH or local access.For RIA stores, this leads to the
ria-layout-versionfiles being required to be served as well (which technically wasn't the case before). It also means that the request URLs now use paths for annexed objects according to the specified layout, whereas before we'd always send the same request and it was up to the server to consider its layout and deliver the right thing.HTTP stores are validated on
annex initremote/annex enableremotethe same way they are when usingssh:orfile:scheme URLs. Therefore, autoenabling will fail under the same conditions, makingclone's reconfiguration business consistent across access methods.Closes #5451