Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We鈥檒l occasionally send you account related emails.

Already on GitHub? Sign in to your account

Don't use download_url in ORA #6609

Merged
merged 1 commit into from
Apr 6, 2022
Merged

Conversation

bpoldrack
Copy link
Member

@bpoldrack bpoldrack commented Apr 1, 2022

Separating reading provider configs from downloading a key, since this
was a per-key operation unnecessarily slowing down HTTP downloads.

Magically, this also enables per-key progress bars (addressing datalad/datalad-ria#13 for the HTTP case). This seems to depend on the persistence of the Provider and/or Downloader instance across keys. Same call path does not work, when Providers are read anew from the configs on a per-key basis (as the implementation using download_url did).

Changelog

馃悰 Bug Fixes

  • ORA special remote didn't show per-file progress bars when downloading over HTTP

馃挮 Enhancements and new features

  • Removed a per-file overhead cost for ORA when downloading over HTTP

@bpoldrack bpoldrack marked this pull request as ready for review April 1, 2022 13:13
Separating reading provider configs from downloading a key, since this
was a per-key operation unneccessarily slowing down HTTP downloads.

This also enables per-key progress bars when downloading over HTTP.
@codeclimate
Copy link

codeclimate bot commented Apr 4, 2022

Code Climate has analyzed commit 2d748a7 and detected 2 issues on this pull request.

Here's the issue category breakdown:

Category Count
Security 2

View more on Code Climate.

@bpoldrack bpoldrack added semver-patch Increment the patch version when merged team-remotes (special) remote implementations (https://github.com/datalad/datalad/issues/6365) labels Apr 6, 2022
@codecov
Copy link

codecov bot commented Apr 6, 2022

Codecov Report

Merging #6609 (2d748a7) into master (97f648e) will increase coverage by 0.01%.
The diff coverage is 100.00%.

@@            Coverage Diff             @@
##           master    datalad/datalad#6609      +/-   ##
==========================================
+ Coverage   91.14%   91.15%   +0.01%     
==========================================
  Files         350      353       +3     
  Lines       44232    44552     +320     
==========================================
+ Hits        40316    40613     +297     
- Misses       3916     3939      +23     
Impacted Files Coverage 螖
datalad/distributed/ora_remote.py 80.72% <100.00%> (+0.66%) 猬嗭笍
datalad/support/entrypoints.py 47.36% <0.00%> (-13.51%) 猬囷笍
datalad/interface/common_cfg.py 96.59% <0.00%> (-3.41%) 猬囷笍
datalad/cmd.py 92.97% <0.00%> (-1.12%) 猬囷笍
datalad/core/local/tests/test_status.py 97.79% <0.00%> (-0.68%) 猬囷笍
datalad/core/local/tests/test_save.py 97.95% <0.00%> (-0.17%) 猬囷笍
datalad/support/annexrepo.py 91.19% <0.00%> (-0.08%) 猬囷笍
datalad/support/tests/test_annexrepo.py 97.92% <0.00%> (-0.07%) 猬囷笍
datalad/coreapi.py 100.00% <0.00%> (酶)
datalad/support/tests/test_extensions.py 100.00% <0.00%> (酶)
... and 10 more

Continue to review full report at Codecov.

Legend - Click here to learn more
螖 = absolute <relative> (impact), 酶 = not affected, ? = missing data
Powered by Codecov. Last update 97f648e...2d748a7. Read the comment docs.

@bpoldrack bpoldrack merged commit 789336f into datalad:master Apr 6, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
semver-patch Increment the patch version when merged team-remotes (special) remote implementations (https://github.com/datalad/datalad/issues/6365)
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

1 participant