Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We鈥檒l occasionally send you account related emails.

Already on GitHub? Sign in to your account

Factor out actual data transfer in push #6618

Conversation

christian-monch
Copy link
Contributor

@christian-monch christian-monch commented Apr 7, 2022

This commit delegates the actual data transfer in
push to a new method: _transfer_data. This is done
to simplify monkey-patching via "datalad_next", in
order to implement export to git-annex special remotes
with type webdav and exporttree enabled.

This is a first step to address issue #3127

Changelog

馃悰 Bug Fixes

馃彔 Internal

  • factor out actual data transfer in push, in order to make it "patchable".

@christian-monch christian-monch added the semver-internal Changes only affect the internal API label Apr 7, 2022
@codecov
Copy link

codecov bot commented Apr 7, 2022

Codecov Report

Merging #6618 (33836bc) into maint (1389fe8) will increase coverage by 0.02%.
The diff coverage is 100.00%.

@@            Coverage Diff             @@
##            maint    #6618      +/-   ##
==========================================
+ Coverage   91.14%   91.17%   +0.02%     
==========================================
  Files         353      353              
  Lines       44484    44496      +12     
==========================================
+ Hits        40544    40568      +24     
+ Misses       3940     3928      -12     
Impacted Files Coverage 螖
datalad/core/distributed/push.py 92.54% <100.00%> (+0.08%) 猬嗭笍
datalad/distributed/create_sibling_ghlike.py 64.64% <100.00%> (+0.39%) 猬嗭笍
datalad/tests/test_utils.py 97.78% <100.00%> (+0.01%) 猬嗭笍
datalad/utils.py 85.51% <100.00%> (+0.02%) 猬嗭笍
datalad/support/tests/test_annexrepo.py 97.87% <0.00%> (+0.06%) 猬嗭笍
datalad/tests/utils.py 89.44% <0.00%> (+0.63%) 猬嗭笍
datalad/support/sshconnector.py 87.19% <0.00%> (+0.69%) 猬嗭笍
datalad/core/local/tests/test_status.py 97.79% <0.00%> (+2.20%) 猬嗭笍

Continue to review full report at Codecov.

Legend - Click here to learn more
螖 = absolute <relative> (impact), 酶 = not affected, ? = missing data
Powered by Codecov. Last update 1389fe8...33836bc. Read the comment docs.

@christian-monch christian-monch changed the title WIP: Factor out actual data transfer in push WIP: POC: Factor out actual data transfer in push Apr 8, 2022
@christian-monch christian-monch changed the title WIP: POC: Factor out actual data transfer in push WIP: Factor out actual data transfer in push Apr 8, 2022
Copy link
Member

@yarikoptic yarikoptic left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since we have nothing better ATM, I guess it is Ok, although I am afraid could be quite fragile "long term" -- extensions could then fight over who monkey patches first, leading to odd/hard to troubleshoot behaviors etc.

@yarikoptic
Copy link
Member

yarikoptic commented Apr 19, 2022

if it isn't WiP, please adjust the title.

edit: I've done it

@yarikoptic yarikoptic changed the title WIP: Factor out actual data transfer in push Factor out actual data transfer in push Apr 19, 2022
@yarikoptic
Copy link
Member

@christian-monch Let's rebase this against maint. Since it is just factoring out, it is IMHO ok. While at it please annotate with the decorator just merged into maint https://github.com/datalad/datalad/pull/6627/files#diff-124eede0287a62652d5fa856d4b11d371ac5484cab67bbc096a0b6656e6f8555R155 . If all kosher with CI after rebase etc, we would then mark this PR for release.

@christian-monch christian-monch force-pushed the issue-3127-annex-export-datalad-part branch from 7d38f28 to a097c98 Compare April 20, 2022 08:08
This commit delegates the actual data transfer in
push to a new method: _transfer_data. This is done
to simplify monkey-patching via "datalad_next", in
order to implement export to git-annex special remotes
with type webdav and exporttree enabled.
This commit add the 'todo_interface_extensions' decorator
that helps to mark functions and methods that can be modified
by monkey-patching via, for example, datalad_next.
@christian-monch christian-monch force-pushed the issue-3127-annex-export-datalad-part branch from a097c98 to 658073a Compare April 20, 2022 08:09
Since '_set_request_headers' in 'create_sibling_ghlike.py'
might be patched by by datalad_next, it is marked with
the newly introduced 'todo_interface_for_extensions'
decorator.
@codeclimate
Copy link

codeclimate bot commented Apr 20, 2022

Code Climate has analyzed commit 33836bc and detected 1 issue on this pull request.

Here's the issue category breakdown:

Category Count
Complexity 1

View more on Code Climate.

@christian-monch christian-monch changed the base branch from master to maint April 20, 2022 08:46
@christian-monch christian-monch merged commit eb55f5a into datalad:maint Apr 20, 2022
@yarikoptic
Copy link
Member

eh, now we need another PR with a release label to kick out the release -- what would it be!? ;)

@mih
Copy link
Member

mih commented Apr 21, 2022

Here is a new candidate: #6635

I doubt there will be a shortage of them over the coming days. ;-)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
semver-internal Changes only affect the internal API
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants