Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added support for HTTP TPC #2007

Merged
merged 24 commits into from
Feb 14, 2022
Merged

Conversation

dynamic-entropy
Copy link
Contributor

@dynamic-entropy dynamic-entropy commented Aug 19, 2021

Added support for HTTP Third Party Copy to the ocdav package.
HTTP-TPC is a protocol, based on HTTP, for copying an entity located on one server to a different server.
The main benefit is to allow for optimal the network use.
One typical optimisation is achieved by allowing the two servers to communicate directly so that the data representing the
entity is not relayed through the controlling client.

TODO:

  • [ ] Add ability to supply storage issued token -> this is addressed by mapping OIDC tokens

Closes #1787

@update-docs
Copy link

update-docs bot commented Aug 19, 2021

Thanks for opening this pull request! The maintainers of this repository would appreciate it if you would create a changelog item based on your changes.

@dynamic-entropy dynamic-entropy force-pushed the dev-http-tpc branch 2 times, most recently from 74ce48a to 79bd34b Compare August 19, 2021 05:45
@lgtm-com
Copy link

lgtm-com bot commented Aug 19, 2021

This pull request introduces 2 alerts when merging 79bd34b into d3a946e - view on LGTM.com

new alerts:

  • 2 for Uncontrolled data used in network request

@dynamic-entropy dynamic-entropy force-pushed the dev-http-tpc branch 2 times, most recently from 44407bc to 0426fb9 Compare August 19, 2021 06:13
@dynamic-entropy dynamic-entropy changed the title [WIP] Added support for HTTP TPC #1787 [WIP] Added support for HTTP TPC (#1787) Aug 19, 2021
@dynamic-entropy dynamic-entropy changed the title [WIP] Added support for HTTP TPC (#1787) [WIP] Added support for HTTP TPC Aug 19, 2021
@dynamic-entropy dynamic-entropy changed the title [WIP] Added support for HTTP TPC Added support for HTTP TPC Aug 19, 2021
@lgtm-com
Copy link

lgtm-com bot commented Aug 19, 2021

This pull request introduces 2 alerts when merging 0426fb9 into d3a946e - view on LGTM.com

new alerts:

  • 2 for Uncontrolled data used in network request

@lgtm-com
Copy link

lgtm-com bot commented Aug 19, 2021

This pull request introduces 2 alerts when merging 56b2105 into d3a946e - view on LGTM.com

new alerts:

  • 2 for Uncontrolled data used in network request

@lgtm-com
Copy link

lgtm-com bot commented Aug 19, 2021

This pull request introduces 2 alerts when merging 45c0cdc into d3a946e - view on LGTM.com

new alerts:

  • 2 for Uncontrolled data used in network request

@lgtm-com
Copy link

lgtm-com bot commented Aug 19, 2021

This pull request introduces 2 alerts when merging dd0093b into d3a946e - view on LGTM.com

new alerts:

  • 2 for Uncontrolled data used in network request

@lgtm-com
Copy link

lgtm-com bot commented Aug 19, 2021

This pull request introduces 2 alerts when merging 6024913 into 7f7fab2 - view on LGTM.com

new alerts:

  • 2 for Uncontrolled data used in network request

@lgtm-com
Copy link

lgtm-com bot commented Aug 19, 2021

This pull request introduces 2 alerts when merging 028a0cd into 7f7fab2 - view on LGTM.com

new alerts:

  • 2 for Uncontrolled data used in network request

@butonic
Copy link
Contributor

butonic commented Aug 30, 2021

for reference:
davix / docs

Davix aims to make the task of managing files over HTTP-based protocols simple. It is being developed by the IT-SDC-ID section at CERN, and while the project’s purpose is its use on the CERN grid, the functionality offered is generic.

gfal2 / docs

GFAL2 offers an a single and simple toolkit for the file operations in grids and cloud environments. The set of supported protocols depends on the installed plugins.

Just trying to understand what this is for...

@glpatcern
Copy link
Member

Just trying to understand what this is for...

Indeed, thanks for adding those references @butonic ! And for completeness, this is the report of the GSoC project related to this PR:

https://gist.github.com/dynamic-entropy/ce664532ab28f8a13b7cf8f68e2d0f2a

Copy link
Member

@glpatcern glpatcern left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Resurrecting this work to converge, so this PR could be merged and then we concentrate on the authentication part in another PR: can you please address those changes?

Then the authentication would touch the identity provider (in particular the oidc one), not this part, which I expect it to just work when Reva is driving the Third-Party Copy. Instead when Reva is the passive side, we'll have to possibly extend the code such that on GET we validate the bearer token against different/multiple IDPs.

internal/http/services/owncloud/ocdav/copy.go Outdated Show resolved Hide resolved
internal/http/services/owncloud/ocdav/tpc.go Outdated Show resolved Hide resolved
internal/http/services/owncloud/ocdav/tpc.go Outdated Show resolved Hide resolved
glpatcern and others added 23 commits February 11, 2022 10:33
Co-authored-by: Alex Unger <6905948+refs@users.noreply.github.com>

Co-authored-by: Alex Unger <6905948+refs@users.noreply.github.com>
Also reworked example configuration for OIDC mapping
as developed within WP4.4
Using for now a top-level group as IAM-ESCAPE seems to not be able
to return optional subgroups as claims when querying the userinfo endpoint.
This module now supports the standard OIDC functionality when
no mapping is defined, and the mapping via group claim when configured.
In the future, oidc.go will be superseded by oidcmapping.go.
@lgtm-com
Copy link

lgtm-com bot commented Feb 11, 2022

This pull request introduces 2 alerts when merging 3450ac2 into 96c9c09 - view on LGTM.com

new alerts:

  • 2 for Uncontrolled data used in network request

@ishank011 ishank011 merged commit b52aaf0 into cs3org:master Feb 14, 2022
Daniel-WWU-IT pushed a commit to Daniel-WWU-IT/reva that referenced this pull request Feb 16, 2022
Co-authored-by: Giuseppe Lo Presti <giuseppe.lopresti@cern.ch>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Extend HTTP COPY in revad
7 participants