Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added support for cs3 api #72

Merged
merged 4 commits into from
Sep 6, 2021
Merged

Conversation

dynamic-entropy
Copy link
Contributor

@dynamic-entropy dynamic-entropy commented Aug 18, 2021

This PR extends libdavix with CS3 Request Protocol interface and corresponding Authorisation, Authentication and Transfer Request parameters, and methods to interact with them as required by PR cern-fts/gfal2#7.

A transfer between a ScienceMesh site and a Grid Site introduces a possibility for FTS to Authenticate with two different tokens for certain transfers. And these identities should be provided by the user on whose behalf the transfers are made.
Later Transfer Headers are determined based on role (active, passive) of endpoints and mode ( push, pull, streamed) of transfer.

Copy link
Contributor

@glpatcern glpatcern left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks all right to me for how I understand GFAL and Davix, of course the final word to @mpatrascoiu ;-)

//Assuming urls with +3rd prefix have been deprecated

std::string src_url, dst_url, token;

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe add a comment referring the code mentioned in cern-fts/gfal2#7 (comment)

Copy link
Contributor

@mpatrascoiu mpatrascoiu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hello,

I've put my most important notes in specific comments.

Just wanted to also point out some occasional trailing white space here and there,
I believe this has to do with the editor used. This is not a must-have, just syntactic sugar 🙂

Cheers,
Mihai

src/utils/davix_cs3_utils.cpp Show resolved Hide resolved
src/utils/davix_cs3_utils.cpp Outdated Show resolved Hide resolved
src/utils/davix_cs3_utils.cpp Outdated Show resolved Hide resolved
src/modules/copy/copy.cpp Outdated Show resolved Hide resolved
src/modules/copy/copy.cpp Outdated Show resolved Hide resolved
}
}

void Credentials::addCredentials(std::string uri, std::string token, bool token_write_access){
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

token can be mode const& reference.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Regarding token_write_access, I found it a bit difficult to convert from this role to isDst role, which is the flag's corresponding purpose in the Credential object.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Earlier we misunderstood the implications of token_write_access. We were trying to add an extra var. I missed to correct it here. Changed it back to token_write_access.

src/utils/davixuri.cpp Outdated Show resolved Hide resolved

//In pull mode original uri = destination uri
if(_params.getCopyMode() == Davix::CopyMode::Pull){
for (reva::CredentialMap::iterator itr = cmap.begin(); itr != cmap.end(); ++itr) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this an iteration over the whole map?
Wouldn't that mean the active_token and passive_token values depend only on the last element in the map?

Shouldn't there be a filtering by uri?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This credMap is supposed to live only in the context of that transfer. So we only have two token at any time.
If this is not destroyed when that transfer job completes then we would have to find an alternative.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I understand. If the map is guaranteed to only have 2 elements in it, it's ok.

Making sure I have to have the correct rationale:

  • active_token / passive_token naming is influenced primarily by 3rd push/pull role (E.g.: destination is active for 3rd pull / source is active for 3rd push)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, that is correct. The names are borrowed from a draft RFC on HTTP-TPC.

- Added cs3s to httpizeProtocol
- corrected isDst to token_wtite_access
Copy link
Contributor

@mpatrascoiu mpatrascoiu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the changes. I've put just one additional review. Looking forward to the final merge!

@@ -49,7 +49,7 @@ class DAVIX_EXPORT Credentials {
class DAVIX_EXPORT CredentialProvider{
public:
CredentialProvider(){};
void updateCredentials(Credentials &creds, std::string uri, bool token_write_access);
static void updateCredentials(Credentials &creds, std::string uri, bool token_write_access);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hello,

On second thought, to keep this consistent with the gcloud::CredentialProvider and the way it is used in Gfal2, let's keep reva::updateCredentials(..) without static in it.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure.


//In pull mode original uri = destination uri
if(_params.getCopyMode() == Davix::CopyMode::Pull){
for (reva::CredentialMap::iterator itr = cmap.begin(); itr != cmap.end(); ++itr) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I understand. If the map is guaranteed to only have 2 elements in it, it's ok.

Making sure I have to have the correct rationale:

  • active_token / passive_token naming is influenced primarily by 3rd push/pull role (E.g.: destination is active for 3rd pull / source is active for 3rd push)

@mpatrascoiu mpatrascoiu merged commit befab11 into cern-fts:devel Sep 6, 2021
@mpatrascoiu
Copy link
Contributor

mpatrascoiu commented Sep 6, 2021

Many thanks for the contributions!
Merged into devel branch.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants