Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

HADOOP-18320. Fixes typos in Delegation Tokens documentation. #4499

Merged
merged 4 commits into from
Jan 9, 2023

Conversation

ahmarsuhail
Copy link
Contributor

Description of PR

Noticed some typos while reading through the documentation, this PR fixes those.

How was this patch tested?

Not required

@hadoop-yetus
Copy link

🎊 +1 overall

Vote Subsystem Runtime Logfile Comment
+0 🆗 reexec 1m 7s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 0s No case conflicting files found.
+0 🆗 codespell 0m 0s codespell was not available.
+0 🆗 detsecrets 0m 0s detect-secrets was not available.
+0 🆗 markdownlint 0m 0s markdownlint was not available.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
_ trunk Compile Tests _
+1 💚 mvninstall 41m 3s trunk passed
+1 💚 mvnsite 0m 54s trunk passed
+1 💚 shadedclient 64m 19s branch has no errors when building and testing our client artifacts.
_ Patch Compile Tests _
+1 💚 mvninstall 0m 35s the patch passed
+1 💚 blanks 0m 0s The patch has no blanks issues.
+1 💚 mvnsite 0m 38s the patch passed
+1 💚 shadedclient 22m 45s patch has no errors when building and testing our client artifacts.
_ Other Tests _
+1 💚 asflicense 0m 43s The patch does not generate ASF License warnings.
91m 24s
Subsystem Report/Notes
Docker ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4499/1/artifact/out/Dockerfile
GITHUB PR #4499
Optional Tests dupname asflicense mvnsite codespell detsecrets markdownlint
uname Linux c7ba420d0b17 4.15.0-175-generic #184-Ubuntu SMP Thu Mar 24 17:48:36 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/bin/hadoop.sh
git revision trunk / 86ef9d0
Max. process+thread count 607 (vs. ulimit of 5500)
modules C: hadoop-tools/hadoop-aws U: hadoop-tools/hadoop-aws
Console output https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4499/1/console
versions git=2.25.1 maven=3.6.3
Powered by Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

Copy link
Contributor

@dannycjones dannycjones left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @ahmarsuhail, looking good. I have a few some comments where we can improve further.

Can you create a JIRA for this work and update the title to include the ID and be specific to S3A?

Comment on lines +63 to +64
Tokens are opaque to clients. Clients simply get a byte array
of data which they must provide to a service when required.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since we are touching this line, add a line break before "Clients simply" to keep sentence diffs independent in future.

Comment on lines +111 to 113
Issued tokens may be included in job submissions, passed to running applications,
etc. This token is specific to an individual bucket; all buckets which a client
wishes to work with must have a separate delegation token issued.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I find this confusing in the context of delegated role tokens, where the token is literally specific to an individual bucket.

Can we make this clearer while we are editing this paragraph?

Suggested change
Issued tokens may be included in job submissions, passed to running applications,
etc. This token is specific to an individual bucket; all buckets which a client
wishes to work with must have a separate delegation token issued.
Issued tokens may be included in job submissions, passed to running applications,
etc.
Delegation tokens are specific to an S3A filesystem.
Each filesystem a client wishes to work with must have a separate delegation token issued,
regardless of the delegation token variant.

Is it correct to describe them specific to a filesystem rather than the bucket?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Discussed offline - will leave it as is for now as the suggestion wouldn't be making it clearer.

@ahmarsuhail ahmarsuhail changed the title Fixes typos in Delegation Tokens documentation. HADOOP-18320. Fixes typos in Delegation Tokens documentation. Jun 30, 2022
Co-authored-by: Daniel Carl Jones <danny@danielcarl.info>
Copy link
Contributor

@dannycjones dannycjones left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1 (non-binding), thanks for spending the time on these docs

@hadoop-yetus
Copy link

🎊 +1 overall

Vote Subsystem Runtime Logfile Comment
+0 🆗 reexec 0m 50s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 0s No case conflicting files found.
+0 🆗 codespell 0m 0s codespell was not available.
+0 🆗 detsecrets 0m 0s detect-secrets was not available.
+0 🆗 markdownlint 0m 0s markdownlint was not available.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
_ trunk Compile Tests _
+1 💚 mvninstall 39m 46s trunk passed
+1 💚 mvnsite 0m 54s trunk passed
+1 💚 shadedclient 63m 13s branch has no errors when building and testing our client artifacts.
_ Patch Compile Tests _
+1 💚 mvninstall 0m 40s the patch passed
+1 💚 blanks 0m 1s The patch has no blanks issues.
+1 💚 mvnsite 0m 41s the patch passed
+1 💚 shadedclient 25m 12s patch has no errors when building and testing our client artifacts.
_ Other Tests _
+1 💚 asflicense 0m 44s The patch does not generate ASF License warnings.
92m 11s
Subsystem Report/Notes
Docker ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4499/2/artifact/out/Dockerfile
GITHUB PR #4499
Optional Tests dupname asflicense mvnsite codespell detsecrets markdownlint
uname Linux cc35919d3ee5 4.15.0-175-generic #184-Ubuntu SMP Thu Mar 24 17:48:36 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/bin/hadoop.sh
git revision trunk / 34c0e34
Max. process+thread count 607 (vs. ulimit of 5500)
modules C: hadoop-tools/hadoop-aws U: hadoop-tools/hadoop-aws
Console output https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4499/2/console
versions git=2.25.1 maven=3.6.3
Powered by Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

@ahmarsuhail
Copy link
Contributor Author

@mehakmeet / @mukund-thakur I'd made some minor updates to the delegation tokens documentation a while ago, would you be able to take a look?

Copy link
Contributor

@mukund-thakur mukund-thakur left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good. Thanks for taking this up. Got a chance to read the whole doc.
Added a few comments.

@@ -108,7 +108,7 @@ password-protected data opaque to clients; they contain the secrets needed
to access the relevant S3 buckets and associated services.

They are obtained by requesting a delegation token from the S3A filesystem client.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure why github not allowing me to comment at L82.
Delegation "Token" support is

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

L106 -> These S3A Delegation Tokens are special in "a way" that

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

l120 : It -> it.

@@ -353,10 +352,10 @@ it is authenticated with; the role token binding will fail.

When the AWS credentials supplied to the Session Delegation Token binding
through `fs.s3a.aws.credentials.provider` are themselves a set of
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Something is off here, maybe "they themselves are part of session credentials."

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this seems fine to me as is. Maybe removing the themselves entirely would help, so it becomes "through fs.s3a.aws.credentials.provider are a set of", but personally feel it's clearer as is.

@ahmarsuhail
Copy link
Contributor Author

thanks for taking a look @mukund-thakur, have made the suggested changes.

@hadoop-yetus
Copy link

🎊 +1 overall

Vote Subsystem Runtime Logfile Comment
+0 🆗 reexec 0m 48s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 0s No case conflicting files found.
+0 🆗 codespell 0m 0s codespell was not available.
+0 🆗 detsecrets 0m 0s detect-secrets was not available.
+0 🆗 markdownlint 0m 0s markdownlint was not available.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
_ trunk Compile Tests _
+1 💚 mvninstall 41m 17s trunk passed
+1 💚 mvnsite 0m 41s trunk passed
+1 💚 shadedclient 64m 27s branch has no errors when building and testing our client artifacts.
_ Patch Compile Tests _
+1 💚 mvninstall 0m 30s the patch passed
+1 💚 blanks 0m 0s The patch has no blanks issues.
+1 💚 mvnsite 0m 32s the patch passed
+1 💚 shadedclient 22m 48s patch has no errors when building and testing our client artifacts.
_ Other Tests _
+1 💚 asflicense 0m 34s The patch does not generate ASF License warnings.
90m 34s
Subsystem Report/Notes
Docker ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4499/3/artifact/out/Dockerfile
GITHUB PR #4499
Optional Tests dupname asflicense mvnsite codespell detsecrets markdownlint
uname Linux c0b8666f4e9e 4.15.0-200-generic #211-Ubuntu SMP Thu Nov 24 18:16:04 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/bin/hadoop.sh
git revision trunk / 736cedb
Max. process+thread count 568 (vs. ulimit of 5500)
modules C: hadoop-tools/hadoop-aws U: hadoop-tools/hadoop-aws
Console output https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4499/3/console
versions git=2.25.1 maven=3.6.3
Powered by Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

Copy link
Contributor

@mukund-thakur mukund-thakur left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM +1

@mukund-thakur mukund-thakur merged commit 9c6eeb6 into apache:trunk Jan 9, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
4 participants