Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

We will use the regional endpoint with STS whenever possible. #45

Merged
merged 5 commits into from
Sep 30, 2021
Merged

We will use the regional endpoint with STS whenever possible. #45

merged 5 commits into from
Sep 30, 2021

Conversation

sharad-oss
Copy link
Contributor

The global endpoint is used in case the region cannot be found.

#44

Issue #, if available:

Testing:

  • Integration tests succeeded
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary for Amazon EMR User Role Mapper 1.2.0-SNAPSHOT:
[INFO] 
[INFO] Amazon EMR User Role Mapper ........................ SUCCESS [  0.273 s]
[INFO] Amazon EMR User Role Mapper Interface .............. SUCCESS [  1.130 s]
[INFO] Amazon EMR User Role Mapper Application ............ SUCCESS [04:35 min]
[INFO] Amazon EMR User Role Mapper URM Credentials Provider SUCCESS [  5.661 s]
[INFO] Amazon EMR User Role Mapper S3 Storage Based Authorizer for Hive SUCCESS [ 13.551 s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  04:56 min
[INFO] Finished at: 2021-09-28T14:43:39-07:00
[INFO] ------------------------------------------------------------------------
  • Old functionality in place
sudo -u sharad1 aws sts get-caller-identity
{
    "Account": "176430881729", 
    "UserId": "AROASSFAZTPA6YOSG5O4F:sharad1", 
    "Arn": "arn:aws:sts::176430881729:assumed-role/test-emr-sk1/sharad1"
}
  • Logs showing we now use the regional STS endpoint
2021-09-28 18:14:30 worker-thread-16 DEBUG PoolingHttpClientConnectionManager:267 - Connection request: [route: {s}->https://sts.us-east-1.amazonaws.com:443][total available: 0; route allocated: 0 of 50; total allocated: 0 of 50]
2021-09-28 18:14:30 worker-thread-16 DEBUG PoolingHttpClientConnectionManager:312 - Connection leased: [id: 1][route: {s}->https://sts.us-east-1.amazonaws.com:443][total available: 0; route allocated: 1 of 50; total allocated: 1 of 50]
2021-09-28 18:14:30 worker-thread-16 DEBUG MainClientExec:234 - Opening connection {s}->https://sts.us-east-1.amazonaws.com:443
2021-09-28 18:14:30 worker-thread-16 DEBUG DefaultHttpClientConnectionOperator:139 - Connecting to sts.us-east-1.amazonaws.com/54.239.24.200:443
2021-09-28 18:14:30 worker-thread-16 DEBUG SdkTLSSocketFactory:137 - connecting to sts.us-east-1.amazonaws.com/54.239.24.200:443

2021-09-28 18:14:30 worker-thread-16 DEBUG DefaultHttpClientConnectionOperator:146 - Connection established 10.81.172.210:56588<->54.239.24.200:443
2021-09-28 18:14:30 worker-thread-16 DEBUG DefaultManagedHttpClientConnection:88 - http-outgoing-1: set socket timeout to 50000
2021-09-28 18:14:30 worker-thread-16 DEBUG MainClientExec:255 - Executing request POST / HTTP/1.1
2021-09-28 18:14:30 worker-thread-16 DEBUG MainClientExec:266 - Proxy auth state: UNCHALLENGED
2021-09-28 18:14:30 worker-thread-16 DEBUG headers:133 - http-outgoing-1 >> POST / HTTP/1.1
2021-09-28 18:14:30 worker-thread-16 DEBUG headers:136 - http-outgoing-1 >> Host: sts.us-east-1.amazonaws.com
2021-09-28 18:14:30 worker-thread-16 DEBUG headers:136 - http-outgoing-1 >> amz-sdk-invocation-id: 50cb1962-198e-64c4-769b-9d76145a94e9
2021-09-28 18:14:30 worker-thread-16 DEBUG headers:136 - http-outgoing-1 >> amz-sdk-request: attempt=1;max=4
2021-09-28 18:14:30 worker-thread-16 DEBUG headers:136 - http-outgoing-1 >> amz-sdk-retry: 0/0/500
2021-09-28 18:14:30 worker-thread-16 DEBUG headers:136 - http-outgoing-1 >> Authorization: AWS4-HMAC-SHA256 Credential=ASIASSFAZTPATNCJLB43/20210928/us-east-1/sts/aws4_request, SignedHeaders=amz-sdk-invocation-id;amz-sdk-request;amz-sdk-retry;host;user-agent;x-amz-date;x-amz-security-token, Signature=8cee0e8701eacd9f8c8960647963e6a6998f4bca125180b4ce538e496dad2f58
2021-09-28 18:14:30 worker-thread-16 DEBUG headers:136 - http-outgoing-1 >> User-Agent: aws-sdk-java/1.12.22 Linux/4.14.154-99.181.amzn1.x86_64 OpenJDK_64-Bit_Server_VM/25.302-b08 java/1.8.0_302 vendor/Red_Hat,_Inc. cfg/retry-mode/legacy
2021-09-28 18:14:30 worker-thread-16 DEBUG headers:136 - http-outgoing-1 >> X-Amz-Date: 20210928T181430Z

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

@sharad-oss
We will use the regional endpoint with STS whenever possible.
736e5a9 
The global endpoint is used in case the region cannot be found.

#44
yInnovation12
yInnovation12 reviewed Sep 29, 2021
View reviewed changes
Copy link
Contributor

@yInnovation12 yInnovation12 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall LGTM. Minor comments added.

...-mapper-application/src/main/java/com/amazon/aws/emr/credentials/STSCredentialsProvider.java Show resolved Hide resolved
...-mapper-application/src/main/java/com/amazon/aws/emr/credentials/STSCredentialsProvider.java Outdated Show resolved Hide resolved
...-mapper-application/src/main/java/com/amazon/aws/emr/credentials/STSCredentialsProvider.java Outdated Show resolved Hide resolved
sharad-oss
sharad-oss commented Sep 29, 2021
View reviewed changes
Copy link
Contributor Author

@sharad-oss sharad-oss left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Made the STS client instantiation configurable. Based on the app props now we will hit the regional/global endpoint.

yInnovation12
yInnovation12 reviewed Sep 29, 2021
View reviewed changes
emr-user-role-mapper-application/src/main/java/com/amazon/aws/emr/ApplicationConfiguration.java Outdated
Comment on lines 104 to 109
public boolean isRegionalStsEnabled() {
return Boolean.parseBoolean(properties.getProperty(Constants.REGIONAL_STS_ENABLED,
String.valueOf("true")));
}


Copy link
Contributor

@yInnovation12 yInnovation12 Sep 29, 2021 •
edited

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

1/ Add some comments to explain this config
2/ nit: regionalStsEnabled -> regionalStsEndpointEnabled (same comment applies to other relevant places)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

yInnovation12
yInnovation12 reviewed Sep 29, 2021
View reviewed changes
...user-role-mapper-application/src/main/java/com/amazon/aws/emr/credentials/STSClientImpl.java Outdated
region = Regions.getCurrentRegion();
regionString = region.getName();
} catch (Exception e) {
log.error("Cannot determine the AWS region. Defaulting to {}", regionString);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If the code cannot get the regionString, how can subsequent code uses the regional endpoint?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's set to "us-west-2" in the beginning. So we will always use us-west-2 in this case.

@sharad-oss
Incorporated the PR comments.
30d4638 
- Renamed constant to use regional STS endpoint
- Added some comments about the constant
sharad-oss
sharad-oss commented Sep 29, 2021
View reviewed changes
Copy link
Contributor Author

@sharad-oss sharad-oss left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Incorporated the code review comments.

  • Renamed the constant
  • Added some comment to enable the regional STS endpoint

yInnovation12
yInnovation12 approved these changes Sep 30, 2021
View reviewed changes
Copy link
Contributor

@yInnovation12 yInnovation12 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@yInnovation12 yInnovation12 merged commit e4a9c4d into master Sep 30, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@yInnovation12 yInnovation12 yInnovation12 approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
2 participants
@sharad-oss @yInnovation12