[STS]Support opt-in regional STS endpoint #2090

cjyclaire · 2019-07-23T00:08:36Z

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

~~WIP~~

This PR support opt-in STS regional endpoints by setting sts_regional_endpoints: 'regional' at client or from ENV AWS_STS_REGIONAL_ENDPOINTS or sts_regional_endpoints in shared config

By default, SDK set sts_regional_endpoints: 'legacy', previous non-regionalized regions will stay with global endpoints

cjyclaire · 2019-07-23T21:19:05Z

cc @jasdel & @howardlopez for review :D

howardlopez

Looks good, but should there be a test that verifies the sts_regional_endpoints value is taken from the env source first over the config file?

Also, does this need a changelog?

cjyclaire · 2019-07-25T19:12:03Z

@howardlopez Added tests and changelog ;)

uthark · 2019-10-23T22:03:27Z

Hi! This PR breaks for us.

ArgumentError: wrong number of arguments (given 3, expected 2)
/opt/chef/embedded/lib/ruby/gems/2.3.0/gems/aws-partitions-1.162.0/lib/aws-partitions/endpoint_provider.rb:94:in `resolve'
/opt/chef/embedded/lib/ruby/gems/2.3.0/gems/aws-sdk-core-3.71.0/lib/aws-sdk-core/plugins/regional_endpoint.rb:39:in `block in <class:RegionalEndpoint>'
/opt/chef/embedded/lib/ruby/gems/2.3.0/gems/aws-sdk-core-3.71.0/lib/seahorse/client/configuration.rb:70:in `call'
/opt/chef/embedded/lib/ruby/gems/2.3.0/gems/aws-sdk-core-3.71.0/lib/seahorse/client/configuration.rb:213:in `block in resolve_defaults'
/opt/chef/embedded/lib/ruby/gems/2.3.0/gems/aws-sdk-core-3.71.0/lib/seahorse/client/configuration.rb:57:in `each'
/opt/chef/embedded/lib/ruby/gems/2.3.0/gems/aws-sdk-core-3.71.0/lib/seahorse/client/configuration.rb:57:in `each'
/opt/chef/embedded/lib/ruby/gems/2.3.0/gems/aws-sdk-core-3.71.0/lib/seahorse/client/configuration.rb:212:in `resolve_defaults'
/opt/chef/embedded/lib/ruby/gems/2.3.0/gems/aws-sdk-core-3.71.0/lib/seahorse/client/configuration.rb:205:in `value_at'
/opt/chef/embedded/lib/ruby/gems/2.3.0/gems/aws-sdk-core-3.71.0/lib/seahorse/client/configuration.rb:189:in `block in resolve'
/opt/chef/embedded/lib/ruby/2.3.0/set.rb:306:in `each_key'
/opt/chef/embedded/lib/ruby/2.3.0/set.rb:306:in `each'
/opt/chef/embedded/lib/ruby/gems/2.3.0/gems/aws-sdk-core-3.71.0/lib/seahorse/client/configuration.rb:189:in `resolve'
/opt/chef/embedded/lib/ruby/gems/2.3.0/gems/aws-sdk-core-3.71.0/lib/seahorse/client/configuration.rb:177:in `apply_defaults'
/opt/chef/embedded/lib/ruby/gems/2.3.0/gems/aws-sdk-core-3.71.0/lib/seahorse/client/configuration.rb:150:in `build!'
/opt/chef/embedded/lib/ruby/gems/2.3.0/gems/aws-sdk-core-3.71.0/lib/seahorse/client/base.rb:62:in `build_config'
/opt/chef/embedded/lib/ruby/gems/2.3.0/gems/aws-sdk-core-3.71.0/lib/seahorse/client/base.rb:19:in `initialize'
/opt/chef/embedded/lib/ruby/gems/2.3.0/gems/aws-sdk-ec2-1.84.0/lib/aws-sdk-ec2/client.rb:252:in `initialize'
/opt/chef/embedded/lib/ruby/gems/2.3.0/gems/aws-sdk-core-3.71.0/lib/seahorse/client/base.rb:99:in `new'

boveus · 2019-10-23T22:26:02Z

We are experiencing the same issue.

cjyclaire · 2019-10-23T22:36:53Z

taking a look, mind let me know gems and versions you are using? @uthark @boveus

boveus · 2019-10-23T22:40:21Z

Our error seems related to the aws-sdk-cloudfront gem. We are calling

Aws::CloudFront::Client
    .new(credentials: credentials, region: 'us-east-1')

It looks like CloudFront was updated with the latest release of aws-sdk-core:

aws-sdk-cloudfront (1.5.0)
      aws-sdk-core (~> 3)
      aws-sigv4 (~> 1.0)

During our application deployment. That results in the following stack trace:

�[1mTraceback�[m (most recent call last):

	20: from ./deployment/scripts/cdn_setup.rb:50:in `<main>'

	19: from ./deployment/scripts/cdn_setup.rb:33:in `invalidate_distribution!'

	18: from ./deployment/scripts/cdn_setup.rb:25:in `cloudfront_client'

	17: from /app/vendor/ruby-2.6.2/lib/ruby/gems/2.6.0/gems/aws-sdk-core-3.71.0/lib/seahorse/client/base.rb:99:in `new'

	16: from /app/vendor/bundle/ruby/2.6.0/gems/aws-sdk-cloudfront-1.5.0/lib/aws-sdk-cloudfront/client.rb:154:in `initialize'

	15: from /app/vendor/ruby-2.6.2/lib/ruby/gems/2.6.0/gems/aws-sdk-core-3.71.0/lib/seahorse/client/base.rb:19:in `initialize'

	14: from /app/vendor/ruby-2.6.2/lib/ruby/gems/2.6.0/gems/aws-sdk-core-3.71.0/lib/seahorse/client/base.rb:62:in `build_config'

	13: from /app/vendor/ruby-2.6.2/lib/ruby/gems/2.6.0/gems/aws-sdk-core-3.71.0/lib/seahorse/client/configuration.rb:150:in `build!'

	12: from /app/vendor/ruby-2.6.2/lib/ruby/gems/2.6.0/gems/aws-sdk-core-3.71.0/lib/seahorse/client/configuration.rb:177:in `apply_defaults'

	11: from /app/vendor/ruby-2.6.2/lib/ruby/gems/2.6.0/gems/aws-sdk-core-3.71.0/lib/seahorse/client/configuration.rb:189:in `resolve'

	10: from /app/vendor/ruby-2.6.2/lib/ruby/2.6.0/set.rb:338:in `each'

	 9: from /app/vendor/ruby-2.6.2/lib/ruby/2.6.0/set.rb:338:in `each_key'

	 8: from /app/vendor/ruby-2.6.2/lib/ruby/gems/2.6.0/gems/aws-sdk-core-3.71.0/lib/seahorse/client/configuration.rb:189:in `block in resolve'

	 7: from /app/vendor/ruby-2.6.2/lib/ruby/gems/2.6.0/gems/aws-sdk-core-3.71.0/lib/seahorse/client/configuration.rb:205:in `value_at'

	 6: from /app/vendor/ruby-2.6.2/lib/ruby/gems/2.6.0/gems/aws-sdk-core-3.71.0/lib/seahorse/client/configuration.rb:212:in `resolve_defaults'

	 5: from /app/vendor/ruby-2.6.2/lib/ruby/gems/2.6.0/gems/aws-sdk-core-3.71.0/lib/seahorse/client/configuration.rb:57:in `each'

	 4: from /app/vendor/ruby-2.6.2/lib/ruby/gems/2.6.0/gems/aws-sdk-core-3.71.0/lib/seahorse/client/configuration.rb:57:in `each'

	 3: from /app/vendor/ruby-2.6.2/lib/ruby/gems/2.6.0/gems/aws-sdk-core-3.71.0/lib/seahorse/client/configuration.rb:213:in `block in resolve_defaults'

	 2: from /app/vendor/ruby-2.6.2/lib/ruby/gems/2.6.0/gems/aws-sdk-core-3.71.0/lib/seahorse/client/configuration.rb:70:in `call'

	 1: from /app/vendor/ruby-2.6.2/lib/ruby/gems/2.6.0/gems/aws-sdk-core-3.71.0/lib/aws-sdk-core/plugins/regional_endpoint.rb:39:in `block in <class:RegionalEndpoint>'

/app/vendor/bundle/ruby/2.6.0/gems/aws-partitions-1.99.0/lib/aws-partitions/endpoint_provider.rb:94:in `resolve': �[1mwrong number of arguments (given 3, expected 2) (�[1;4mArgumentError�[m�[1m)�[m

cjyclaire · 2019-10-23T22:40:50Z

From your log, looks like you are on an outdated partition gem version, mind update to https://rubygems.org/gems/aws-partitions/versions/1.228.0 and let me know whether it works?

uthark · 2019-10-23T22:41:50Z

@cjyclaire Already testing it.
tests and integration tests were succesful after upgrade.

boveus · 2019-10-23T22:47:47Z

@cjyclaire Testing it now.

cjyclaire · 2019-10-23T23:00:16Z

Thanks again for reporting! if confirming the new version has fixed the issue, I'll update core gemspec to make sure that version is a minimum requirement

boveus · 2019-10-24T17:57:17Z

Sorry for the delay - we had to make a few other changes in addition to the fix for this one. We are good to go after upgrading that dependency. Thanks for the quick response, @cjyclaire!

cjyclaire added 3 commits July 22, 2019 17:08

support opt-in regional sts endpoint

768f535

[ci skip] typo

ca67390

move to regional endpoint plugin instead

699560a

howardlopez reviewed Jul 24, 2019

View reviewed changes

cjyclaire added 3 commits July 25, 2019 10:19

more tests

1e32a38

Merge branch 'master' into sts

f7a573f

[ci skip] changelog

c037dc4

howardlopez approved these changes Jul 25, 2019

View reviewed changes

cjyclaire mentioned this pull request Aug 26, 2019

[v2] support opt-in regional sts endpoint #2106

Merged

cjyclaire added 2 commits August 26, 2019 14:46

rename spec file name, test should be failing

017d55f

Merge branch 'master' into sts

dadddea

snorlaX-sleeps mentioned this pull request Sep 25, 2019

AWS IRSA - Region not being passed from created Client to STS to get JWT #2126

Closed

cjyclaire added 3 commits October 21, 2019 13:09

Merge branch 'master' into sts

e5dc8b0

also update min core version

f224550

Merge branch 'master' into sts

4fdffd2

cjyclaire merged commit b74930e into master Oct 23, 2019

cjyclaire mentioned this pull request Oct 23, 2019

[Core] update minimum partition gem requirement #2144

Merged

timhargreaves mentioned this pull request Oct 29, 2019

STS Endpoint AWS CLI calls feature request aws/aws-cli#4463

Closed

mullermp deleted the sts branch November 1, 2019 19:05

scalp42 mentioned this pull request Nov 9, 2019

bump aws-partitions for aws-sdk-core fix sous-chefs/aws#398

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[STS]Support opt-in regional STS endpoint #2090

[STS]Support opt-in regional STS endpoint #2090

cjyclaire commented Jul 23, 2019 •

edited

cjyclaire commented Jul 23, 2019

howardlopez left a comment

cjyclaire commented Jul 25, 2019

uthark commented Oct 23, 2019

boveus commented Oct 23, 2019

cjyclaire commented Oct 23, 2019 •

edited

boveus commented Oct 23, 2019

cjyclaire commented Oct 23, 2019 •

edited

uthark commented Oct 23, 2019 •

edited

boveus commented Oct 23, 2019

cjyclaire commented Oct 23, 2019

boveus commented Oct 24, 2019

[STS]Support opt-in regional STS endpoint #2090

[STS]Support opt-in regional STS endpoint #2090

Conversation

cjyclaire commented Jul 23, 2019 • edited

cjyclaire commented Jul 23, 2019

howardlopez left a comment

Choose a reason for hiding this comment

cjyclaire commented Jul 25, 2019

uthark commented Oct 23, 2019

boveus commented Oct 23, 2019

cjyclaire commented Oct 23, 2019 • edited

boveus commented Oct 23, 2019

cjyclaire commented Oct 23, 2019 • edited

uthark commented Oct 23, 2019 • edited

boveus commented Oct 23, 2019

cjyclaire commented Oct 23, 2019

boveus commented Oct 24, 2019

cjyclaire commented Jul 23, 2019 •

edited

cjyclaire commented Oct 23, 2019 •

edited

cjyclaire commented Oct 23, 2019 •

edited

uthark commented Oct 23, 2019 •

edited