Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Endpoint resolution v2: builtin parameter values #2759

Merged
merged 11 commits into from
Sep 16, 2022

Conversation

jonemo
Copy link
Contributor

@jonemo jonemo commented Sep 10, 2022

This PR is part of the ongoing endpoint resolution v2 work, together with #2737, #2740, #2751, #2747. It contains the code that assigns values to a list of "builtin parameters" that may be referenced by endpoint rulesets. It's glue code between the existing ClientCreator and ClientArgsCreator and the future EndpointProvider (see #2737).

Q: By itself, this PR adds only unused code. Why is it a separate PR? — A: I decided to break this out of a larger PR with all the middleware for calling EndpointProvider to keep this a separate review/discussion. Even though the code is pretty simple, there could be a complex discussion hiding behind each parameter assignment and how it is tested. Let's get these debates out of the way first :)

@codecov-commenter
Copy link

codecov-commenter commented Sep 10, 2022

Codecov Report

Merging #2759 (e0700d3) into develop (6d4551e) will increase coverage by 0.11%.
The diff coverage is 100.00%.

@@             Coverage Diff             @@
##           develop    #2759      +/-   ##
===========================================
+ Coverage    95.13%   95.25%   +0.11%     
===========================================
  Files           61       62       +1     
  Lines        12538    12872     +334     
===========================================
+ Hits         11928    12261     +333     
- Misses         610      611       +1     
Impacted Files Coverage Δ
botocore/args.py 100.00% <100.00%> (ø)
botocore/regions.py 95.34% <100.00%> (+0.38%) ⬆️
botocore/endpoint_provider.py 99.67% <0.00%> (ø)
botocore/utils.py 93.59% <0.00%> (+<0.01%) ⬆️
botocore/exceptions.py 99.55% <0.00%> (+<0.01%) ⬆️
botocore/compat.py 92.66% <0.00%> (+0.04%) ⬆️

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

Copy link
Contributor

@nateprewitt nateprewitt left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is looking great. Left some feedback on testing and a couple questions on how we're setting defaults. Let me know if I can clarify anything.

botocore/args.py Outdated Show resolved Hide resolved
botocore/args.py Show resolved Hide resolved
botocore/args.py Show resolved Hide resolved
botocore/regions.py Outdated Show resolved Hide resolved
botocore/regions.py Outdated Show resolved Hide resolved
botocore/args.py Outdated Show resolved Hide resolved
Comment on lines 464 to 467
should = self.args_create._should_set_global_sts_endpoint(
region_name='us-west-1', endpoint_url=None, endpoint_config=None
)
self.assertTrue(should)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Typically we won't test private methods directly, instead verifying the desired behavior at whatever the top level interface is. That allows us to make more substantial signature changes without having to update existing tests.

How are you intending to retrieve this from the args module? We may need to expose a "public" method even if it's not intended for use, similar to compute_s3_config. We can note support expectations in the docstring.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How are you intending to retrieve this from the args module?

_should_set_global_sts_endpoint is a method in the class ClientArgsCreator. The assignment of builtin parameter values happens within the same class.

This is, in fact, the reason why the _compute_endpoint_resolver_v2_builtin_defaults ended up in ClientArgsCreator: Most of the required information is available locally in this class.

tests/unit/test_args.py Outdated Show resolved Hide resolved
)
self.assertEqual(bins['AWS::S3::DisableMultiRegionAccessPoints'], True)

def test_builtin_sdk_endpoint_1(self):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Instead of using 1,2,3 can we make these test names a bit clearer?

e.g.

test_sdk_endpoint_both_inputs_set
test_sdk_endpoint_legacy_set_with_builtin_data
test_sdk_endpoint_legacy_set_without_builtin_data

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm also just now realizing that builtin is overloaded here. We might want to change the method name for detecting default endpoints.json data. uses_default_endpoint_data?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tests are renamed in 87be3a7.

Re overloaded use of term "builtin": Unfortunately, both uses already have precedent. "Builtin data files" is consistent with botocore.loaders. BUILTIN_DATA_PATH and botocore.loaders.BUILTIN_EXTRAS_TYPES. (And of course the other use of "builtin" comes from the cross-SDK endpoints spec.)

tests/unit/test_args.py Outdated Show resolved Hide resolved
Copy link
Contributor

@nateprewitt nateprewitt left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor tweak due to a bad comment from me earlier. Otherwise, looks great, thanks!

botocore/regions.py Outdated Show resolved Hide resolved
Copy link
Contributor

@nateprewitt nateprewitt left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jonemo jonemo merged commit d10816a into boto:develop Sep 16, 2022
@jonemo jonemo deleted the endpointresolver-v2-builtins branch September 16, 2022 21:13
aws-sdk-python-automation added a commit that referenced this pull request Sep 19, 2022
* release-1.27.76:
  Bumping version to 1.27.76
  Update to latest endpoints
  Update to latest models
  Endpoint resolution v2: builtin parameter values (#2759)
  Support for context params in service models (#2740)
@jonemo jonemo mentioned this pull request Oct 21, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants