Changes to enable Smithy S3 CRT

[sbera87]

Issue #, if available:
Description of changes:
The changes are pre-requisites to enable smithy s3 crt client. This covers:

• changes in smithy s3 codegen to also extend to s3-crt clients
• refactor code in smithy base for reusability for s3-crt client as well as rest of the clients
• fixes to accommodate recent accountid endpoint changes

Check all that applies:

• Did a review by yourself.
• Added proper tests to cover this PR. (If tests are not applicable, explain.)
• Checked if this PR is a breaking (APIs have been changed) change.
• Checked if this PR will not introduce cross-platform inconsistent behavior.
• Checked if this PR would require a ReadMe/Wiki update.

Check which platforms you have built SDK on to verify the correctness of this PR.

• Linux
• Windows
• Android
• MacOS
• IOS
• Other Platforms

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

[sbera87]

This is refactor of existing code into a helper API for reuse

[sbera87]

Move this block so that service params set before account id based identity resolution happens. Without this, s3 express wont work

[sbera87]

This defaults initialization of few members which was previously being done in request processing

[SergeyRyabinin]

1. pExecutor and responseHandler assumes that the function is async, why does it also return a simple boolean? How does it going to fit into the existing async design?
2. Why do we need 2 method of ResolveIdentityAuth?

[sbera87]

1. This is a Private helper method , the reason it returns bool is simply for the caller to know if any error occurred. The actual type specific outcome is handled in responseHandler. The return type is bool because this function throws different types of outcome errors which will need a variant and complicate the code much more in the calling function. The equivalent is achieved by choosing a simple return type but still processing and returning correct outcome type through the responseHandler.
2. ResolveIdentityAuth is a private method that just only houses the code for identity and auth resolution. It is simply reused in preexisting 'AwsSmithyClientBase::MakeRequestAsync'. The reason this part of code is moved to a helper function is because in S3-CRT operations like put/getObjectAsync, we only resolve the endpoint in sdk and for rest of the request handling happens in crt.

[SergeyRyabinin]

Why ResolveIdentity also resolves an endpoint?

[sbera87]

The helper function just houses the code to select auth scheme, resolve identity and resolve endpoint.
In pass case, the crt methods need the endpointOutcome.
For example, from s3-crtclient.cpp, for one of the async apis, the crt initialization takes endpoint, which is simply obtained in the smithy auth call flow aforementioned.

auto endpointResolutionOutcome = TracingUtils::MakeCallWithTiming<ResolveEndpointOutcome>(
  [&]() -> ResolveEndpointOutcome {
    return AwsSmithyClientT::ResolveIdentityAuth(
    &request,
    request.GetServiceRequestName(),
    std::move(endpointCallback));
   },
  TracingUtils::SMITHY_CLIENT_ENDPOINT_RESOLUTION_METRIC,
  *meter,
  {{TracingUtils::SMITHY_METHOD_DIMENSION, request.GetServiceRequestName()}, {TracingUtils::SMITHY_SERVICE_DIMENSION, this->GetServiceClientName()}});

  // make aws_s3_meta_request with callbacks
  CrtRequestCallbackUserData *userData = Aws::New<CrtRequestCallbackUserData>(ALLOCATION_TAG);
  aws_s3_meta_request_options options;
  AWS_ZERO_STRUCT(options);
  aws_uri endpoint;
  InitCrtEndpointFromUri(endpoint, endpointResolutionOutcome.GetResult().GetURI());

[SergeyRyabinin]

Why ResolveIdentity also validates DNS host name?

[sbera87]

Since the helper function just houses the code to select auth, resolve identity and then resolve endpoint, this validation code is the existing check we have to ensure endpoint resolved is valid or not. In case it is not valid, it reflects that in the outcome.

[SergeyRyabinin]

Async in MakeRequestAsync stands for async execution of this method, the handler is not being called, the caller will wait forever for a request.

[sbera87]

I am not sure which handler is being referred to, but 'responseHandler' is being called. This is tested with S3-crt and S3-crt express integration and unit tests

[SergeyRyabinin]

Please provide a design doc / explanation why this change is needed.
The function below ResolveIdentityAuth has a very questionable design (creates a new executor, then executes a function in a sync blocking mode and then returns).

[sbera87]

This callback is used in this smithy client the same way for other clients. It is simply used in existing resolveEndpoint function.

EndpointUpdateCallback endpointCallback = [&](Aws::Endpoint::AWSEndpoint& resolvedEndpoint){
      resolvedEndpoint.AddPathSegments(request.GetKey());
    };
  auto meter = m_clientConfiguration.telemetryProvider->getMeter(this->GetServiceClientName(), {});
  auto endpointResolutionOutcome = TracingUtils::MakeCallWithTiming<ResolveEndpointOutcome>(
  [&]() -> ResolveEndpointOutcome {
    return AwsSmithyClientT::ResolveIdentityAuth(
    &request,
    request.GetServiceRequestName(),
    std::move(endpointCallback));
   },

It is used in client base as:

auto epResolutionOutcome = this->ResolveEndpoint(std::move(epParams), std::move(endpointCallback));