AssumeRoleWithWebIdentityCredentialProvider defaults to STS regional endpoint in us-east-1 #3197
Description
Describe the bug
After the AWS outage in us-east-1 on Monday October 20, we discovered that we were using the regional STS endpoint in us-east-1 from our application. (I.e. not the global STS service at sts.amazonaws.com, but specifically sts.us-east-1.amazonaws.com.
We are running a PHP application on Amazon EKS, and are using IAM roles for service accounts to automatically provide credentials. This integration configures a set of environment variables, including AWS_STS_REGIONAL_ENDPOINTS=regional and AWS_REGION=eu-north-1:
AWS_STS_REGIONAL_ENDPOINTS: regional
AWS_DEFAULT_REGION: eu-north-1
AWS_REGION: eu-north-1
AWS_ROLE_ARN: arn:aws:iam::123456789012:role/SomeRole
AWS_WEB_IDENTITY_TOKEN_FILE: /var/run/secrets/eks.amazonaws.com/serviceaccount/token
Our code to initialize the client uses CredentialProvider::defaultProvider():
$cache = new PsrCacheAdapter(new ApcuAdapter());
$provider = CredentialProvider::defaultProvider();
$client = new DynamoDbClient([
'version' => '2012-08-10',
'region' => 'eu-north-1',
'credentials' => CredentialProvider::cache($provider, $cache),
]);This ends up calling the STS AssumeRoleWithWebIdentity API through sts.us-east-1.amazonaws.com to get credentials.
Regression Issue
- Select this option if this issue appears to be a regression.
Expected Behavior
AssumeRoleWithWebIdentityCredentialProvider should either use the regional STS endpoints in the current region, or the global endpoints (which would be automatically mapped to the current region thanks to recent changes).
Current Behavior
AssumeRoleWithWebIdentityCredentialProvider specifically chooses to use us-east-1 with regional endpoints when no region is specifically configured.
Reproduction Steps
(This issue is hard to create a good reproducer for, since it is mainly about how the AWS PHP SDK interacts with the environment set up by AWS EKS.)
To reproduce this issue, we need an EKS cluster configured in a different region than us-east-1, and a service account in that cluster that is configured with IAM roles for service accounts.
Then run the following PHP script in a pod that uses that service account:
#!/usr/bin/env php
<?php
require_once('vendor/autoload.php');
$client = new \Aws\Sts\StsClient([
'credentials' => \Aws\Credentials\CredentialProvider::defaultProvider(),
]);
$client->getCallerIdentity([]);We can then use the CloudTrail logs to see the behavior.
In this case, we will get one CloudTrail entry in the us-east-1 region for the AssumeRoleWithWebIdentity call. This entry shows that the request was directed at the regional endpoint in us-east-1:
{
# ...
"eventName": "AssumeRoleWithWebIdentity",
"awsRegion": "us-east-1",
# ...
"additionalEventData": {
# ...
"RequestDetails": {
"awsServingRegion": "us-east-1",
"endpointType": "regional"
}
},
# ...
"tlsDetails": {
# ...
"clientProvidedHostHeader": "sts.us-east-1.amazonaws.com"
}
}Meanwhile, we will find the log entry for the GetCallerIdentity call in the CloudTrail logs in the region of the EKS cluster:
{
# ...
"eventName": "GetCallerIdentity",
"awsRegion": "eu-north-1",
# ...
"additionalEventData": {
# ...
"RequestDetails": {
"awsServingRegion": "eu-north-1",
"endpointType": "regional"
}
},
# ...
"tlsDetails": {
# ...
"clientProvidedHostHeader": "sts.eu-north-1.amazonaws.com"
}
}Possible Solution
No response
Additional Information/Context
As far as I can tell, the current code that sets the region is the following:
$region = $config['region'] ?? 'us-east-1';
if (isset($config['client'])) {
$this->client = $config['client'];
} else {
$this->client = new StsClient([
'credentials' => false,
'region' => $region,
'version' => 'latest'
]);
}(AssumeRoleWithWebIdentityCredentialProvider.php#L75-L84)
SDK version used
3.356.42
Environment details (Version of PHP (php -v)? OS name and version, etc.)
PHP 8.2.29 from Docker image php:8.2-fpm-bullseye