Describe the bug
Summary: boto3 seems to use $AWS_DEFAULT_PROFILE in preference to $AWS_PROFILE, which is the opposite to the aws cli's behavior.
Details:
I'm using SSO.
I've got a ~/.aws/config file containing multiple profiles, including [profile foo] and [profile bar].
I've set AWS_DEFAULT_PROFILE=foo and AWS_PROFILE=bar.
I'm logged in - the aws cli is working and accessing stuff in "bar" (as expected).
I've got a python script that gets a session by calling boto3.Session()
...but client calls fail unless I unset AWS_DEFAULT_PROFILE.
Expected Behavior
I expected boto3.Session() to use the profile from $AWS_PROFILE and to ignore $AWS_DEFAULT_PROFILE (because $AWS_PROFILE takes precedence and it was set).
i.e. the default should be ignored if a non-default value is provided.
Current Behavior
Traceback (most recent call last):
File "~/test.py", line 5, in <module>
cw.describe_alarms(AlarmNamePrefix="i2o-uk-test-pre-prod")
File "/usr/local/lib/python3.9/site-packages/botocore/client.py", line 535, in _api_call
return self._make_api_call(operation_name, kwargs)
File "/usr/local/lib/python3.9/site-packages/botocore/client.py", line 963, in _make_api_call
http, parsed_response = self._make_request(
File "/usr/local/lib/python3.9/site-packages/botocore/client.py", line 986, in _make_request
return self._endpoint.make_request(operation_model, request_dict)
File "/usr/local/lib/python3.9/site-packages/botocore/endpoint.py", line 119, in make_request
return self._send_request(request_dict, operation_model)
File "/usr/local/lib/python3.9/site-packages/botocore/endpoint.py", line 198, in _send_request
request = self.create_request(request_dict, operation_model)
File "/usr/local/lib/python3.9/site-packages/botocore/endpoint.py", line 134, in create_request
self._event_emitter.emit(
File "/usr/local/lib/python3.9/site-packages/botocore/hooks.py", line 412, in emit
return self._emitter.emit(aliased_event_name, **kwargs)
File "/usr/local/lib/python3.9/site-packages/botocore/hooks.py", line 256, in emit
return self._emit(event_name, kwargs)
File "/usr/local/lib/python3.9/site-packages/botocore/hooks.py", line 239, in _emit
response = handler(**kwargs)
File "/usr/local/lib/python3.9/site-packages/botocore/signers.py", line 105, in handler
return self.sign(operation_name, request)
File "/usr/local/lib/python3.9/site-packages/botocore/signers.py", line 180, in sign
auth = self.get_auth_instance(**kwargs)
File "/usr/local/lib/python3.9/site-packages/botocore/signers.py", line 284, in get_auth_instance
frozen_credentials = self._credentials.get_frozen_credentials()
File "/usr/local/lib/python3.9/site-packages/botocore/credentials.py", line 611, in get_frozen_credentials
self._refresh()
File "/usr/local/lib/python3.9/site-packages/botocore/credentials.py", line 499, in _refresh
self._protected_refresh(is_mandatory=is_mandatory_refresh)
File "/usr/local/lib/python3.9/site-packages/botocore/credentials.py", line 515, in _protected_refresh
metadata = self._refresh_using()
File "/usr/local/lib/python3.9/site-packages/botocore/credentials.py", line 662, in fetch_credentials
return self._get_cached_credentials()
File "/usr/local/lib/python3.9/site-packages/botocore/credentials.py", line 672, in _get_cached_credentials
response = self._get_credentials()
File "/usr/local/lib/python3.9/site-packages/botocore/credentials.py", line 2117, in _get_credentials
token = self._token_loader(self._start_url)['accessToken']
File "/usr/local/lib/python3.9/site-packages/botocore/utils.py", line 3164, in __call__
raise SSOTokenLoadError(error_msg=error_msg)
botocore.exceptions.SSOTokenLoadError: Error loading SSO Token: Token for https://d-12345abcde.awsapps.com/start#/ does not exist
Reproduction Steps
set AWS_DEFAULT_PROFILE=foo and AWS_PROFILE=bar, where foo and bar are valid aws profiles described in ~/.aws/config
Login to aws by doing aws sso login
Check aws cli is working; it should be accessing stuff in "bar".
Run a python script:
#!/usr/bin/python
import boto3
session = boto3.Session()
cw = session.client('cloudwatch')
cw.describe_alarms(AlarmNamePrefix="something-that-exists-in-foo")
... then instead of getting alarms read (which this script doesn't output), we get an exception thrown:
botocore.exceptions.SSOTokenLoadError: Error loading SSO Token: Token for https://d-12345abcde.awsapps.com/start#/ does not exist
If I unset AWS_DEFAULT_PROFILE or if I set AWS_DEFAULT_PROFILE to be == $AWS_PROFILE then things work again.
Similarly, if I change the code to say session = boto3.Session(profile_name='bar') then it all works.
Possible Solution
I suspect that some (if not all) of the code is looking at AWS_DEFAULT_PROFILE when it should be looking at "AWS_PROFILE if that's set, otherwise use AWS_DEFAULT_PROFILE".
I found that if I did export AWS_DEFAULT_PROFILE=$AWS_PROFILE or I unset AWS_DEFAULT_PROFILE then boto3 was happy again ... but "in an ideal world" these workarounds shouldn't be necessary.
Additional Information/Context
As an end user, having different AWS technologies use different authentication controls & requirements is confusing. I want them all to work the same way, so as the aws cli uses AWS_PROFILE if it's set but defaulting to AWS_DEFAULT_PROFILE (if AWS_PROFILE isn't set) then that's the behaviour I expect from boto3.
It might be that this issue was the underlying cause of the confusion reported in #913, although these two scenarios are significantly different.
Environment details:
I'm using a docker container running under Windows WSL2 & docker desktop (layers in layers in layers...) but the short version is that "it's basically linux".
python --version reports Python 3.9.16
pip show boto3 reports:
Name: boto3
Version: 1.28.57
Summary: The AWS SDK for Python
Home-page: https://github.com/boto/boto3
Author: Amazon Web Services
Author-email:
License: Apache License 2.0
Location: /usr/local/lib/python3.9/site-packages
Requires: jmespath, botocore, s3transfer
Required-by: aws-sam-translator
...and pip show botocore tells me it's Version: 1.31.57
SDK version used
boto3 version 1.28.57
Environment details (OS name and version, etc.)
Linux 752a5f9ee677 5.10.102.1-microsoft-standard-WSL2 #1 SMP Wed Mar 2 00:30:59 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux with python 3.9.16
Describe the bug
Summary: boto3 seems to use $AWS_DEFAULT_PROFILE in preference to $AWS_PROFILE, which is the opposite to the aws cli's behavior.
Details:
I'm using SSO.
I've got a
~/.aws/configfile containing multiple profiles, including[profile foo]and[profile bar].I've set AWS_DEFAULT_PROFILE=foo and AWS_PROFILE=bar.
I'm logged in - the aws cli is working and accessing stuff in "bar" (as expected).
I've got a python script that gets a session by calling
boto3.Session()...but client calls fail unless I unset AWS_DEFAULT_PROFILE.
Expected Behavior
I expected
boto3.Session()to use the profile from$AWS_PROFILEand to ignore$AWS_DEFAULT_PROFILE(because$AWS_PROFILEtakes precedence and it was set).i.e. the default should be ignored if a non-default value is provided.
Current Behavior
Reproduction Steps
set AWS_DEFAULT_PROFILE=foo and AWS_PROFILE=bar, where foo and bar are valid aws profiles described in
~/.aws/configLogin to aws by doing
aws sso loginCheck aws cli is working; it should be accessing stuff in "bar".
Run a python script:
... then instead of getting alarms read (which this script doesn't output), we get an exception thrown:
botocore.exceptions.SSOTokenLoadError: Error loading SSO Token: Token for https://d-12345abcde.awsapps.com/start#/ does not existIf I unset AWS_DEFAULT_PROFILE or if I set AWS_DEFAULT_PROFILE to be == $AWS_PROFILE then things work again.
Similarly, if I change the code to say
session = boto3.Session(profile_name='bar')then it all works.Possible Solution
I suspect that some (if not all) of the code is looking at AWS_DEFAULT_PROFILE when it should be looking at "AWS_PROFILE if that's set, otherwise use AWS_DEFAULT_PROFILE".
I found that if I did
export AWS_DEFAULT_PROFILE=$AWS_PROFILEor I unset AWS_DEFAULT_PROFILE then boto3 was happy again ... but "in an ideal world" these workarounds shouldn't be necessary.Additional Information/Context
As an end user, having different AWS technologies use different authentication controls & requirements is confusing. I want them all to work the same way, so as the aws cli uses AWS_PROFILE if it's set but defaulting to AWS_DEFAULT_PROFILE (if AWS_PROFILE isn't set) then that's the behaviour I expect from boto3.
It might be that this issue was the underlying cause of the confusion reported in #913, although these two scenarios are significantly different.
Environment details:
I'm using a docker container running under Windows WSL2 & docker desktop (layers in layers in layers...) but the short version is that "it's basically linux".
python --versionreportsPython 3.9.16pip show boto3reports:...and
pip show botocoretells me it'sVersion: 1.31.57SDK version used
boto3 version 1.28.57
Environment details (OS name and version, etc.)
Linux 752a5f9ee677 5.10.102.1-microsoft-standard-WSL2 #1 SMP Wed Mar 2 00:30:59 UTC 2022 x86_64 x86_64 x86_64 GNU/Linuxwith python 3.9.16