Skip to content

SSO auth failure if AWS_DEFAULT_PROFILE != AWS_PROFILE #3891

Description

@Peter-Darton-i2

Describe the bug

Summary: boto3 seems to use $AWS_DEFAULT_PROFILE in preference to $AWS_PROFILE, which is the opposite to the aws cli's behavior.

Details:
I'm using SSO.
I've got a ~/.aws/config file containing multiple profiles, including [profile foo] and [profile bar].
I've set AWS_DEFAULT_PROFILE=foo and AWS_PROFILE=bar.
I'm logged in - the aws cli is working and accessing stuff in "bar" (as expected).
I've got a python script that gets a session by calling boto3.Session()
...but client calls fail unless I unset AWS_DEFAULT_PROFILE.

Expected Behavior

I expected boto3.Session() to use the profile from $AWS_PROFILE and to ignore $AWS_DEFAULT_PROFILE (because $AWS_PROFILE takes precedence and it was set).

i.e. the default should be ignored if a non-default value is provided.

Current Behavior

Traceback (most recent call last):
  File "~/test.py", line 5, in <module>
    cw.describe_alarms(AlarmNamePrefix="i2o-uk-test-pre-prod")
  File "/usr/local/lib/python3.9/site-packages/botocore/client.py", line 535, in _api_call
    return self._make_api_call(operation_name, kwargs)
  File "/usr/local/lib/python3.9/site-packages/botocore/client.py", line 963, in _make_api_call
    http, parsed_response = self._make_request(
  File "/usr/local/lib/python3.9/site-packages/botocore/client.py", line 986, in _make_request
    return self._endpoint.make_request(operation_model, request_dict)
  File "/usr/local/lib/python3.9/site-packages/botocore/endpoint.py", line 119, in make_request
    return self._send_request(request_dict, operation_model)
  File "/usr/local/lib/python3.9/site-packages/botocore/endpoint.py", line 198, in _send_request
    request = self.create_request(request_dict, operation_model)
  File "/usr/local/lib/python3.9/site-packages/botocore/endpoint.py", line 134, in create_request
    self._event_emitter.emit(
  File "/usr/local/lib/python3.9/site-packages/botocore/hooks.py", line 412, in emit
    return self._emitter.emit(aliased_event_name, **kwargs)
  File "/usr/local/lib/python3.9/site-packages/botocore/hooks.py", line 256, in emit
    return self._emit(event_name, kwargs)
  File "/usr/local/lib/python3.9/site-packages/botocore/hooks.py", line 239, in _emit
    response = handler(**kwargs)
  File "/usr/local/lib/python3.9/site-packages/botocore/signers.py", line 105, in handler
    return self.sign(operation_name, request)
  File "/usr/local/lib/python3.9/site-packages/botocore/signers.py", line 180, in sign
    auth = self.get_auth_instance(**kwargs)
  File "/usr/local/lib/python3.9/site-packages/botocore/signers.py", line 284, in get_auth_instance
    frozen_credentials = self._credentials.get_frozen_credentials()
  File "/usr/local/lib/python3.9/site-packages/botocore/credentials.py", line 611, in get_frozen_credentials
    self._refresh()
  File "/usr/local/lib/python3.9/site-packages/botocore/credentials.py", line 499, in _refresh
    self._protected_refresh(is_mandatory=is_mandatory_refresh)
  File "/usr/local/lib/python3.9/site-packages/botocore/credentials.py", line 515, in _protected_refresh
    metadata = self._refresh_using()
  File "/usr/local/lib/python3.9/site-packages/botocore/credentials.py", line 662, in fetch_credentials
    return self._get_cached_credentials()
  File "/usr/local/lib/python3.9/site-packages/botocore/credentials.py", line 672, in _get_cached_credentials
    response = self._get_credentials()
  File "/usr/local/lib/python3.9/site-packages/botocore/credentials.py", line 2117, in _get_credentials
    token = self._token_loader(self._start_url)['accessToken']
  File "/usr/local/lib/python3.9/site-packages/botocore/utils.py", line 3164, in __call__
    raise SSOTokenLoadError(error_msg=error_msg)
botocore.exceptions.SSOTokenLoadError: Error loading SSO Token: Token for https://d-12345abcde.awsapps.com/start#/ does not exist

Reproduction Steps

set AWS_DEFAULT_PROFILE=foo and AWS_PROFILE=bar, where foo and bar are valid aws profiles described in ~/.aws/config
Login to aws by doing aws sso login
Check aws cli is working; it should be accessing stuff in "bar".

Run a python script:

#!/usr/bin/python 
import boto3
session = boto3.Session()
cw = session.client('cloudwatch')
cw.describe_alarms(AlarmNamePrefix="something-that-exists-in-foo")

... then instead of getting alarms read (which this script doesn't output), we get an exception thrown:
botocore.exceptions.SSOTokenLoadError: Error loading SSO Token: Token for https://d-12345abcde.awsapps.com/start#/ does not exist

If I unset AWS_DEFAULT_PROFILE or if I set AWS_DEFAULT_PROFILE to be == $AWS_PROFILE then things work again.
Similarly, if I change the code to say session = boto3.Session(profile_name='bar') then it all works.

Possible Solution

I suspect that some (if not all) of the code is looking at AWS_DEFAULT_PROFILE when it should be looking at "AWS_PROFILE if that's set, otherwise use AWS_DEFAULT_PROFILE".
I found that if I did export AWS_DEFAULT_PROFILE=$AWS_PROFILE or I unset AWS_DEFAULT_PROFILE then boto3 was happy again ... but "in an ideal world" these workarounds shouldn't be necessary.

Additional Information/Context

As an end user, having different AWS technologies use different authentication controls & requirements is confusing. I want them all to work the same way, so as the aws cli uses AWS_PROFILE if it's set but defaulting to AWS_DEFAULT_PROFILE (if AWS_PROFILE isn't set) then that's the behaviour I expect from boto3.

It might be that this issue was the underlying cause of the confusion reported in #913, although these two scenarios are significantly different.


Environment details:

I'm using a docker container running under Windows WSL2 & docker desktop (layers in layers in layers...) but the short version is that "it's basically linux".

python --version reports Python 3.9.16

pip show boto3 reports:

Name: boto3
Version: 1.28.57
Summary: The AWS SDK for Python
Home-page: https://github.com/boto/boto3
Author: Amazon Web Services
Author-email: 
License: Apache License 2.0
Location: /usr/local/lib/python3.9/site-packages
Requires: jmespath, botocore, s3transfer
Required-by: aws-sam-translator

...and pip show botocore tells me it's Version: 1.31.57

SDK version used

boto3 version 1.28.57

Environment details (OS name and version, etc.)

Linux 752a5f9ee677 5.10.102.1-microsoft-standard-WSL2 #1 SMP Wed Mar 2 00:30:59 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux with python 3.9.16

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugThis issue is a confirmed bug.credentialsneeds-major-versionCan only be considered for the next major releasep2This is a standard priority issue

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions