DBT_TEST_ATHENA_AWS_PROFILE_NAME should be optional if using SSO

[mattiamatrix]

DBT_TEST_ATHENA_AWS_PROFILE_NAME should be optional for those who use SSO and don't have a profile

dbt-athena/tests/conftest.py

Line 23 in f31a863

"aws_profile_name": os.getenv("DBT_TEST_ATHENA_AWS_PROFILE_NAME"),

[Jrmyy]

DBT_TEST_ATHENA_AWS_PROFILE_NAME is optional : indeed if it is not passed, the value will be None and then it is the same as not putting the value, leading to AthenaConnection having default profile.

[mattiamatrix]

DBT_TEST_ATHENA_AWS_PROFILE_NAME is optional : indeed if it is not passed, the value will be None and then it is the same as not putting the value, leading to AthenaConnection having default profile.

But I don't use any profile because I export AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, AWS_SESSION_TOKEN directly in the terminal. This works well with dbt run or dbt compile.

It works if I comment out `"aws_profile_name": os.getenv("DBT_TEST_ATHENA_AWS_PROFILE_NAME"),

It will be needed if make test is executed by a job that assumes an IAM Role instead of having the credentials of IAM User.

[nicor88]

@mattiamatrix I've your same setup, I'm assuming a role using sso. I don't think that we need to change anything here, as in the cases where we use AWS_ env (doesn't matter how they are set), we can just ignore to setup the DBT_TEST_ATHENA_AWS_PROFILE_NAME.

[mattiamatrix]

mmh, @nicor88 can you run make test locally? I can't with the current setup. Let me know if you can reproduce.

My .env looks like this make test fails.

DBT_TEST_ATHENA_S3_STAGING_DIR=s3://athena-output-results-eu-west-2
DBT_TEST_ATHENA_REGION_NAME=eu-west-2
DBT_TEST_ATHENA_DATABASE=awsdatacatalog
DBT_TEST_ATHENA_SCHEMA=athena_dbt_tests
DBT_TEST_ATHENA_WORK_GROUND=primary
# DBT_TEST_ATHENA_AWS_PROFILE_NAME=

I believe this is because

@pytest.fixture(scope="class")
def dbt_profile_target():
    return {
        "type": "athena",
        "s3_staging_dir": os.getenv("DBT_TEST_ATHENA_S3_STAGING_DIR"),
        "schema": os.getenv("DBT_TEST_ATHENA_SCHEMA"),
        "database": os.getenv("DBT_TEST_ATHENA_DATABASE"),
        "region_name": os.getenv("DBT_TEST_ATHENA_REGION_NAME"),
        "threads": 1,
        "num_retries": 0,
        "work_group": os.getenv("DBT_TEST_ATHENA_WORK_GROUND"),
        "aws_profile_name": os.getenv("DBT_TEST_ATHENA_AWS_PROFILE_NAME"),
    }

if DBT_TEST_ATHENA_AWS_PROFILE_NAME is not defined, it returns

"aws_profile_name": None

But, this makes AthenaCredentials or boto3 look for a default profile that doesn't exist in my setup.

I can make it works with

@pytest.fixture(scope="class")
def dbt_profile_target():
    return {
        "type": "athena",
        "s3_staging_dir": os.getenv("DBT_TEST_ATHENA_S3_STAGING_DIR"),
        "schema": os.getenv("DBT_TEST_ATHENA_SCHEMA"),
        "database": os.getenv("DBT_TEST_ATHENA_DATABASE"),
        "region_name": os.getenv("DBT_TEST_ATHENA_REGION_NAME"),
        "threads": 1,
        "num_retries": 0,
        "work_group": os.getenv("DBT_TEST_ATHENA_WORK_GROUND"),
        # "aws_profile_name": os.getenv("DBT_TEST_ATHENA_AWS_PROFILE_NAME"),
    }

[Jrmyy]

I updated the PR, I tested with SSO user, it works now, let me know if it works for you.
There was indeed an issue with the aws_profile_name set as blank string instead of None ...
"aws_profile_name": os.getenv("DBT_TEST_ATHENA_AWS_PROFILE_NAME") or None
with an empty env variable works for me.