Wrong AWS profile being retrieved in case of non default profile configured

[grufino]

There is already an issue reported by another user which I was able to reproduce in a slightly different scenario, but the error and conditions are apparently the same: #118

I drilled down and was able to identify the exact error scenario, steps to reproduce:

1. Configure a dbt profile that uses a non default aws profile, e.g.:

raptor:
  outputs:
    dev:
      database: awsdatacatalog
      region_name: us-west-2
      s3_staging_dir: s3://athenagraphs/athena/
      schema: graphs_db
      aws_profile_name: athena
      type: athena
  target: dev

2. Have the same config on your ~/.aws/credentials and ~/.aws/config files, e.g.:
   credentials:

[athena]
aws_access_key_id = ***********************
aws_secret_access_key = ***********************************

config:

[athena]
region = us-west-2

3. create a data materialization in dbt (anything will work, just materialize to a table) and try to run it twice.

In the first time it will be created fine, but in the second, when it has to run the macro drop_relation (which maps to the python function clean_up_table), there will be an error in the dbt logs:

17:44:02.203689 [debug] [Thread-1 ]: Athena adapter: Error running SQL: macro drop_relation
17:44:02.212093 [error] [MainThread]: local variable 'table' referenced before assignment

Which when debugged from within python, inside the function clean_up_table, you notice the following:

ipdb> table = glue_client.get_table(
                DatabaseName=database_name,
                Name=table_name
            )
*** botocore.exceptions.ClientError: An error occurred (UnrecognizedClientException) when calling the GetTable operation: The security token included in the request is invalid.

So I noticed that the profile being used by boto3 was the default, it was trying to use my default profile AWS credentials to run this command, which justifies the credentials error. I have also tested that without specifying a profile (so it tries to get the default one) this error doesn't occur.

The solution is very specific to the problem I had, but I believe it might be an issue for the other functions that use the same logic to acquire a client, I just don't have scenarios to test all of them so changed only this one. I tried to look for an earlier place in the code to put this so it would already be the default when needed and this function wouldn't have to worry about it, but since we only have the profile after opening the connection, this is the best I could do, please let me know if you can see a better place to put this.

[cstruct]

This is a duplicate of #56 and #121. Setting boto3.setup_default_session there also won't fix the issue in all cases.