clarify AWS config

[kidbrax]

I'm having issues running a simple query due to my AWS creds and the docs weren't quite clear enough.

I updated ~/.steampipe/config/default.spc and edited the default AWS profile to:

connection "gov_dev" {
  plugin      = "aws"
  profile     = "gov-dev"
  regions     = ["us-gov-west-1"]
}

When I run select title from gov_dev.aws_vpc_route_table. I get:

Error: NoCredentialProviders: no valid providers in chain. Deprecated.
        For verbose messaging see aws.Config.CredentialsChainVerboseErrors

So, a few questions:

1. Does filename of the config file matter? Does it correspond to AWS profile name at all?
2. I initially had the connection name as gov-dev to match the actual profile name, but it told me that was invalid. So I guess dashes aren't allowed? Does that matter? Does it need to match the profile name?
3. How can I set aws.Config.CredentialsChainVerboseErrors to get more verbose output?

[cbruno10]

Hey @kidbrax , sorry you ran into issues setting up Steampipe with your AWS creds! Hopefully some of my responses/follow-ups are helpful below.

1. The filename of the config file doesn't need to correspond to the profile or connection name; as long as there are aws plugin connections in those files, Steampipe will load them, e.g.:

Codys-MacBook-Pro:~ cbruno$ cat ~/.steampipe/config/test.spc
connection "my_aws_connection" {
  plugin  = "aws"
  profile = "govcloud-creds"
  regions = ["us-gov-west-1"]
}

Codys-MacBook-Pro:~ cbruno$ cat ~/.aws/credentials
[default]
aws_access_key_id = ***
aws_secret_access_key = ***

[govcloud-creds]
aws_access_key_id = ***
aws_secret_access_key = ***

Codys-MacBook-Pro:~ cbruno$ cat ~/.aws/config
[default]
output = json
region = us-gov-west-1

Codys-MacBook-Pro:~ cbruno$ steampipe query "select arn from aws_iam_role;"
+---------------------------------------------------------------------------------------------------------------------+
| arn                                                                                                                 |
+---------------------------------------------------------------------------------------------------------------------+
| arn:aws-us-gov:iam::123456789012:role/aws-service-role/trustedadvisor.amazonaws.com/AWSServiceRoleForTrustedAdvisor |
| arn:aws-us-gov:iam::123456789012:role/aws-service-role/support.amazonaws.com/AWSServiceRoleForSupport               |
+---------------------------------------------------------------------------------------------------------------------+

2. The connection name doesn't have to match the profile name, but can't include hyphens. The error message isn't super helpful right now if you do include hyphens, so I'll open an issue to improve that.
3. This one I'm not sure on, but if you have time, can you please run through some of the steps below which can help us get more information about the issue?

• Are you able to run AWS CLI commands using the profile creds you're using in ~/.steampipe/config/default.spc, e.g., aws --profile gov-dev --region us-gov-west-1 s3 ls?
• Are you able to run Steampipe queries using any other profiles in GovCloud or Commercial regions?
• Can you please paste your ~/.aws/config file contents here as well?

If you have anymore questions in the meantime, please let us know, thanks!

[kidbrax]

Thanks for the detailed reply. I am able to run regular AWS CLI queries. Here is relevant output:

AWS Config:

[nonroot@b5c17e1509c6 app]$ cat ~/.aws/config 
[default]
output=json

[profile gov-dev]
region = us-gov-west-1

[profile gov-stage]
region = us-gov-west-1

[profile gov-prod]
region = us-gov-west-1

AWS CLI call:

[nonroot@b5c17e1509c6 app]$ aws sts get-caller-identity --profile gov-dev
{
    "UserId": "********:*****@*****.com",
    "Account": "*******",
    "Arn": "arn:aws-us-gov:sts::********:assumed-role/govcloud_devops_iam/*****@******.com"
}

And Steampipe calls. I'd also like some clarification on the format of the calls. My understanding is that the should be select x from <steampipe_connection_name>.table:

[nonroot@b5c17e1509c6 app]$ steampipe query "select arn from aws_iam_role;"
Warning: executeQueries: query 1 of 1 failed: pq: rpc error: code = Unknown desc = InvalidClientTokenId: The security token included in the request is invalid
        status code: 403, request id: 14051add-fe0b-452c-8042-501963ed4977

[nonroot@b5c17e1509c6 app]$ steampipe query "select arn from gov_dev.aws_iam_role;"
Warning: executeQueries: query 1 of 1 failed: pq: rpc error: code = Unknown desc = InvalidClientTokenId: The security token included in the request is invalid
        status code: 403, request id: feba460d-db0d-48d9-8e1f-c17541c86288

And here are my Steampipe configs:

[nonroot@b5c17e1509c6 app]$ cat ~/.steampipe/config/default.spc 

#
# For detailed descriptions, see the reference documentation
# at https://steampipe.io/docs/reference/cli-args
#

# options "connection" {
#   cache     = true # true, false
#   cache_ttl = 300  # expiration (TTL) in seconds
# }

# options "database" {
#   port        = 9193    # any valid, open port number
#   listen      = "local" # local, network
#   search_path =  ""     # comma-separated string
# }

# options "terminal" {
#   multi               = false   # true, false
#   output              = "table" # json, csv, table, line
#   header              = true    # true, false
#   separator           = ","     # any single char
#   timing              = false   # true, false
#   search_path         =  ""     # comma-separated string
#   search_path_prefix  =  ""     # comma-separated string
#   watch                           =  true   # true, false
# }

# options "general" {
#   update_check = true # true, false
# }

[nonroot@b5c17e1509c6 app]$ cat ~/.steampipe/config/aws.spc     
connection "gov_dev" {
  plugin    = "aws"    
  profile   = "gov-dev"
}

[kidbrax]

Ok, I got it to work using AWS_PROFILE=gov-dev steampipe query "select arn from gov_dev.aws_iam_role;". I was missing the AWS_PROFILE var.

But it also works with AWS_PROFILE=gov-dev steampipe query "select arn from aws_iam_role;" (without the connection name in the query).

This is very confusing. When do I need to specify the profile in the config file vs pass the environment variable? It seems that the ENV var is required but the config file isn't? Regardless, it would help if this was better documented.

[cbruno10]

Hey @kidbrax , I'm glad to hear you eventually got it working, but not great that you had to walk through a few different steps to get there, so sorry about that!

Out of curiosity, do you have any other AWS environment variables exported/set in your local environment?

[kidbrax]

There are no other AWS env vars.