Use of AWS non default credentials profile - Same issue as closed #18402 #20599

Marcus-James-Adams · 2019-03-07T12:12:14Z

Terraform version 0.11.11 on Ubuntu 16.04
gives the following error

$ AWS_PROFILE=marcus terraform init
Initializing modules...
- module.vpc

Initializing the backend...

Error configuring the backend "s3": No valid credential sources found for AWS Provider.
  Please see https://terraform.io/docs/providers/aws/index.html for more information on
  providing credentials for the AWS Provider

Please update the configuration in your Terraform files to fix this error.
If you'd like to update the configuration interactively without storing
the values in your configuration, run "terraform init".

AWS config file

[default]
output = json
region = eu-west-2

[profile marcusa]
role_arn = arn:aws:iam::xxxxxxxxxxxxx:role/marcusa
source_profile = default
region = us-east-1
output = json

credentials

[default]
aws_access_key_id = xxxxxxxxxxxxxxxxxxxxxxxxxxxxx
aws_secret_access_key = xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx

following some of the suggestions in #18402
I tried
AWS_SDK_LOAD_CONFIG=1 AWS_PROFILE=marcus terraform init
this then works ok.

The text was updated successfully, but these errors were encountered:

brilong · 2019-03-14T17:56:13Z

I have confirmed with 0.11.13 that this is still an issue. Can we get some traction on fixing this once and for all? The previous bug and PR did NOT fix the issue.

sksinghvi · 2019-03-17T23:27:24Z

Setting AWS_PROFILE during init doesn't work for me. I get following error:
Error configuring the backend "s3": NoCredentialProviders: no valid providers in chain. Deprecated.
For verbose messaging see aws.Config.CredentialsChainVerboseErrors

I don't have credential file in .aws folder. Instead I have assumed role something like this in my config file
[profile okta-sandbox-proxy]
credential_process = aws-okta exec okta-sandbox -- cmd /c C:\tools\proxy-aws-okta-result.bat
region = us-west-2
[profile okta-sandbox]
aws_saml_url = home/amazon_aws/0oahmb006mRgL5aHI0x7/272
role_arn = arn:aws:iam::XXXXXXXXXX:role/VA-Role-Devops-L3
region = us-west-2

Not sure what the alternate solution is. Anybody can suggest the alternate solution to create s3 backend until main issue gets fixed.

dtelaroli · 2019-03-19T13:36:14Z

It's just working if I use AWS_PROFILE=<my_profile> terraform init with S3 Backend.

This issue was reported many times and the terraform team is ignoring it, causing repeated problem report.

timothyclarke · 2020-03-12T13:35:01Z

Just my $0.02
I was receiving Error: Invalid AWS Region: when I was using s3 for state and forgot to add -backend-config="bucket=<terraform-bucket-state>" -backend-config="region=<region>"
doing a terraform 12.5 -> 12.23 required the backend to be re-initialized
It was a little annoying that it didn't flag the error was related to the s3 bucket

Reference: #13410 Reference: #18774 Reference: #19482 Reference: #20062 Reference: #20599 Reference: #22103 Reference: #22161 Reference: #22601 Reference: #22992 Reference: #24252 Reference: #24253 Reference: #24480 Reference: #25056 Changes: ``` NOTES * backend/s3: Deprecated `lock_table`, `skip_get_ec2_platforms`, `skip_requesting_account_id` arguments have been removed * backend/s3: Credential ordering has changed from static, environment, shared credentials, EC2 metadata, default AWS Go SDK (shared configuration, web identity, ECS, EC2 Metadata) to static, environment, shared credentials, default AWS Go SDK (shared configuration, web identity, ECS, EC2 Metadata) * The `AWS_METADATA_TIMEOUT` environment variable no longer has any effect as we now depend on the default AWS Go SDK EC2 Metadata client timeout of one second with two retries ENHANCEMENTS * backend/s3: Always enable shared configuration file support (no longer require `AWS_SDK_LOAD_CONFIG` environment variable) * backend/s3: Automatically expand `~` prefix for home directories in `shared_credentials_file` argument * backend/s3: Add `assume_role_duration_seconds`, `assume_role_policy_arns`, `assume_role_tags`, and `assume_role_transitive_tag_keys` arguments BUG FIXES * backend/s3: Ensure configured profile is used * backend/s3: Ensure configured STS endpoint is used during AssumeRole API calls * backend/s3: Prefer AWS shared configuration over EC2 metadata credentials * backend/s3: Prefer ECS credentials over EC2 metadata credentials * backend/s3: Remove hardcoded AWS Provider messaging ``` Output from acceptance testing: ``` --- PASS: TestBackend (16.32s) --- PASS: TestBackendConfig (0.58s) --- PASS: TestBackendConfig_AssumeRole (0.02s) --- PASS: TestBackendConfig_conflictingEncryptionSchema (0.00s) --- PASS: TestBackendConfig_invalidKey (0.00s) --- PASS: TestBackendConfig_invalidSSECustomerKeyEncoding (0.00s) --- PASS: TestBackendConfig_invalidSSECustomerKeyLength (0.00s) --- PASS: TestBackendExtraPaths (13.21s) --- PASS: TestBackendLocked (28.98s) --- PASS: TestBackendPrefixInWorkspace (5.65s) --- PASS: TestBackendSSECustomerKey (17.60s) --- PASS: TestBackend_impl (0.00s) --- PASS: TestForceUnlock (17.50s) --- PASS: TestKeyEnv (50.25s) --- PASS: TestRemoteClient (4.78s) --- PASS: TestRemoteClientLocks (16.85s) --- PASS: TestRemoteClient_clientMD5 (12.08s) --- PASS: TestRemoteClient_impl (0.00s) --- PASS: TestRemoteClient_stateChecksum (17.92s) ```

* deps: Update github.com/hashicorp/aws-sdk-go-base@v0.5.0 Updated via: ``` $ go get github.com/hashicorp/aws-sdk-go-base@v0.5.0 $ go mod tidy $ go mod vendor ``` * backend/s3: Updates for Terraform v0.13.0 Reference: #13410 Reference: #18774 Reference: #19482 Reference: #20062 Reference: #20599 Reference: #22103 Reference: #22161 Reference: #22601 Reference: #22992 Reference: #24252 Reference: #24253 Reference: #24480 Reference: #25056 Changes: ``` NOTES * backend/s3: Deprecated `lock_table`, `skip_get_ec2_platforms`, `skip_requesting_account_id` arguments have been removed * backend/s3: Credential ordering has changed from static, environment, shared credentials, EC2 metadata, default AWS Go SDK (shared configuration, web identity, ECS, EC2 Metadata) to static, environment, shared credentials, default AWS Go SDK (shared configuration, web identity, ECS, EC2 Metadata) * The `AWS_METADATA_TIMEOUT` environment variable no longer has any effect as we now depend on the default AWS Go SDK EC2 Metadata client timeout of one second with two retries ENHANCEMENTS * backend/s3: Always enable shared configuration file support (no longer require `AWS_SDK_LOAD_CONFIG` environment variable) * backend/s3: Automatically expand `~` prefix for home directories in `shared_credentials_file` argument * backend/s3: Add `assume_role_duration_seconds`, `assume_role_policy_arns`, `assume_role_tags`, and `assume_role_transitive_tag_keys` arguments BUG FIXES * backend/s3: Ensure configured profile is used * backend/s3: Ensure configured STS endpoint is used during AssumeRole API calls * backend/s3: Prefer AWS shared configuration over EC2 metadata credentials * backend/s3: Prefer ECS credentials over EC2 metadata credentials * backend/s3: Remove hardcoded AWS Provider messaging ``` Output from acceptance testing: ``` --- PASS: TestBackend (16.32s) --- PASS: TestBackendConfig (0.58s) --- PASS: TestBackendConfig_AssumeRole (0.02s) --- PASS: TestBackendConfig_conflictingEncryptionSchema (0.00s) --- PASS: TestBackendConfig_invalidKey (0.00s) --- PASS: TestBackendConfig_invalidSSECustomerKeyEncoding (0.00s) --- PASS: TestBackendConfig_invalidSSECustomerKeyLength (0.00s) --- PASS: TestBackendExtraPaths (13.21s) --- PASS: TestBackendLocked (28.98s) --- PASS: TestBackendPrefixInWorkspace (5.65s) --- PASS: TestBackendSSECustomerKey (17.60s) --- PASS: TestBackend_impl (0.00s) --- PASS: TestForceUnlock (17.50s) --- PASS: TestKeyEnv (50.25s) --- PASS: TestRemoteClient (4.78s) --- PASS: TestRemoteClientLocks (16.85s) --- PASS: TestRemoteClient_clientMD5 (12.08s) --- PASS: TestRemoteClient_impl (0.00s) --- PASS: TestRemoteClient_stateChecksum (17.92s) ```

bflad · 2020-06-05T20:48:51Z

Multiple fixes for credential ordering, automatically using the AWS shared configuration file if present, and profile configuration handling of the S3 Backend have been merged and will release with version 0.13.0-beta2 of Terraform.

* deps: Update github.com/hashicorp/aws-sdk-go-base@v0.5.0 Updated via: ``` $ go get github.com/hashicorp/aws-sdk-go-base@v0.5.0 $ go mod tidy $ go mod vendor ``` * backend/s3: Updates for Terraform v0.13.0 Reference: #13410 Reference: #18774 Reference: #19482 Reference: #20062 Reference: #20599 Reference: #22103 Reference: #22161 Reference: #22601 Reference: #22992 Reference: #24252 Reference: #24253 Reference: #24480 Reference: #25056 Changes: ``` NOTES * backend/s3: Deprecated `lock_table`, `skip_get_ec2_platforms`, `skip_requesting_account_id` arguments have been removed * backend/s3: Credential ordering has changed from static, environment, shared credentials, EC2 metadata, default AWS Go SDK (shared configuration, web identity, ECS, EC2 Metadata) to static, environment, shared credentials, default AWS Go SDK (shared configuration, web identity, ECS, EC2 Metadata) * The `AWS_METADATA_TIMEOUT` environment variable no longer has any effect as we now depend on the default AWS Go SDK EC2 Metadata client timeout of one second with two retries ENHANCEMENTS * backend/s3: Always enable shared configuration file support (no longer require `AWS_SDK_LOAD_CONFIG` environment variable) * backend/s3: Automatically expand `~` prefix for home directories in `shared_credentials_file` argument * backend/s3: Add `assume_role_duration_seconds`, `assume_role_policy_arns`, `assume_role_tags`, and `assume_role_transitive_tag_keys` arguments BUG FIXES * backend/s3: Ensure configured profile is used * backend/s3: Ensure configured STS endpoint is used during AssumeRole API calls * backend/s3: Prefer AWS shared configuration over EC2 metadata credentials * backend/s3: Prefer ECS credentials over EC2 metadata credentials * backend/s3: Remove hardcoded AWS Provider messaging ``` Output from acceptance testing: ``` --- PASS: TestBackend (16.32s) --- PASS: TestBackendConfig (0.58s) --- PASS: TestBackendConfig_AssumeRole (0.02s) --- PASS: TestBackendConfig_conflictingEncryptionSchema (0.00s) --- PASS: TestBackendConfig_invalidKey (0.00s) --- PASS: TestBackendConfig_invalidSSECustomerKeyEncoding (0.00s) --- PASS: TestBackendConfig_invalidSSECustomerKeyLength (0.00s) --- PASS: TestBackendExtraPaths (13.21s) --- PASS: TestBackendLocked (28.98s) --- PASS: TestBackendPrefixInWorkspace (5.65s) --- PASS: TestBackendSSECustomerKey (17.60s) --- PASS: TestBackend_impl (0.00s) --- PASS: TestForceUnlock (17.50s) --- PASS: TestKeyEnv (50.25s) --- PASS: TestRemoteClient (4.78s) --- PASS: TestRemoteClientLocks (16.85s) --- PASS: TestRemoteClient_clientMD5 (12.08s) --- PASS: TestRemoteClient_impl (0.00s) --- PASS: TestRemoteClient_stateChecksum (17.92s) ```

ghost · 2020-07-06T01:52:24Z

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.

jbardin added the backend/s3 label Mar 8, 2019

scalp42 mentioned this issue Mar 21, 2019

Invalid AWS Region hashicorp/terraform-provider-aws#8040

Closed

aeschright added the provider/aws label Sep 13, 2019

bflad self-assigned this Jun 2, 2020

bflad added this to the v0.13.0 milestone Jun 2, 2020

bflad mentioned this issue Jun 2, 2020

Always Enable Shared Configuration File Support hashicorp/aws-sdk-go-base#35

Closed

bflad mentioned this issue Jun 4, 2020

backend/s3: Updates for Terraform v0.13.0 #25134

Merged

bflad closed this as completed in #25134 Jun 5, 2020

ghost locked and limited conversation to collaborators Jul 6, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Use of AWS non default credentials profile - Same issue as closed #18402 #20599

Use of AWS non default credentials profile - Same issue as closed #18402 #20599

Marcus-James-Adams commented Mar 7, 2019

brilong commented Mar 14, 2019

sksinghvi commented Mar 17, 2019 •

edited

dtelaroli commented Mar 19, 2019

timothyclarke commented Mar 12, 2020 •

edited

bflad commented Jun 5, 2020

ghost commented Jul 6, 2020

Use of AWS non default credentials profile - Same issue as closed #18402 #20599

Use of AWS non default credentials profile - Same issue as closed #18402 #20599

Comments

Marcus-James-Adams commented Mar 7, 2019

brilong commented Mar 14, 2019

sksinghvi commented Mar 17, 2019 • edited

dtelaroli commented Mar 19, 2019

timothyclarke commented Mar 12, 2020 • edited

bflad commented Jun 5, 2020

ghost commented Jul 6, 2020

sksinghvi commented Mar 17, 2019 •

edited

timothyclarke commented Mar 12, 2020 •

edited