Support authenticating with credentials from VM Metadata

[dpowley]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment. If the issue is assigned to the "modular-magician" user, it is either in the process of being autogenerated, or is planned to be autogenerated soon. If the issue is assigned to a user, that user is claiming responsibility for the issue. If the issue is assigned to "hashibot", a community member has claimed the issue already.

Description

The GCP SDKs are able to pick up the following credentials by default (for reference, I am citing the Python SDK docs):

1. If the environment variable GOOGLE_APPLICATION_CREDENTIALS is set to the path of a valid service account JSON private key file, then it is loaded and returned. The project ID returned is the project ID defined in the service account file if available (some older files do not contain project ID information).

2. If the Google Cloud SDK is installed and has application default credentials set they are loaded and returned.

To enable application default credentials with the Cloud SDK run:

gcloud auth application-default login
If the Cloud SDK has an active project, the project ID is returned. The active project can be set using:

gcloud config set project

3. If the application is running in the App Engine standard environment (first generation) then the credentials and project ID from the App Identity Service are used.

4. If the application is running in Compute Engine or Cloud Run or the App Engine flexible environment or the App Engine standard environment (second generation) then the credentials and project ID are obtained from the Metadata Service.

I specifically would like to see support for #4, so if authenticating with the TF GCP provider on a VM, it will by default use the credentials of the service account provided in the VMs metadata service.

Currently, to authenticate with the provider on a VM, you would need to have a service account key stored on the VM and the GOOGLE_APPLICATION_CREDENTIALS would need to be set to point the provider to that file. With this feature request, you would be able to forego storing service account keys on the VM. The metadata service would vend the provider an access token that can be used to authenticate with Google.

Potential Terraform Configuration

There wouldn't be a change to Terraform files as credentials from the metadata service would be picked up by default. Note that ideally the provider would recognize the GCE_METADATA_HOST env variable similar to how the SDKs are written to query from the metadata service. If that env variable is not set, it would default to querying from metadata.google.internal

cc @megan07

[dpowley]

Closing, as TF finds defaults to the default credentials, which includes querying metadata service.

[github-actions]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.