Load GOOGLE_APPLICATION_CREDENTIALS json content via an environment variable instead of a file #185

Closed
leighmcculloch opened this Issue Jan 12, 2017 · 8 comments

Comments

Projects
None yet
6 participants
@leighmcculloch

馃憢

What from I can see the golang.org/x/oauth2/google package only supports loading the JSON application credentials file from the path given in the GOOGLE_APPLICATION_CREDENTIALS environment variable, without digging deep into the inner workings.

When deploying go apps on Heroku or Pivotal Web Services secrets are normally kept in environment variables. Code and files get pushed together on every deploy and it's not ideal to keep the credentials JSON with the code and it's not easy to manage the credentials file alongside the code since it needs to be committed in the case of Heroku, or present at every push in the case of Pivotal Web Services,

I figured out how to do this but I had to dig through the source and multiple docs. This was the resulting code:

json := os.Getenv("GOOGLE_APPLICATION_CREDENTIALS_JSON") // `{"type": "service_account", "project_id": "my-project", ...}`
ctx := context.Background()
jwtConfig, err := google.JWTConfigFromJSON([]byte(json), datastore.ScopeDatastore)
if err != nil {
	...
}
ts := jwtConfig.TokenSource(ctx)
datastoreClient, err := datastore.NewClient(ctx, projectID, option.WithTokenSource(ts))

I think there need to be another environment variable that will pickup the config and set default credentials without needing it to be in a file, or we need clearer documentation on how to load credentials from elsewhere.

@leighmcculloch leighmcculloch changed the title from Load GOOGLE_APPLICATION_CREDENTIALS json content file via an environment variable instead of a file to Load GOOGLE_APPLICATION_CREDENTIALS json content via an environment variable instead of a file Jan 12, 2017

@zombiezen

This comment has been minimized.

Show comment
Hide comment
@zombiezen

zombiezen Jan 12, 2017

Member

Hey Leigh. Application Default Credentials is a cross-language specification, so we would need to change how all languages work in order to support this use case. As you've seen, it is possible to pass along credentials in your own way, but it sounds like we could improve our documentation around this.

@rakyll, any thoughts on how we could improve?

Member

zombiezen commented Jan 12, 2017

Hey Leigh. Application Default Credentials is a cross-language specification, so we would need to change how all languages work in order to support this use case. As you've seen, it is possible to pass along credentials in your own way, but it sounds like we could improve our documentation around this.

@rakyll, any thoughts on how we could improve?

@rakyll

This comment has been minimized.

Show comment
Hide comment
@rakyll

rakyll Jan 12, 2017

Member

We are documenting how to construct clients with custom token sources (see https://github.com/googlecloudplatform/google-cloud-go#authorization), but we can improve the text around it and give a more comprehensive example rather than having the two line reference.

Member

rakyll commented Jan 12, 2017

We are documenting how to construct clients with custom token sources (see https://github.com/googlecloudplatform/google-cloud-go#authorization), but we can improve the text around it and give a more comprehensive example rather than having the two line reference.

@broady

This comment has been minimized.

Show comment
Hide comment
@broady

broady Jan 12, 2017

Member

Another pattern you might want to look at is to encrypt the key JSON and pass along the secrets in environment variables. You'd need to decrypt the file on application startup and set the GOOGLE_APPLICATION_CREDENTIALS env var.

Member

broady commented Jan 12, 2017

Another pattern you might want to look at is to encrypt the key JSON and pass along the secrets in environment variables. You'd need to decrypt the file on application startup and set the GOOGLE_APPLICATION_CREDENTIALS env var.

@leighmcculloch

This comment has been minimized.

Show comment
Hide comment
@leighmcculloch

leighmcculloch Jan 13, 2017

@zombiezen: Documentation could solve this problem.

@rakyll: Thanks for the link. I was using the Google Developers page https://developers.google.com/identity/protocols/application-default-credentials below as reference point which hadn't mentioned it. The docs seem to live in few places and is confusing to navigate because of that. I see that I should have posted this issue in the googlecloudplatform/google-cloud-go project. The docs you linked to were clearer and it would be helpful of all the Google Developer docs were in sync.

@broady: I don't think I'd want to further complicate the solution by introducing encryption. If the encryption key lives in an environment variable the credentials may as well, in which case the solution I shared above has less moving parts.


I think clearer documentation for this alternative way of providing the credentials to the library would go a long way, but for devs using non-AppEngine/Compute services like Heroku and PWS there is still increased friction. It wouldn't surprise me if the friction of figuring out how to do auth within these PaaS providers contributes to lost adoption.

Closing since @rakyll pointed out this was opened in the wrong project.

@zombiezen: Documentation could solve this problem.

@rakyll: Thanks for the link. I was using the Google Developers page https://developers.google.com/identity/protocols/application-default-credentials below as reference point which hadn't mentioned it. The docs seem to live in few places and is confusing to navigate because of that. I see that I should have posted this issue in the googlecloudplatform/google-cloud-go project. The docs you linked to were clearer and it would be helpful of all the Google Developer docs were in sync.

@broady: I don't think I'd want to further complicate the solution by introducing encryption. If the encryption key lives in an environment variable the credentials may as well, in which case the solution I shared above has less moving parts.


I think clearer documentation for this alternative way of providing the credentials to the library would go a long way, but for devs using non-AppEngine/Compute services like Heroku and PWS there is still increased friction. It wouldn't surprise me if the friction of figuring out how to do auth within these PaaS providers contributes to lost adoption.

Closing since @rakyll pointed out this was opened in the wrong project.

@avsuresh04

This comment has been minimized.

Show comment
Hide comment
@avsuresh04

avsuresh04 Oct 5, 2017

Hi @leighmcculloch,

Do you have the full code to set JSON content inline rather than file. I need in .NET but may be I'll convert from Java to .NET.

Thanks
Anil Jain

Hi @leighmcculloch,

Do you have the full code to set JSON content inline rather than file. I need in .NET but may be I'll convert from Java to .NET.

Thanks
Anil Jain

@leighmcculloch

This comment has been minimized.

Show comment
Hide comment
@leighmcculloch

leighmcculloch Oct 6, 2017

The JSON content isn't something that you create, but something that you get from the Google Cloud Console. You'll want to follow the instructions here to create a new service account key, which is a type of credential you can create which is attached to a service. When it gives you the option of key type, select JSON.

It'll end up looking something like:

{"type": "service_account", "project_id": "my-project", ...}

The JSON content isn't something that you create, but something that you get from the Google Cloud Console. You'll want to follow the instructions here to create a new service account key, which is a type of credential you can create which is attached to a service. When it gives you the option of key type, select JSON.

It'll end up looking something like:

{"type": "service_account", "project_id": "my-project", ...}
@avsuresh04

This comment has been minimized.

Show comment
Hide comment
@avsuresh04

avsuresh04 Oct 6, 2017

Hi @leighmcculloch
I already have all the credentials required in the form of JSON. All I was looking was for something that could be loaded dynamically in json format pointed from environment variable rather than having it in a specific json file.

Hi @leighmcculloch
I already have all the credentials required in the form of JSON. All I was looking was for something that could be loaded dynamically in json format pointed from environment variable rather than having it in a specific json file.

@Macilias

This comment has been minimized.

Show comment
Hide comment
@Macilias

Macilias Oct 13, 2017

why is this closed? I麓m looking also for a way to bypass using dedicated file for credentials. Plenty environments allow only variables outside of the mounted environment.

why is this closed? I麓m looking also for a way to bypass using dedicated file for credentials. Plenty environments allow only variables outside of the mounted environment.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment