Support passive environmental service account authentication in Google Cloud #1720
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
The call to the `getHost()` method on the `URI` instance returns null for a GCS uri like `gs://my_bucket/my_object`. Checking the URI docs from Oracle (https://docs.oracle.com/en/java/javase/11/docs/api/java.base/java/net/URI.html) we should be using `getAuthority()`. This commit changes the method call resolving the NPE. I've manually tested with a test case hitting a publicly accessible object in Google Cloud Storage (since no unit tests exist).
Previously, we forced using service account private keys set using the GOOGLE_APPLICATION_CREDENTIALS environment variable pointing to a generated json private key. This isn't required in native GCP environments as the official Google Cloud clients will identify if they're running in a GCP service (e.g. GAE, CloudRun, GCE, etc.) and provide automatic authentication as a service account. This change adds a new "authenticationType" and renames the old type: - SERVICE becomes PRIVATE_KEY to explicitly describe that it's expecting to use a private key file. - GCP_ENVIRONMENT is added to be used when running in GCP. The logic is that for PRIVATE_KEY usage we still check the environment for a set GOOGLE_APPLICATION_CREDENTIALS value. A runtime error is thrown if it's not set. For GCP_ENVIRONMENT, the environment check is ignored as the Google client library will take care of validation.
Add in details on the new authentication types and behavior, replacing the old settings. Tweak some verbiage.
sarmbruster
pushed a commit
that referenced
this pull request
Nov 13, 2020
…e Cloud (#1720) * Fix broken GCS support that's throwing NPEs The call to the `getHost()` method on the `URI` instance returns null for a GCS uri like `gs://my_bucket/my_object`. Checking the URI docs from Oracle (https://docs.oracle.com/en/java/javase/11/docs/api/java.base/java/net/URI.html) we should be using `getAuthority()`. This commit changes the method call resolving the NPE. I've manually tested with a test case hitting a publicly accessible object in Google Cloud Storage (since no unit tests exist). * fix typo in Google Cloud doc url * Support GCP environmental service account auth Previously, we forced using service account private keys set using the GOOGLE_APPLICATION_CREDENTIALS environment variable pointing to a generated json private key. This isn't required in native GCP environments as the official Google Cloud clients will identify if they're running in a GCP service (e.g. GAE, CloudRun, GCE, etc.) and provide automatic authentication as a service account. This change adds a new "authenticationType" and renames the old type: - SERVICE becomes PRIVATE_KEY to explicitly describe that it's expecting to use a private key file. - GCP_ENVIRONMENT is added to be used when running in GCP. The logic is that for PRIVATE_KEY usage we still check the environment for a set GOOGLE_APPLICATION_CREDENTIALS value. A runtime error is thrown if it's not set. For GCP_ENVIRONMENT, the environment check is ignored as the Google client library will take care of validation. * Update Google Cloud documentation for new auth Add in details on the new authentication types and behavior, replacing the old settings. Tweak some verbiage.
mneedham
pushed a commit
that referenced
this pull request
Nov 13, 2020
…e Cloud (#1720) * Fix broken GCS support that's throwing NPEs The call to the `getHost()` method on the `URI` instance returns null for a GCS uri like `gs://my_bucket/my_object`. Checking the URI docs from Oracle (https://docs.oracle.com/en/java/javase/11/docs/api/java.base/java/net/URI.html) we should be using `getAuthority()`. This commit changes the method call resolving the NPE. I've manually tested with a test case hitting a publicly accessible object in Google Cloud Storage (since no unit tests exist). * fix typo in Google Cloud doc url * Support GCP environmental service account auth Previously, we forced using service account private keys set using the GOOGLE_APPLICATION_CREDENTIALS environment variable pointing to a generated json private key. This isn't required in native GCP environments as the official Google Cloud clients will identify if they're running in a GCP service (e.g. GAE, CloudRun, GCE, etc.) and provide automatic authentication as a service account. This change adds a new "authenticationType" and renames the old type: - SERVICE becomes PRIVATE_KEY to explicitly describe that it's expecting to use a private key file. - GCP_ENVIRONMENT is added to be used when running in GCP. The logic is that for PRIVATE_KEY usage we still check the environment for a set GOOGLE_APPLICATION_CREDENTIALS value. A runtime error is thrown if it's not set. For GCP_ENVIRONMENT, the environment check is ignored as the Google client library will take care of validation. * Update Google Cloud documentation for new auth Add in details on the new authentication types and behavior, replacing the old settings. Tweak some verbiage.
mneedham
pushed a commit
that referenced
this pull request
Nov 19, 2020
…e Cloud (#1720) * Fix broken GCS support that's throwing NPEs The call to the `getHost()` method on the `URI` instance returns null for a GCS uri like `gs://my_bucket/my_object`. Checking the URI docs from Oracle (https://docs.oracle.com/en/java/javase/11/docs/api/java.base/java/net/URI.html) we should be using `getAuthority()`. This commit changes the method call resolving the NPE. I've manually tested with a test case hitting a publicly accessible object in Google Cloud Storage (since no unit tests exist). * fix typo in Google Cloud doc url * Support GCP environmental service account auth Previously, we forced using service account private keys set using the GOOGLE_APPLICATION_CREDENTIALS environment variable pointing to a generated json private key. This isn't required in native GCP environments as the official Google Cloud clients will identify if they're running in a GCP service (e.g. GAE, CloudRun, GCE, etc.) and provide automatic authentication as a service account. This change adds a new "authenticationType" and renames the old type: - SERVICE becomes PRIVATE_KEY to explicitly describe that it's expecting to use a private key file. - GCP_ENVIRONMENT is added to be used when running in GCP. The logic is that for PRIVATE_KEY usage we still check the environment for a set GOOGLE_APPLICATION_CREDENTIALS value. A runtime error is thrown if it's not set. For GCP_ENVIRONMENT, the environment check is ignored as the Google client library will take care of validation. * Update Google Cloud documentation for new auth Add in details on the new authentication types and behavior, replacing the old settings. Tweak some verbiage.
conker84
pushed a commit
to conker84/neo4j-apoc-procedures
that referenced
this pull request
Mar 29, 2021
…e Cloud (neo4j-contrib#1720) * Fix broken GCS support that's throwing NPEs The call to the `getHost()` method on the `URI` instance returns null for a GCS uri like `gs://my_bucket/my_object`. Checking the URI docs from Oracle (https://docs.oracle.com/en/java/javase/11/docs/api/java.base/java/net/URI.html) we should be using `getAuthority()`. This commit changes the method call resolving the NPE. I've manually tested with a test case hitting a publicly accessible object in Google Cloud Storage (since no unit tests exist). * fix typo in Google Cloud doc url * Support GCP environmental service account auth Previously, we forced using service account private keys set using the GOOGLE_APPLICATION_CREDENTIALS environment variable pointing to a generated json private key. This isn't required in native GCP environments as the official Google Cloud clients will identify if they're running in a GCP service (e.g. GAE, CloudRun, GCE, etc.) and provide automatic authentication as a service account. This change adds a new "authenticationType" and renames the old type: - SERVICE becomes PRIVATE_KEY to explicitly describe that it's expecting to use a private key file. - GCP_ENVIRONMENT is added to be used when running in GCP. The logic is that for PRIVATE_KEY usage we still check the environment for a set GOOGLE_APPLICATION_CREDENTIALS value. A runtime error is thrown if it's not set. For GCP_ENVIRONMENT, the environment check is ignored as the Google client library will take care of validation. * Update Google Cloud documentation for new auth Add in details on the new authentication types and behavior, replacing the old settings. Tweak some verbiage.
conker84
pushed a commit
that referenced
this pull request
Apr 12, 2021
…e Cloud (#1720) * Fix broken GCS support that's throwing NPEs The call to the `getHost()` method on the `URI` instance returns null for a GCS uri like `gs://my_bucket/my_object`. Checking the URI docs from Oracle (https://docs.oracle.com/en/java/javase/11/docs/api/java.base/java/net/URI.html) we should be using `getAuthority()`. This commit changes the method call resolving the NPE. I've manually tested with a test case hitting a publicly accessible object in Google Cloud Storage (since no unit tests exist). * fix typo in Google Cloud doc url * Support GCP environmental service account auth Previously, we forced using service account private keys set using the GOOGLE_APPLICATION_CREDENTIALS environment variable pointing to a generated json private key. This isn't required in native GCP environments as the official Google Cloud clients will identify if they're running in a GCP service (e.g. GAE, CloudRun, GCE, etc.) and provide automatic authentication as a service account. This change adds a new "authenticationType" and renames the old type: - SERVICE becomes PRIVATE_KEY to explicitly describe that it's expecting to use a private key file. - GCP_ENVIRONMENT is added to be used when running in GCP. The logic is that for PRIVATE_KEY usage we still check the environment for a set GOOGLE_APPLICATION_CREDENTIALS value. A runtime error is thrown if it's not set. For GCP_ENVIRONMENT, the environment check is ignored as the Google client library will take care of validation. * Update Google Cloud documentation for new auth Add in details on the new authentication types and behavior, replacing the old settings. Tweak some verbiage.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR adds the ability to leverage passive authentication if Neo4j is running in Google Cloud, like in the case of running in GCE on a VM host or as a container instance. This simplifies and improves APOC's Google Cloud Storage support by:
getHost()and notgetAuthority()on theURIinstance)authenticationTypeoptions (NONE,PRIVATE_KEY,GCP_ENVIRONMENT)PRIVATE_KEYreplaces the previous behavior used bySERVICE, i.e. using theGOOGLE_APPLICATION_CREDENTIALSapproach of pointing the client libraries at the private key json on the file systemGCP_ENVIRONMENTlets the client libraries naturally identify the service account and fetch a token for authenticating with Cloud StorageThis PR supersedes #1718 (of which I will close).
Note: Ultimately this should be cherry-picked on the 4.2 branch when that's ready.