Add service account key file for data provider (#3)

* add service account key file for data provider

* update tfdocs

README.md

@@ -63,6 +63,9 @@ Note: Terraform cloud state management must be setup before it is referenced as
 
 These steps cover how to control the infrastructure found within this repository.
 
+| <span style="text-align:left;float:left;font-weight:normal;">:exclamation:  Please note: after applying the Terraform code with the steps below, a `service-account.json` file is added to your local `/utilities/data-provider` directory which contains sensitive data which may enable access to your cloud resources. __This file should not be checked into source control!__</span>   |
+|-----------------------------------------|
+
 1. Make adjustments to the content as necessary (for example, this readme file).
 1. Fill in [terraform.tfvars](terraform/operations/terraform.tfvars) with values that make sense for your initiative (note: these are by default filled in from cookiecutter values).
 1. Terraform __init__: to ensure Terraform is initialized use command `terraform -chdir=terraform/operations init`.

terraform/operations/README.md

@@ -14,6 +14,7 @@
 | Name | Version |
 |------|---------|
 | <a name="provider_google"></a> [google](#provider\_google) | 4.83.0 |
+| <a name="provider_local"></a> [local](#provider\_local) | 2.4.1 |
 
 ## Modules
 
@@ -24,9 +25,11 @@ No modules.
 | Name | Type |
 |------|------|
 | [google_service_account.service_account](https://registry.terraform.io/providers/hashicorp/google/latest/docs/resources/service_account) | resource |
+| [google_service_account_key.key](https://registry.terraform.io/providers/hashicorp/google/latest/docs/resources/service_account_key) | resource |
 | [google_storage_bucket.target_bucket](https://registry.terraform.io/providers/hashicorp/google/latest/docs/resources/storage_bucket) | resource |
 | [google_storage_bucket_iam_binding.bucket_get_binding](https://registry.terraform.io/providers/hashicorp/google/latest/docs/resources/storage_bucket_iam_binding) | resource |
 | [google_storage_bucket_iam_member.member](https://registry.terraform.io/providers/hashicorp/google/latest/docs/resources/storage_bucket_iam_member) | resource |
+| [local_file.service_account_key](https://registry.terraform.io/providers/hashicorp/local/latest/docs/resources/file) | resource |
 
 ## Inputs
 

terraform/operations/accounts.tf

@@ -3,3 +3,9 @@
 resource "google_service_account" "service_account" {
   account_id = "waylab-em-tokyo-svc-account"
 }
+
+#Create a service-account key for the associated service account
+resource "google_service_account_key" "key" {
+  service_account_id = google_service_account.service_account.name
+  public_key_type    = "TYPE_X509_PEM_FILE"
+}

terraform/operations/local.tf

@@ -0,0 +1,5 @@
+# tf local output
+resource "local_file" "service_account_key" {
+  filename = "../utilities/data-provider/service-account.json"
+  content  = base64decode(google_service_account_key.key.private_key)
+}

utilities/data-provider/README.md

@@ -13,6 +13,7 @@ Please see the following instructions on uploading data to the Google Cloud buck
 1. [Configure rclone with Google Cloud Storage](https://rclone.org/googlecloudstorage/).
 
    - Note: if using a Google account, make sure to authenticate using this account when prompted within the web browser (the terminal will prompt through the browser)
+   - If unable to use the browser with a Google account, consider using the `service-account.json` file and leveraging the `Service Account Credentials JSON file path` during `rclone` configuration.
 
 1. Test access to Google Cloud Storage bucket with the configured rclone access using, for example, `rclone ls <configured_name>:waylab-em-morphology-tokyo-bucket`
 
@@ -34,17 +35,19 @@ Please see the following instructions on uploading data to the Google Cloud buck
 
 __Note:__ The gsutil command provided within the script makes use of the `-m` option for multi-threading performance increases and the `-r` option for recursive data transfer.
 
+1. Ensure `service-account.json` key is found within the same directory where script is run.
 1. Prepare data to be uploaded under `./data` directory relative to `gsutil_sync.bat` location.
-1. Run the `gsutil_sync.bat` script by double clicking it or from a command line prompt (for example, by typing: `gsutil_sync.bat` and hitting the enter key). Note: you may be asked to login using your associated Google credentials.
+1. Run the `gsutil_sync.bat` script by double clicking it or from a command line prompt (for example, by typing: `gsutil_sync.bat` and hitting the enter key).
 
 Please reference the following directory tree structure for an example of what the path should contain:
 
 ```shell
 .
 ├── README.md
 ├── data
-│   └── <data to be synchronized>
-└── gsutil_sync.bat
+│   └── <data to be synchronized>
+├── gsutil_sync.bat
+└── service-account.json
 ```
 
 ### Additional Notes (gsutil)

utilities/data-provider/gsutil_sync.bat

@@ -9,10 +9,11 @@
 ::  see gsutil docs for more information:
 ::  https://cloud.google.com/storage/docs/gsutil_install
 
-::  authenticate gcloud using a related account
+::  authenticate gcloud for the service account
+::  note: this is the preferred method for authenticating gsutil
 ::  see the following for more details:
-::  https://cloud.google.com/sdk/gcloud/reference/auth
-call gcloud auth login
+::  https://cloud.google.com/storage/docs/gsutil/commands/config#configuring-service-account-credentials
+call gcloud auth activate-service-account --key-file=service-account.json
 
 ::  synchronize data from local directory `./data`
 ::  to bucket waylab-em-morphology-tokyo-bucket