Getting malformed json key using terraform 1.0.0

[johnsmartco]

The bug that was reported and closed in 0.13 (#25986) seems to be a problem in 1.0.0. After successfully entering terraform init, terraform plan, and terraform apply, I'm seeing a malformed private key in a json file with \n and \n inserted. My commands to generate the secret:

terraform output -json service_account_key > medusa_gcp_key.json
kubectl create secret generic prod-k8ssandra-medusa-key --from-file=medusa_gcp_key.json=./medusa_gcp_key.json
secret/prod-k8ssandra-medusa-key created

After a subsequent install of K8ssandra in the terraform-created GKE cluster, most of the containers/pods start, except for the medusa container, which complains about the malformed key.

Terraform Version

Terraform v1.0.0
on linux_amd64
+ provider registry.terraform.io/hashicorp/google v3.73.0
+ provider registry.terraform.io/hashicorp/google-beta v3.73.0

Your version of Terraform is out of date! The latest version
is 1.0.1. You can update by downloading from https://www.terraform.io/downloads.html

Terraform Configuration Files

Refer to:

https://github.com/k8ssandra/k8ssandra-terraform

Debug Output

Crash Output

Expected Behavior

Private key does not contain \n ...

Actual Behavior

Private key has \n, resulting in malformed private key.

Here's an example with sensitive info changed - notice the \n and \n inserted in the generated private key:

cat medusa_gcp_key.json 
"{\n  \"type\": \"service_account\",\n  \"project_id\": \"gcp-myproject\",\n  \"private_key_id\": \"fakevaluesinsertedhere\",\n  \"private_key\": \"-----BEGIN PRIVATE KEY-----\\fakevaluesinsertedhere/P7AC\\nkLxqjN8RGrg6Q5iDVzD/PDlZMvyGXin73LIibnJnpexueKDzVbyEC8OGfj8PxlkY\\nfakevaluesinsertedhere/S7S\\nzIRuPjUT2ISaTxCTV1xRNgXzOLaiur0/O9wGaytzG6IpPV4dsS1IcmuEQU3Cjmuo\\nfakevaluesinsertedhereKDF8D7hMo41GoO8\\nfL3umdW4BYoGvdhus6WYAFh6wHRBQQLZIK8C9UYfc6O8YNB09AuFDjoyY/UJv5nm\\njInoxT+XAgMBAAECggEAafRQITrhB7o08PIVgJEpOqEtf4XNWgl+fakevaluesinsertedhere\\n0+OvqEmLGmWa99+fakevaluesinsertedhere+t+w1Nlj+\\ne2u6eEe1OKz5o81qXU5OBWxXSnV9CJByUnGYuDStxXYZgaVBQb1MsPvPtq9U8m5V\\nm1tCJ4o/t0y9Uchv21L6R2s/q6N4AG6nLXuKbyKIsEa9inD2WdpyjfHepx96FXJd\\ndA2xLj52+fakevaluesinsertedhere\\ndDEZ1msYdizfTMmcaeBqUdabwsbWBTOHwf6q0/khYQKBgQD3gnDvJmdIBNLZOyoL\\n7Fy6B6cuLObTb+fakevaluesinsertedhere+fakevaluesinsertedhere+SLn+2e\\n7rJKLswjspzs+2h6vt2Q/kHvEgiFx3UwxP1BInf9LstiY4KPjVHQW5UYRDrk9zsA\\n1tgAQLzM7y9iaFjUkw0JYwksSQKBgQDcFM1UXsPFzConoCo+6Z+21sEsoBRVnGdO\\nq3K5TmPOK61y4NvlovbYv44v7r6T7yf1B0Fjh08GpBzHcnLtb0my7beaUhLDec4h\\nZPNf8EKp9DCrkF8nThX7VNFIGPELFLC9ir+mqbjE1r1irWslzVvZq+P/gYihCGeA\\nLA1bYZJM3wKBgEoXD/tmwY/7cap5XXLIRFGjrNXTtx0f6XjawWbT4kKEA1LKmgZs\\nUeMTXA2Tds6cQbff6L+8LcuLL8TkENt8lH5EV/Nvqi3+bB4iOG0Iz2/jNA3n0RrS\\nPQVcbhKqCVPgedrC8PSwFYd4FTEpGM59gZdBycKG7uZEWiL4CfT/YmWRAoGAR3pG\\nGcHRaZtekZIWRmqQIczr3nd7QzbR4p4SW5bXGW3cHnRjVtivvUOxhWXP5bIF0zHx\\nDcczZl/ErQ5Bv1WmpQNJ30gKqgwUY4oq9RzOOe6CJCZ2kQhuYTtx4S2gk827fUPj\\n7Ngwl1V7GuJh61wgGbkXCrPiURKustUb3XL6FhcCgYAUlLWUNRLWdo4a1pxSxp2g\\nY1aQb9Nu23NEtK6B/eX9X37OLspaztjHuXWQrrxTMCtnu5eIwQFRadc41efCSV8B\\n82D1l59eSkdMJtYgGYybu1qL17MdsPu8ASX92bTjJPjTz7TjfE/uaj8ZEQQk2D8b\\nVRPqDLnnY92zOvPu2FE6RQ==\\n-----END PRIVATE KEY-----\\n\",\n  \"client_email\": \"prod-fakevaluesinsertedhere@gcp-v.iam.gserviceaccount.com\",\n  \"client_id\": \"1fakevaluesinsertedhere138399914224\",\n  \"auth_uri\": \"https://accounts.google.com/o/oauth2/auth\",\n  \"token_uri\": \"https://oauth2.googleapis.com/token\",\n  \"auth_provider_x509_cert_url\": \"https://www.googleapis.com/oauth2/v1/certs\",\n  \"client_x509_cert_url\": \"https://www.googleapis.com/robot/v1/metadata/x509/prod-fakevaluesinsertedhere-sa%40gcp-techpubs.iam.gserviceaccount.com\"\n}\n"

Steps to Reproduce

See https://docs.k8ssandra.io/install/gke/, but with one diff: testing with terraform 1.0.0, not the 0.14 version cited.

Additional Context

Can you fix and release in a 1.0.next patch?

References

Similar prior bug reported by another user in 0.13 seems to be an issue again in terraform 1.0.0.

[apparentlymart]

Hi @johnsmartco! Thanks for reporting this.

From reading the modules in the repository you listed, I see that service_account_key ultimately derives from google_service_account_key.service_account_key.private_key in the iam module. The documentation for private_key says that this attribute contains "The private key in JSON format, base64 encoded". The iam module uses base64decode to remove the base64 encoding before returning, and so the expected value for service_account_key is a string containing a JSON representation of an object.

You then asked terraform output to print this output value in JSON format itself, and so Terraform produced a JSON representation of that string which itself contains JSON, as you saw. Terraform seems to be behaving as expected here: it produced a correctly-JSON-encoded representation of the string you specified.

If you want to get just the raw JSON data that is in the service_account_key string then you could use terraform output in -raw mode instead of in -json mode, and then Terraform will just print out that string exactly as the module returned it, without any additional JSON encoding:

terraform output -raw service_account_key

From Terraform's perspective then it will just print out the string saved in that output value.

Since Terraform seems to be behaving as intended here, I'm going to close this issue. If you have any follow-up questions about what I've suggested here, please feel free to start a topic in our community forum. Thanks again!

[github-actions]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.