Ability to not store secrets/store encrypted secrets in state

[ramilmsh]

Hello!

• Vote on this issue by adding a 👍 reaction
• If you want to implement this feature, comment to let us know (we'll work with you on design, scheduling, etc.)

Issue details

I have a hard time coming up with a solution to a particular scenario - using auto-generated passwords in a secure manner. Here is an example of the problem.

Say, you have a PostgresSQL database and a Kubernetes cluster with a PgBouncer instance deployed in it. PgBouncer requires credentials for a database user with certain permissions. Now, you can create a random.RandomPassword resource and then create a PostgreSQL user and a Kubernetes secret for PgBouncer from it.

But this is were I stumble, random.RandomPassword stores the generated password in the state in plain-text (as far as I understand). That means that anyone who has access to the state, has access to the credentials, which is not acceptable security-wise. If I were to self-host the backend - if anyone could gain access to the state they would have access to all the sensitive fields (and even if I encrypted the storage i would still have to create a separate backend for each stack, to manage access), and if were to use Pulumi Cloud - then Pulumi has access to my company's secrets.

Whichever way you slice it, I have but one real option - let Pulumi fail, when attempting to create the deployment, create the secret manually, run Pulumi again. This is fine, but seems kinda clunky.

Which leads me to my question - is there a canonical way to solve this?

If not, I would like to propose:

Either

Creating a RandomPassword provider, which relies on secretsprovider (if that's even possible), such that the secrets are at least encrypted.

Or

What in my eyes would be the ideal solution

1. not store secrets at all, only their hash
2. during plan step assume that the secret did not change, proceed as normal and check all the dependents of the secret
3. if any of the dependents of the secret require modification, discard plan for the secret and all it's children, regenerate the secret and start over from that node

But I realize, this strategy might fly directly in the face of the core design of Pulumi

Here is a similar issue on terraform:

Affected area/feature

Pulumi, Terraform Bridge

[justinvp]

Thanks for the suggestion

[Frassle]

stores the generated password in the state in plain-text (as far as I understand)

This isn't correct. We do not store the plain text in the state file. Pulumi has first class support for encryption in state files and values that are marked secret are stored on disk (or cloud) as encrypted bytes.

Given that I don't think there's anything to do here?

[justinvp]

random.RandomPassword stores the generated password in the state in plain-text (as far as I understand

Ah, I was thinking RandomPassword may be storing the generated password as its ID in plaintext in the Pulumi state, but I confirmed that's not the case. Maybe this was a problem in the past or I'm misremembering.

This program:

package main

import (
	"github.com/pulumi/pulumi-random/sdk/v4/go/random"
	"github.com/pulumi/pulumi/sdk/v3/go/pulumi"
)

func main() {
	pulumi.Run(func(ctx *pulumi.Context) error {
		_, err := random.NewRandomPassword(ctx, "mypass", &random.RandomPasswordArgs{Length: pulumi.Int(10)})
		if err != nil {
			return err
		}

		return nil
	})
}

Results in the following Pulumi state for the RandomPassword:

            {
                "urn": "urn:pulumi:dev::randpasstest::random:index/randomPassword:RandomPassword::mypass",
                "custom": true,
                "id": "none",
                "type": "random:index/randomPassword:RandomPassword",
                "inputs": {
                    "length": 10
                },
                "outputs": {
                    "__meta": "{\"schema_version\":\"3\"}",
                    "bcryptHash": {
                        "4dabf18193072939515e22adb298388d": "1b47061264138c4ac30d75fd1eb44270",
                        "ciphertext": "AAABAG6xrQOEoSW40gRLdDS6kk63pkMM4Gtuez2LHCZvfpCQmGoufrQLS8ORn9+cG/37eFR9DlKsscMropoR8nAXGcIC//fxXxtCglJvtQL+chVaTgyNabB+CS4Rrg=="
                    },
                    "id": "none",
                    "length": 10,
                    "lower": true,
                    "minLower": 0,
                    "minNumeric": 0,
                    "minSpecial": 0,
                    "minUpper": 0,
                    "number": true,
                    "numeric": true,
                    "result": {
                        "4dabf18193072939515e22adb298388d": "1b47061264138c4ac30d75fd1eb44270",
                        "ciphertext": "AAABAHLKDA+Pg05Zl70jZp6WLqD+tjnNyDXQB60DHgEcJW0BH4eRk+ntE/k="
                    },
                    "special": true,
                    "upper": true
                },
                "parent": "urn:pulumi:dev::randpasstest::pulumi:pulumi:Stack::randpasstest-dev",
                "provider": "urn:pulumi:dev::randpasstest::pulumi:providers:random::default::ec926f8f-5688-47a1-bca5-a3c17493f5a0",
                "propertyDependencies": {
                    "length": null
                },
                "additionalSecretOutputs": [
                    "bcryptHash",
                    "result"
                ],
                "created": "2023-10-30T17:53:46.623529Z",
                "modified": "2023-10-30T17:53:46.623529Z"
            }

Using RandomPassword.Result as an input to other resources will flow the secretness through to those other resources.

---

I'm going to close this as I don't think there's anything to do here. @ramilmsh, please re-open with more details if we've misunderstood or you feel otherwise.

[sam-utila]

I totally relate to the problem.

Whether the secrets are encrypted in the backend or not - I don't want some bucket / pulumi SaaS or any subsequent Pulumi run to have access to the secrets that were generated on the first run.

Since this is how Pulumi (and TF) works - there is no easy way around it.

In our case we are using GCP Cloud SQL (Postgres), and we did the following

1. Create the sql DB in Pulumi _, err := sqlclassic.NewDatabaseInstance(... without a password
2. Use a local.NewCommand to run an out of process executable

_, err = local.NewCommand(
		ctx,
		"sql-db-post-create-ident-init-tool",
		&local.CommandArgs{Create: pulumi.Sprintf("go run ../initdbident -project %s -region %s -instance %s -migrator %s", ...)...

This is an executable that uses GCP API to set a new random root password, throws it away, but before that, uses cloud sql proxy to run a migration that gives a GCP SA root access to the DB
3. Continue in Pulumi

So in order for Pulumi to not have the throwaway root password in its state forever, we use this external process that does not have any secrets in the inputs/outputs

--

On the same note I would add that it would be extremely helpful to be able to add an option to a stack to block any secrets (added by mistake)