Structured configuration not supported in Automation API

[ringods]

Expected Behavior

I need a programmatic counterpart of structured configuraiton.

Current Behavior

I can only set a single value (SetConfig) or all values at once SetAllConfig), but structured config doesn't seem to be supported at the moment

Context (Environment)

• https://www.pulumi.com/docs/intro/concepts/config/#structured-configuration

• pulumi/sdk/go/x/auto/local_workspace.go

  Lines 166 to 184 in 3be1ea9

  	// SetConfig sets the specified key-value pair on the provided stack name.
  	// LocalWorkspace writes this value to the matching Pulumi.<stack>.yaml file in Workspace.WorkDir().
  	func (l *LocalWorkspace) SetConfig(ctx context.Context, stackName string, key string, val ConfigValue) error {
  	err := l.SelectStack(ctx, stackName)
  	if err != nil {
  	return errors.Wrapf(err, "could not set config, unable to select stack %s", stackName)
  	}
  	secretArg := "--plaintext"
  	if val.Secret {
  	secretArg = "--secret"
  	}
  	stdout, stderr, errCode, err := l.runPulumiCmdSync(ctx, "config", "set", key, val.Value, secretArg)
  	if err != nil {
  	return newAutoError(errors.Wrap(err, "unable set config"), stdout, stderr, errCode)
  	}
  	return nil
  	}

Support

• Go
• [auto/python] Add support for the path option for config operations #13052
• [auto/nodejs] Add support for the path option for config operations #14305
• [auto] Add support for the path option for config operations pulumi-dotnet#191

[EvanBoyle]

As a workaround, is it possible to read the entire top-level config entry, change the appropriate property, and then write it back?

[ringods]

@EvanBoyle I don't understand what you hint here.

There is no config yet in my case: I developped my stack with structured config and now using the automation api rolling out customer specific versions of that stack. This means I have a Git workspace, upserting the stack after configuring it.

[EvanBoyle]

So you want to set a leaf value:

SetConfig("a.b.c.d", "foo");

This is equivalent to:

val = {
b: {
c: {
d: "foo"
}
}
}

SetConfig("a", toJSON(val))

[alexeyzimarev]

If the config object has one of the leafs as a secret, I would assume it won't work.

[alexeyzimarev]

Oh, I see, the whole object is set as secret if one value is a secret value.

[tmccombs]

So you want to set a leaf value:

SetConfig("a.b.c.d", "foo");

This is equivalent to:

val = {
b: {
c: {
d: "foo"
}
}
}

SetConfig("a", toJSON(val))

It doesn't look like that works. With the equivalent (I think) code:

await stack.setConfig("a", {
      value: JSON.stringify({
        b: {
          c: {
            d: "foo",
          },
        },
      }),
    });

I get config that looks like:

  my-stack:a: '{"b":{"c":{"d":"foo"}}}'

[JaveriaK]

Just want to chime in that updating leaf values through setConfig doesn’t work.

code:

	pItem, err := stack.GetConfig(ctx, Params.ListKey)
	if err != nil {
		return MajorError, fmt.Errorf(“[Error]: Parameter %v does not exist in stack”, Params.ListKey)
	}
	var arr []map[string]interface{}
	_ = json.Unmarshal([]byte(pItem.Value), &arr)
	for _, elements := range arr {
		if elements[“apiVersion”] != Params.Value {
			elements[“apiVersion”] = Params.Value
		}
	}
	outList, _ := json.Marshal(&arr)
	err = stack.SetConfig(ctx, Params.ListKey, auto.ConfigValue{
		Value:  string(outList),
		Secret: false,
	})
	if err != nil {
		return MajorError, fmt.Errorf(“[Error]: Error Updating Stack %v”, err)
	}

generated change in stack file:

-  kubernetes:platform_pools:
-  - apiVersion: 1.19.11
-    name: platform1
-    resourceName: platform-pool
-    vmSize: Standard_D16s_v3
+  kubernetes:platform_pools: ‘[{“apiVersion”:“1.21.4",“name”:“platform1",“resourceName”:“platform-pool”,“vmSize”:“Standard_D16s_v3"}]’

Neither does trying to get leaf elements work: stack.GetConfig(ctx, "kubernetes:platform_pools[0].apiVersion")

[EvanBoyle]

@JaveriaK unfortunately, mixing structure config with automation api is not supported at the moment. The workaround allows you to achieve the same effect as structured config, but you won't be able to get and set leaf paths from the CLI using the --path command.

[tmccombs]

The workaround allows you to achieve the same effect as structured config

Only for custom configuration. It doesn't let you modify existing structured configuration, such as the aws:assumeRole config.

[radirobi]

Dear @JaveriaK @EvanBoyle @tmccombs @ringods @komalali and everybody

could you tell me how to achieve the following structured config by using Pulumi GO sdk:

config:
  myNamepace:myStorage:
    name: myStorageName
    size: 50

I have tried the following:

jsonString := `{"name": "test", "size": 100}`
err = stack.SetConfig(ctx, "myNamepace:myStorage", auto.ConfigValue{
        Value:  jsonString,
	Secret: false,
})

I got the following result which is not I want:

config:
  myNamepace:myStorage: '{"name": "test", "size": 100}'

[shreecodes]

As a workaround to this issue, we're thinking about writing our own helper function that invokes the CLI with pulumi config set --path ... --non-interactive with an optional --secret. The automation SDK is a wrapper around the CLI, after all.

This approach might be a little fragile if/when the CLI changes in the future... but are there any other reasons to avoid this approach?

[Aylr]

Is there any update on if/when the Automation API might support structured configuration? Has anyone else here found a decent workaround?

[mDemianchuk]

+1, any update on the --path support?

[coopbri]

Is there any update on if/when the Automation API might support structured configuration? Has anyone else here found a decent workaround?

Hey @Aylr, I'm not sure if you've tried this already but processing structured config as a JSON string works for me.

E.g. for an array/list:

await stack.setConfig("test:array", {
  value: JSON.stringify(["value1", "value2"]),
});

console.log(await stack.getConfig("test:array"));
// output: { value: '["value1","value2"]', secret: false }, works properly in program

[farvour]

Kind of disappointed here that the this method cannot straight up take any type as an input or use the CLI to run the same logic to set the configuration....

One such configuration that doesn't seem to behave is:

pulumi:disable-default-providers:
  - '*'

I ended up creating a helper python method that just invokes the run method. It's syntactically labeled as a private method so hopefully it doesn't break in the future. I'm also not even sure if it would work correctly in real type-safe languages like TS or C#. Maybe it might be useful to abstract another method in the LocalWorkspace class or just allow this to be public called?

def disable_default_providers(stack: auto.Stack) -> None:
    """
    Runs the pulumi cmd to disable all default providers.
    """
    stack._run_pulumi_cmd_sync(
        args=[
            "config",
            "set",
            "--path",
            "pulumi:disable-default-providers[0]",
            "*",
        ]
    )

[justinvp]

Added Node.js and .NET support in #14305 and pulumi/pulumi-dotnet#191

[EronWright]

A quick comment on a remaining gap: stack.GetAllConfig() doesn't fully support structured config because it doesn't accept a path or return object values. A possible workaround is to use stack.StackSettings()?

[sfc-gh-jlangefeld]

+1 to @EronWright's question. It is a bit confusing that the stack object can't set objects as config, but a *workspace.ProjectStack object retrieved via stack.Workspace().StackSettings(ctx, stack.Name()) can via .Config.Set(key, value, true).