How to apply update's automatically onto my configs

[mbauhardt]

hi community.
I have a question regarding the maintaining of config files. The manual way would be to checkout my config files and update the entries I want to change manually with an editor.
E.g. my config has a version field I update from time to time to a newer version number.

Now let's say I don't want to do it manually anymore, I want to use an automatic way. within an github action for example.
In a json or yaml world I would use jq/yq and apply update in place functionality to update the file itself.

• Is there a way in pkl to achieve the same?
• If not, would this be a missing feature or is this a use case which is not intended in pkl.

[bioball]

Pkl isn't meant to be written to programmatically.

However, if your data is meant to come from an automated source, you can use read() to import that data into Pkl.

For example, if your data is simply a version file:

version = read("path/to/version.txt").text.trim()

Or if your data is JSON, you'd use an read-then-parse technique:

import "pkl:json"
import "package://pkg.pkl-lang.org/pkl-pantry/pkl.experimental.deepToTyped@1.1.0#/deepToTyped.pkl"

local parser = new json.Parser {}

class MyData {
  foo: String
  bar: Int
}

parsed = parser.parse(read("path/to/data.json"))

myData = deepToTyped.apply(MyData, parsed)

[mbauhardt]

I don't want to use or work with json. I want to work with pkl directly.
Let's demonstrate how I work around this with a shell script directly on my pkl files.

A simple account definition, with a mapping between name and id.
Accounts/AccountConfig.pkl

module AccountConfig

accounts: Mapping<String, String>

A config which amends this type.
Accounts/accounts.pkl

amends "AccountConfig.pkl"

accounts {
    ["team-a"] = "1234"
    ["team-b"] = "4321"
}

A script which creates a temporary config which amends the config above,

• by overwriting one account with a new number.
• Using eval on this temporary file and prepend the line to amends AccountConfig.pkl
• and pipe it into the original accounts.pkl config.
• At the end remove the temporary file

update.sh

#!/bin/sh

update_component() {
    to_amend=$(basename $file)
    out_dir=$(dirname $file)
    echo "amends \"$to_amend\"" > $out_dir/${to_amend}-upgrade.pkl
    echo "accounts {" >> $out_dir/${to_amend}-upgrade.pkl
    echo "[\"$account\"] = \"$number\"" >> $out_dir/${to_amend}-upgrade.pkl
    echo "}" >> $out_dir/${to_amend}-upgrade.pkl
    $pkl eval $out_dir/${to_amend}-upgrade.pkl -o $file
    $sed  -i '1i amends \"AccountConfig.pkl\"' $file
    rm $out_dir/${to_amend}-upgrade.pkl
}

main() {
    sed=$(command -v gsed || command -v sed)
    pkl=$(command -v pkl) || exit 1

    file="Accounts/accounts.pkl"
    account="team-a"
    number="0000"

    update_component
}

main "$@"

and call this script with ./update.sh, the result is

amends "AccountConfig.pkl"
accounts {
  ["team-a"] = "0000"
  ["team-b"] = "4321"
}

[StefMa]

I guess the best way to do this is like you do already. Or going dynamic as @bioball suggested.

But let raise the question why you want to do this. Maybe if we understand the need, we can offer a better/alternative solution?!

[mbauhardt]

Why

I want to be able to update my configuration, which holds a set of kubernetes components.

A component has several key:value pairs to describe the properties of a component.
A property, e.g. to define a docker image of an component contains tag:version.
The value of version changes frequently.
Changes are made automatically with github actions or more accurate with github workflows instead manually with an editor.

If I would use yaml to describe components I could use yq to update the version with a bash statement within in a github step.

yq -i ".$component.image.tag = \"${{ inputs.version }}\"" $filename

But I want to use pkl and not yaml to describe my kubernetes components.
As far as I see, the pkl-cli doest not support an out of the box way to update a value within my config.

A valid work-around/solution is to amends an existing configuration and override $component.image.tag and eval this amended config.
And render this result with the PCF renderer and pipe this result back to my original configuration.
The thing is that I will lose all the amends and import statement when using this solution.

What

Provide a command in the pkl-cli to update config values without loosing amend/import or any other expressions I defined.
Using -x (expression) seems to be a good entry point to do it, but loosing any other added statements with that as well.

[bioball]

If you have known values that you'd like to be able to override from the CLI, you should look into using read() for that. For example, your code can look like:

amends "AccountConfig.pkl"

accounts {
  ["team-a"] = read?("prop:team-a") ?? "1234"
  ["team-b"] = read?("prop:team-b") ?? "4321"
}

Then, you can eval via:

pkl eval -p team-a=1000 myModule.pkl

If you don't have a program that allows you to override specific values using read(), -x is a good escape hatch (like you mentioned). For example:

pkl eval -x "(module) { accounts { [\"team-a\"] = \"$number\" } }.output.text" myModule.pkl

This is a little more fragile, though, because you are relying on shell scripting to construct valid Pkl programs. It's also harder to read, IMO.

[bioball]

By the way, you can also glob read, which can be quite useful in some circumstances.

amends "AccountConfig.pkl"

local teamOverrides = read*("prop:team**")

accounts {
  ["team-a"] = "1234"
  ["team-b"] = "4321"
} {
  for (uri, value in teamOverrides) {
    [uri.drop(5)] = value
  }
}

This has the effect of: for every prop name that starts with team, add or override that entry in accounts.
So, this works:

pkl eval -p team-a=1000 -p team-b=2000 myModule.pkl

[mbauhardt]

Wow. This is great. I will use and feel this way in the next days when using this approach.

Thank you.

BTW: quick feedback regarding the read() properties solution.
i tried to use properties as well, but failed because I misused it by using wrong syntax.
I wasn't able to find examples in the user guide or language reference. would be good to have documented this helpful feature.

thx
marko

[bioball]

Did you see this section? If so, definitely let us know what you found confusing about it!

https://pkl-lang.org/main/current/language-reference/index.html#resources