pkl Writer

[sebholstein]

I would like to be able to evaluate existing pkl files and also write changes.

For my use case, I want to provide an UI for a pkl-based configuration. I then want write back the changes the user has made to the original pkl file.

[holzensp]

When you consume a .pkl file through language bindings, it gets fully evaluated and the bindings copy the fully expanded data over to the host language. You can't preserve lazy relationships and/or for/when generators this way. This is why we've never built such a Writer; it seems against the point.

If you want UI-manipulated static data to be validated by Pkl, I'd dump it into JSON and let your Pkl module use pkl:json's Parser to consume/check it in Pkl.

[sebholstein]

When you consume a .pkl file through language bindings, it gets fully evaluated and the bindings copy the fully expanded data over to the host language. You can't preserve lazy relationships and/or for/when generators this way. This is why we've never built such a Writer; it seems against the point.

Ok, makes sense for me for the current model. But don't you think it would be valuable to use Pkl from both sides? E.g. to have a package that is capable of parsing Pkl to an AST and writing it back after a manipulation.

If you want UI-manipulated static data to be validated by Pkl, I'd dump it into JSON and let your Pkl module use pkl:json's Parser to consume/check it in Pkl.

That's the main point why I opened the issue: I could totally move forward this way, but the user would loose all the awesome features of Pkl when the final format is always JSON.

[holzensp]

capable of parsing Pkl to an AST and writing it back after a manipulation.

In very many discussions, this is a source of cognitive dissonance; Pkl isn't a static data format. Yes, you could parse it into an AST, but it would be deeply non-trivial for your GUI to decide how to mutate said AST to change a setting. You'd have to resolve all names and determine whether you're at the end of an amends chain before you can write things out.

That's the main point why I opened the issue: I could totally move forward this way, but the user would loose all the awesome features of Pkl when the final format is always JSON.

If you're giving people a GUI, they don't see the code at all, so there hardly is any Pkl features GUI users will ever see. If you want a strong validation stage, because your GUI could generate settings that don't work (mutually incompatible property settings, for example), then you can still have that using pkl:json. I'm not sure what the strict win is from having the GUI write Pkl, which humans are not supposed to touch.

[alecthomas]

While I completely understand what you're saying, I think the reason for the cognitive dissonance is that one of, if not the main, use-cases for language bindings is to programmatically load configuration directly into an application. That is, I just want to allow users of my application to write .pkl, rather than using pkl to generate another format, a use case that pkl does advertise quite prominently:

Integrated application configuration

Embed Pkl into your applications for runtime configuration, and receive code generation for Java, Kotlin, Swift, and Go.

In that scenario, being able to write back to the configuration is very common - basically every other static configuration format supports this, preserving comments, etc. If we assume that this is a requirement of my application, then I can't really use .pkl as the native format, I have to use JSON, YAML, TOML or something similar. In that case all that pkl brings to the table is validation - I can't write .pkl and I can't take advantage of IDE integration IIUC? At that point I can get a similar experience from something like JSON-schema (albeit much more verbosely).

[alecthomas]

All that is not to say that pkl-go should support writing BTW, just to explain why I think there's confusion.

[holzensp]

one of, if not the main, use-cases for language bindings is to programmatically load configuration directly into an application

Entirely agreed.

In that scenario, being able to write back to the configuration is very common

True, but that's when the configuration on disk is just a serialised form of application-determined (simple - i.e. numbers, booleans, strings) values.

basically every other static configuration format supports this, preserving comments, etc.

So, here's the thing to solve. If your Pkl file says

myValue = myListing.fold(0, (acc, it) -> acc + it)

and your application determines that myValue = 5; what are you writing back? myValue = 5 or do you know the underlying reason why it's now 5 (and what that means in terms of myListing)? This is a problem you don't have in static formats, because they don't allow you to express these computations.

This gets worse when you have amends chains; if you have the following files;

someTemplate.pkl:

someValue = 10

someAmender.pkl:

amends "someTemplate.pkl"

someValue = super.someValue + 1

someFurtherAmender.pkl:

amends "someAmender.pkl"

Now suppose your application loads someFurtherAmender.pkl. If your applications now changes someValue to be 42, which file should you write what to?