Expand templates during compilation

[joeduffy]

At the moment, we don't expand templates during compilation. We need to.

The intent is to use Go text templates, because it has all of the expressiveness we want out of a templating language. (Conditionals, custom functions, etc.) An alternative is to use something like Jsonnet, however (a) it is probably less familiar to the population at large, and (b) we want something that is file format agnostic.

As part of this, we need to decide what goes into the "context" object available to templates.

[joeduffy]

One thing that makes this more complicated is deciding precisely when to expand templates.

If we do it after parsing happens, then the YAML/JSON parsers will encounter syntax errors.

If we do it before parsing, on the other hand, we don't yet have type information about the stack's properties; as such, the context object's parameters will be weakly typed during the template's execution (a map[string]interface{} or somesuch). This seems messy.

Catch-22, kinda sorta. More thinking required ...

[joeduffy]

After noodling on this more, I wonder whether templates are the right solution. The more I dig into the design/implementation, the more C++ deja vu I get, complete with all the problems known to afflict the preprocessor. (Clang's modules doc lays out many of them.) The challenge is that it's less obvious how JSON/YAML could be used in a more "module-like" way without devolving into template substitution, and without us basically reinventing our own templating system.

[joeduffy]

For YAML, it's possible some combination of anchors (&), references (*), and associative array merges (<<) can do part of this. But this would be YAML-specific and obscure and still not as expressive as we may need (e.g., you can't do concatenation, call arbitrary functions, etc).

[joeduffy]

Another thought on this. If we think about the metadata, there are actually two discrete sections:

1. the true metadata about the name, description, properties, and so on;
2. the blueprint for service construction.

As we consider the DSL extensions (#22), we would still consider 1 to be metadata (in a Mu.yaml file), while 2 would be taken from source code.

And this is the key insight. Perhaps templating should only be allowed within the service blueprint section (services) of the Mu.yaml file, just as "code execution" in the DSL sense is limited to the code we run to create the services. If we took that approach, we could do the templating in a sensible way, although it requires incremental parsing/semantic analysis of multiple artifacts.

[joeduffy]

We've decided to move away from templating and instead use a real language approach. Therefore, this work item is obsolete. Closing.