Skip to content
This repository has been archived by the owner on Dec 7, 2023. It is now read-only.

Add proper templating support to step definitions #23

Closed
JeanMertz opened this issue Jun 22, 2019 · 1 comment
Closed

Add proper templating support to step definitions #23

JeanMertz opened this issue Jun 22, 2019 · 1 comment
Milestone

Comments

@JeanMertz
Copy link
Contributor

JeanMertz commented Jun 22, 2019

Thinking a bit more on #22, I think if we are going to add more complexity to the current self-rolled templating support, we might as well switch to an existing/well-known templating language, so that we can trim down on the added complexity of our own system, and expand the templating capabilities beyond what was proposed in #22.

It seems to make most sense to switch to Handlebars for templating:

  • it's well-known
  • easy to learn
  • good Rust support
  • extensible

Given that last point, we could solve the issues in #22 by adding our own helpers, such that you could access variables like so (to be bikeshedded):

  • {{ var "my pipeline variable" }}
  • {{ global "my global variable" }}
  • {{ local "my temporary variable" }}

I'm still on the fence on that last one, as we might as well combine 1 and 3 into a single var helper. In #22 I mentioned that temporary step output variables could shadow pipeline variables, which is true, but given that the pipeline author is in control of both what step output and pipeline variables exist in a pipeline run, it seems less of an issue that shadowing is allowed, and is easily prevented by not naming your step output variables and temporary variables the same.

I also think we should rename the $input variable to last step output, so you can use the implicitly created {{ var "last step output" }}. We'll just have to make that a reserved variable name, so that pipelines can't use that name as a pipeline variable. Then, the issue raised at the end of #22 is also moot, as we always implicitly assign the last step output value to this variable, even if you also explicitly assigned the output to a different variable.

JeanMertz added a commit that referenced this issue Jul 11, 2019
Before, when a job was created from a task, the variable values would
get injected into the processor configuration before the configuration
was saved to the database in a JSON blob.

With this change, variable values of a job are now stored in a separate
table, and the values are encrypted using Postgres' `pgcrypto` module.

The variable values are decrypted at runtime, right before they are
injected into the processor configuration, which from that point on only
lives in memory.

WARNING: The server uses a default encryption secret "default secret" to
make it easier to run things locally. This will change before a 1.0
release, and you should always set the secret using the `SERVER_SECRET`
environment variable when running Automaat in production.

This closes #30.

This also has the added benefit that variable substitution is moved to a
single place in the code base, making it easier to implement
#23 in the future.

Furthermore, the implemented encryption strategy will make it easier to
implement #24.
@JeanMertz JeanMertz added this to the v1.0.0 milestone Jul 11, 2019
@JeanMertz
Copy link
Contributor Author

JeanMertz commented Jul 12, 2019

This has been implemented in 858d44d.

Not exactly as the original issue describes, but closely related.

The major differences are:

  • Using Tera instead of Handlebars for templating support (mostly because filter chaining isn't great in Handlebars)
  • Variables are accessed using {{ var["My Variable"] }} instead of the originally proposed {{ var "My Variable" }}
  • The old $input variable is accessed using {{ sys["previous step output"] }} instead of the proposed {{ var "last step output" }}.
  • There is no local object, although there will be a global object, once Add concept of "global variables" #24 lands.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant