Hoist functions from their templates to resolve authoring awkwardness

[pivotaljohn]

⚠️ under construction ⚠️

(scratchpad notes: https://hackmd.io/coYgaB_MQQ2EvSHxbXuLOw)

Today, it is easy to write a function definitions in a YAML template that results in surprising behavior (see scenarios, below).

There is an intuitive understanding that functions are not really part of the template at the location in which they are defined, but that they potentially "capture" some

In order to improve the template authoring experience, let's make good on that intuitive understanding and hoist function definitions out of the document in which they are defined.

Of course, this should be done in a way that preserves the semantics of the template.

Problems:

• "disowned parent node" — a function wrapping map items will leave behind the containing map; function hoisting should include the parent node if the extraction empties that parent.
• "guilt by association" — on a node that carries the function definition start (i.e. def ...()), if an annotation appears within that function definition, it must also be hoisted; if an annotation appears above the function definition, it should not be hoisted.
• ...

Scenario: default value for an array in schema

When writing schema and specifying a default value (especially for an array of items), it's natural to do so using YAML Fragment functions.

Today, it's easy for this to be a BadTime™️: what seems like a straight-forward move results in what is likely a confusing error message.

🟢 Capturing a collection of map items in a YAML Fragment function works:

#@data/values-schema
---
#@ def foos():
bar: 42
#@ end

#@schema/default foos()
foo:
  bar: 0

🔴 But doing the same for array items does not:

#@data/values-schema
---
#@ def foos():
- b
#@ end

#@schema/default foos()
foo: [""]

... and instead results:

ytt: Error: Unmarshaling YAML template 'array.yml': yaml: line 7: did not find expected '-' indicator

... (which is a sharp edge in ytt templates, in general) when 'fixed" (by injecting a document start immediately after the function declaration):

#@data/values-schema
---
#@ def foos():
- b
#@ end

#@data/values-schema
---
#@schema/default foos()
foo: [""]

🔴 ... which results in:

$ ytt -f array.yml
ytt: Error:
Invalid schema - wrong number of items in array definition
==========================================================

array.yml:
    |
  2 | ---
    |

    = found: 0 array items
    = expected: exactly 1 array item, of the desired type
    = hint: in schema, the one item of the array implies the type of its elements.
    = hint: in schema, the default value for an array is always an empty list.
    = hint: default values can be overridden via a data values overlay.

🟢 ... only when finally the function is pulled entirely out of the schema-flavored document:

#@ def foos():
- b
#@ end

#@data/values-schema
---
#@schema/default foos()
foo: [""]

... do we get the desired result.

$ ytt -f schema.yml --data-values-inspect
foo:
- b

Incidents:
(moments we are aware of this situation occurring)

• #carvel @ 2020-01-25 @ 07:35PT

---

Vote on this request

This is an invitation to the community to vote on issues, to help us prioritize our backlog. Use the "smiley face" up to the right of this comment to vote.

👍 "I would like to see this addressed as soon as possible"
👎 "There are other more important things to focus on right now"

We are also happy to receive and review Pull Requests if you want to help working on this issue.

[cppforlife]

just for the record, this is not a problem related to schemas. it's a general inconvenience of using arrays in functions within map. if we are planning to address this somehow in error messaging etc, i would expect that it would be done generically.

[pivotaljohn]

just for the record, this is not a problem related to schemas. it's a general inconvenience of using arrays in functions within map. if we are planning to address this somehow in error messaging etc, i would expect that it would be done generically.

Yes, sir! 👍🏻 If there's any massaging of the AST after executing a template, will happen within yamltemplate (or maybe even template?).