text/template, html/template: add ExecuteFuncs

[zeripath]

This proposal is about providing a way to pass a template.FuncMap at template execution overriding specific functions within the template.

When executing a template using one of new methods, then the provided template.FuncMap will be checked first.

Summary

Reasons:

• Passing values to templates can only be achieved using the data any parameter and this is suboptimal for contextual rendering elements such as web nonces like csrf, or translation functions.
• This is especially problematic when using sub-templates called from within template rendering. The template calling syntax only allows setting a single variable as its data, making passing this contextual rendering functions difficult.
• The current suggestion is to use Clone(), Funcs() and then Execute() but this is slow and optimizing this appears very difficult. (text/template: optimize Clone+Funcs+Execute sequence #38114) It would require significant changes throughout the code and even if parent templates could be changed to allow inheritance, It would requires cloning every template that could be called in advance.

Advantages:

• The proposal is easy to implement (see text/template: add ExecuteFuncMap and ExecuteTemplateFuncMap #54432)
• The additional methods are simple to use and represent a very small API change.
• The change will allow significant simplification of templates in web services.

Detailed rationale

Sometimes data needs to be rendered differently based on the context it is rendered in.

The two most common examples anti-CSRF tokens and locales.

Although currently these can be added to the data struct provided when rendering - these are not strictly data and it significantly complicates rendering especially when considering subtemplating.

Proposed API

I propose that the following methods are added to text/template.Template and to html/template.Template

text/template:

func (t *Template) ExecuteFuncMap(wr io.Writer, data any, funcs FuncMap) error { ... }

func (t *Template) ExecuteTemplateFuncMap(wr io.Writer, name string, data any, funcs FuncMap) error { ... }

html/template:

func (t *Template) ExecuteFuncMap(wr io.Writer, data any, funcs FuncMap) error { ... }

func (t *Template) ExecuteTemplateFuncMap(wr io.Writer, name string, data interface{}, funcs FuncMap) error {

Calling these methods would execute the template as per Execute and ExecuteTemplate with the only difference being that when a template calls a function the provided funcs template.FuncMap would be checked first. In this way funcs can be considered a map of function overrides for template execution and acts similarly to the calling the Funcs(...) method on the template but would also override functions in templates called by the current executing template.

Examples

Current Status

Let us assume we have a function with the following signature:

func (l *Locale) Tr(key string, args ...any) string { ... }

In this example this function takes a key and provides the translated value for it.

Clearly, this cannot be compiled into shared templates as the locale will be different on a per request basis.

So assuming we don't want to have multiple compiled templates per locale we will need to pass this in as data. Let's say as .Tr.

This leads to:

...
{{.Tr "key" args}}
...

But... what about when ranging over a set of items? Now, . will refer to an item in the range of items so we need use $...

{{range .Items}}
   ...
   {{$.Tr "key" .}}
   ...
{{end}}

If we forget we will get a runtime error. So we won't know that it's a problem until it's run.

But... what if we want to reuse that range block as a subtemplate or even just extract it out because it is becoming unwieldy? Ideally we'd want to do:

{{range .Items}}
   {{template "render-item" .}}
{{end}}

That of course doesn't work because we lose access to $. So now we need to do something horrible like:

{{range .Items}}
   {{template "render-item" dict "Item" . "Tr" $.Tr}} {{/* or "root" $ */}}
{{end}}

and then in our subtemplate refer to .Item and .Tr (or .root.Tr). This means that we need to now know that we're in a subtemplate and every time we call the subtemplate we need to do a similar hack.

Again if we forget to add the correct prefix we'll get a runtime error but only at time of render.

Why isn't template.Clone() template.Funcs() template.Execute() sufficient?

The problem here is multifactorial, Clone() isn't very lightweight and requires locks, Funcs(...) also requires locks and every template would need to be cloned and its funcs updated in case when you execute the template it calls another template.

The referenced issue has been around for over 2 years and it is likely that improving the efficiency of this route significantly is not possible without significant refactors.

Instead, the proposed API makes for a clear and simple implementation with a minimal changeset.

Proposed

The proposed API allows for much simpler templating. The Tr function would be passed in within an override FuncMap e.g.:

err := tmpl.ExecuteFuncMap(wr, data, template.FuncMap(map[string]any{"Tr": locale.Tr})

Then the downstream templates look like:

{{Tr "key" args...}}
{{range .Items}}
    {{Tr "key" .}}
    {{template "item-template" .}}
{{end}}

And the Tr function is available everywhere without having to determine whether it needs a ., a $ or even a $.root. (Or even if it's not been passed in to this subtemplate.)

Compatibility

The proposed functionality is completely backwards compatible.

Implementation

The proposal is easy to implement and has been implemented as #54432.

[robpike]

It isn't clear what you are proposing. Does the argument function map replace the complete set available, does it replace only those in the template, or is it just a place to look first?

You don't say what the function signatures are, either. Please edit the proposal to make it clearer.

[zeripath]

I hope that I've made this clearer for you. PR #54432 hopefully shows the intended API in practice.

[zeripath]

A further optimization in #54432 would be to only check and convert the overlay functions in to a valueFunc when the function is called instead of beforehand.

[robpike]

I hope that I've made this clearer for you. PR #54432 hopefully shows the intended API in practice.

Thanks, that is clear now.

The new names are cumbersome (but could be changed), while thought needs to be given to whether this is the right approach.

[zeripath]

I agree that they're slightly cumbersomely named but they are consistent with naming practices elsewhere in std. Another option could be to make them ExecuteWithFuncMap.

---

There is genuinely a need to provide funcmaps at time of execution differently from those used at parsing, and whilst such funcmaps are related - certainly the parsing funcmap constrains what types of funcs and the names of funcs - they're not the same.

This is recognised in the current API by having template.Funcs do two different things before and after parsing. This results in Funcs having to have an explicit documentation warning about this different behaviour(!). Internally they're also stored separately. The problem is this method necessarily changes the template - and there is no way to express a desire only change a single execution of a template.

In a lot of ways the naturality of execution funcmaps can be seen in how simple it is to implement. Apart from a small difficulty regarding changing the provided funcmap in to a valuemap there is almost nothing to do to implement it.

Now, how else could this be implemented?

The suggestion of Clone Funcs Execute hasn't worked in the past and isn't going to work. It's too hard to speed up and in a lot of ways it's not the right thing. We don't want to change the template - just this execution of the template.

That leaves the question of is there a way to expose execution state outside of the current functions? Well... The current API means that a template.Template is a lot of somewhat disparate things. It is not simply a single parsed template, but a collection of parsed templates, a parser and an executor all of itself. That's fine and makes for a simple API but at some point it might be sensible to actually separate those things out and expose them individually. Is this the reason to do that? I don't necessarily believe it is. (Certainly any attempt to implement this functionality without the proposed functions would at least result in partial separation out of the executor from the template.) Does this API prevent that or make that migration harder? Again I don't think so. Any separation out is going to have to expose this kind of functionality and it would map nicely on to that.

[zeripath]

I should add that the slight distaste in the naming of the go API here should be balanced against the substantial cumbersomeness the current API is imposing on the template API, (and the impracticality of the suggested workaround).

A codebase using multiple templates is likely to have only one or two places where ExecuteFuncMap is written - compare that to every template having to account for the lack of execution funcmaps, pretending that contextual functions are somehow data and then having to fiddle about trying to remember if it is .Tr, $.Tr, $.root.Tr or even having to go back and change every call of a template to add root in. Whilst one can think of a locale as data to be rendered it is somewhat orthogonal to it and is genuinely a context of a render. (This extends to other things but locale is the simplest case.)

The currently suggested workaround of using Clone() + Funcs() + Execute() is slow and would require cloning all of the templates needed on a per execution basis. The current structure of templates means that can never be quick because the execution funcmaps are stored within the templates themselves and these have to have a concurrency lock because templates can parse themselves. This fundamentally cannot be sped up without significant restructuring of the whole of the text/template package whereas the proposed change is easy to do.

When balancing these factors, although these method names are slightly cumbersome, it is likely that only one of these 4 methods will be used only once or twice in a codebase. Not having them is making templates incredibly cumbersome and error prone. This suggests really that they would be a net reduction in cumbersomeness.

[earthboundkid]

@bep, would this simplify the implementation of Hugo? Being able to call a page function in templates would be nice.

[zeripath]

Dear golang team have there been any further thoughts about this?

[bep]

@carlmjohnson I don't understand this issue very well (I have only skimmed it). In Hugo we have a fork of the execution part of Go's template packages with some small adjustments (we have a script that keep the 2 in sync); one of them is a ExecuteWithContext method, which I think make more sense re. passing contextual data. And yes, it will eventually allow you to do page.Foo once I find time to wire everything ...