templating in defaults file

[brainchild0]

Problem

Since defaults files were introduced, the utility of interpolation of variables in options values has been raised several times (see #5790, #5977).

Further discussion has surrounded functional needs that may be satisfied through the addition of interpolative processing (see #5871).

Overview

An optimal solution would carry the following features:

• Familiar: Idioms derive from established patterns supported by existing tools and languages.
• Noninvasive: Uses not involving the feature are minimally affected by its inclusion into the application.
• Flexible: Users are able to achieve different effects accommodating a variety of simple or sophisticated cases.
• Comprehensive: Most cases demanded by users are supported.
• Extensible: Support for further use cases can be added in the future without breaking compatibility or adding convolutions.

Design considerations might by divided into two categories, syntax and semantics. Syntax, in this context, relates to how expressions are demarcated from normal text, and how they are represented through a character sequence. Semantics relates to which data and operations are available and how they are referenced using syntax.

Concept

Syntax

Template language Mustache appears to offer an effective solution for syntax. Its advantages include simplicity, context agnosticism, rigorous specification, cross-language support, compound data types, and lambdas.

A Haskell implementation of Mustache is available in Hackage.

Each evaluation of a Mustache template involves a dictionary mapping data object, called the context, with keys typed to strings. Supported types for dictionary values are primitives, lists, functions, and dictionaries. Within a template, interpolated expressions are delimited by two pairs of squiggly braces. The text {{foo}} evaluates to the value corresponding to the foo key in the context. Similarly, {{foo.bar}} evaluates to the value found by the bar key in a dictionary corresponding the the foo key in the context; and {{foo.0}} evaluates to the first element of a list corresponding to the foo key in the context. The text {{#func}}val{{/func}} evaluates to the value returned from passing the string val to the lambda found with the key func.

Semantics

Using basic building blocks offered by Mustache, it is easy to imagine how to build powerful yet plain semantics.

A few examples might be:

• {{usr.home}}: Evaluates to the user home directory.
• {{op.cwd}}: Evaluates to the current working directory.
• {{env.varname}}: Evaluates to environment variable varname.
• {{md.title}}: Evaluates to the metadata variable title.
• {{op.infiles.0}}: Evaluates to the name of the first input file.
• {{#pd.filesafe}}{{md.title}}{{/pd.filesafe}}.pdf: Evaluates to a string derived from the title, but safe for file names and without whitespace, followed by a .pdf suffix.
• {{#pd.basename}}{{op.infile.0}}{{/pd.basename}}.pdf: Evaluates to the name of the first input file, with the original extension replaced by .pdf.

These examples are based on the assumption that the context would be populated with a few predefined tables, each representing conceptually a namespace of related data or functions. The namespaces usr and op represent data related to the user's configuration and the current application invocation. The namespaces env and md show views of the environment variables and metadata fields. The namespace pd provides a set of functions useful for data transformations.

Remarks

Mustache syntax makes the solution familiar and noninvasive. Separation of data and functions into distinctive namespaces makes the solution flexible, comprehensive, and extensible.

[iandol]

This would be a flexible "enhanced" solution to the issue I raised in #5982 with a {{usr:datadir}} added to the option list.

[brainchild0]

Suggest keeping open, since 6dd7520 has much more narrow scope than immediate topic.