Include files or directories using the template engine

[simobasso]

follow up #1485 (comment) from @HappyEinara

I do have one question on the implementation: the PR seems well able to include blocks of file content, but just as important would be the optional inclusion of whole files, directories or trees of directories in the same way (and analogously to the way cookiecutter permits templating in filenames). @simobasso do you have any thoughts on how that would be done?

[HappyEinara]

Hi @simobasso I've been thinking about this on my morning walk and I've struck upon a solution I think is quite elegant.

I propose a new Jinja filter built in to cookiecutter. For the sake of the following I'm naming it if, but see Naming below.

The if filter returns the expanded variable if the expression passed to it evaluates truthily, otherwise it returns the empty string. Then if the filename resolves to the empty string, it signals to cookiecutter that the current path and, if it's a directory, all descendant paths, are to be discarded and not processed or output. For example:

{{ cookiecutter.package }}/                                                                  
  |                                                                                          
  {{ cli | if(cookiecutter.include_cli) }}/                                                  
      |                                                                                      
      _ __init__.py                                                                          
  __init__.py

In this case, if the value include_cli in the template is False or falsy, the cli directory is not processed further and neither the directory nor /cli/__init__.py is included. As the empty string is not a valid filename, this would be perfectly backwards-compatible with existing templates.

Using a filter plus a conditional expression means you could do things like:

{{ cli | if("cli" in cookiecutter.components) }}

where components is a list variable of optional components or

{{ colorcli.py | if (cookiecutter.interface == "cli" and cookiecutter.use_colors) }}

for multiple conditions.

Bonus: applicability in files

If in the example above the file {{ cookiecutter.package }}/__init__.py contained the following:

{{ "import .cli" | if(cookiecutter.include_cli) }}

then that line would also automatically resolve to the empty string and the import statement would be left out. I think this is an attractively readable alternative to:

{% if cookiecutter.include_cli %}import cli{% endif %}

(NB: check if the newline would be removed with the filter, as it would for the if-block)

The problem

Templating syntax needs to be filename-friendly. I haven't exhaustively considered what is valid for expressions-in-jinja but not in filenames, but it needs guarding against. Perhaps the argument to if could be a string which is passed to eval, but that might also be problematic.

One solution I'd use in practice would be to set up booleans in cookiecutter.json, like this:

{                                                                                        
  "cookiecutter":                                                                        
    "package": "package_name",                                                           
    "include_lambda": false,                                                             
    "include_ecs": false,                                                                
    "include_aws": {{ cookiecutter.include_lambda or cookiecutter.include_ecs }}         
  }
}                                                                                        

and then just use:

{{ aws.py | if(cookiecutter.include_aws) }}

in the template pathnames themselves.

I should add that presumably handling the compatibility of template logic and filesystem paths is already a concern for cookiecutter and has been thought out. I haven't investigated the current situation but I don't think adding a new filter would make it worse unless cookiecutter, for example, limits filenames to only plain variables.

One possibility is to keep the logic that says a filename that's an empty string means "cookiecutter please ignore this", and then have:

  aws_dir: "{{ 'aws' | if (cookiecutter.include_aws) }}"

in the cookiecutter.json.

A further alternative is to put aside the filter idea, and just add the rule that makes cookiecutter discard trees under empty-string paths and use python logic:

aws_dir: "{{ ('aws' if cookiecutter.include_aws else '') }}"

But I'm not sure if that's valid Jinja, I can't remember.

Naming

I've used if above as the name for my filter because it's short (important for filenames), and it's incredibly readable. The code examples above are intuitive to understand. However this would clash with the Jinja2 keyword {%if%} and the python keyword. I don't know whether it would be possible to use that name and it's another question to resolve here. An alternative would be include_if but it's longer and less readable.

Keen to hear your thoughts.

[simobasso]

@HappyEinara, thanks for your proposal, it's exciting.

The idea of include directories is great.

I'm not sure about the if use; at this point add a new jinja extension can be a breaking change if someone already implemented it.

I have an alternative version of the implementation to reach the same results without inserting a new jinja extension, and I really appreciate your feedback about it. I only need few days to formalize it.

Thanks