Introduce features, a way to include files conditionally

[mainrs]

It would be really cool if cargo-generate would have something like cargo's features, where a user could select to include certain configuration/setup files of a template if he wants to.

An example: My template contains GitHub Actions workflow files for creating releases on tag pushes on master as well as creating documentation on the gh-pages branch when a push to master happens. The template would, by default, include everything. But if a user doesn't want the docs workflow, he could run something similar to this:

cargo-generate --git <url> --no-docs-workflow
cargo generate --git <url> --no-default-features --features docs-workflow

The first is a somewhat nicer syntax for this use-case I think. The second one would be the same as the one from cargo.

I imagine something like this inside the configuration file:

[[features]]
name = "docs-workflow"
include = [".github/workflows/docs.yml"]

[[features]]
name = "bors"
include = ["bors.toml"]

This comes really handy in situations where templates offer a lot of pre-defined configuration files with sane and good default values for various services (GitHub Actions, bors etc). That way, a user can just pick what he wants and be done.

Feedback is appreciated!

Edit: Additionally, this could be expanded upon composing features into "super-features". For example:

[[features]]
name = "workflows"
uses = ["docs-workflow", "publish-workflow", "pr-workflow"]

This would introduce more code to maintain, as something like cyclic dependency checks have to be performed on the features before running them.

[k0pernicus]

You're 100% right - a feature like that can be appreciate, especially for big templates!

I really like the idea behind the configuration file.
This can be introduced as a big new feature like "project customisation", as you only take the "add-ons" of the project you want...

[mainrs]

For reference, I finally had some time to finish up old projects of mine and tinkered around with this idea for https://github.com/SirWindfield/cargo-create. The name is a little bit misleading, as the tool is more of a general purpose project template generator than one specific for cargo. It does, however, register two binaries to make it more ergonomic inside the cargo workflow.

It doesn't contain much right now. It has some default template variables that get filled in (system, datetime, author and project name). But it does support the features feature I mentioned here.

The crate isn't meant as competition or anything similar. It was more meant to be a workspace that I can use to try out some crates that I wanted to use in other projects (tera, inventory, tracing).

However, it does work without major problems (that I found) and I use it for project generation.

This issue can easily be expanded with stuff that builds on top of this. For example patches that could be run (template engines are quite powerful these days though, so this might not be worth it). Conditional features that are enabled as soon as another feature is enabled as well, kind of creating chains.

If you want to give it a go:

cargo install cargo-create --locked
cargo-create -g SirWindfield/template-test -n projectName -f ci-workflow msrv

I am not sure why cargo create doesn't work 🤔

[Rahix]

This is also an interesting features for project templates in embedded Rust: A single project template might support a number of different microcontrollers or variants of MCUs which currently has to be configured by hand after generating the project (see cortex-m-quickstart's README for an example). I think it would be nice if one could, either interactively or via command-line flags, select the various configuration options during project generation.

The OP in this issue proposes a purely command-line flag based approach but as cargo-generate is already interactively querying information from the user, I could also see it asking questions for the feature/config selection.

[taurr]

@Rahix
#17 Offers cusom variables now - AFAIK I believe it only supports strings for now, but given it is expanded to support e.g. booleans, wouldn't that solve this issue??

I could see #17 expanded with a --default-vars <file-with-simple-vairable-assignments in order to even support templating the templates 😆

[taurr]

#17 (Custom Variables), fully supports several types, including a bool that can be used for this exact purpose.
It even supports the external file with variable values.

[sassman]

There is a way to make conditional decisions within one file, not to include files conditionally.
To give an example:

{% if network_enabled %}
use std::net::TcpListener;

fn main() {
    let listener = TcpListener::bind("0.0.0.0:8080").unwrap();
    loop {
        let (conn, addr) = listener.accept().unwrap();
        println!("Incoming Connection from {}", addr);
        std::io::copy(&mut &conn, &mut &conn).unwrap();
    }
}
{% else %}
fn main() {
    println!("Hello Rusty Hermit 🦀");
}
{% endif %}

the only missing thing is a way to supply the variable via a dedicated cli argument.

[taurr]

Variables can be supplied via cli arguments now.

But we still need a way to selectively include files don't we (or maybe as a filter upon .genignore)?

[taurr]

I actually like the idea of doing this in the cargo-generate.toml file:

[template]
cargo_generate_version = ">=0.9.0"
# default include/exclude goes here

[placeholders]
# default placeholders go here

[feature.default]
# "default" feature include/exclude goes here, will be added to the default include/exclude above

[feature.default.placeholders]
# additional placeholders for "default" feature goes here, will be added to the default placeholders above

[feature.foo]
uses = [ "default" ]
# "foo" feature include/exclude goes here, will be added to those of feature "defaults"

[feature.foo.placeholders]
# additional placeholders for "foo" feature goes here, will be added to those of the "default" feature

then we could do stuff like

cargo generate favorite --feature foo

and maybe even

cargo generate favorite --list-features

to list the possible features.

[MarcoIeni]

I think adding features just to include files conditionally is overkill since we already have custom variables.

Example use case

Say that we want to include conditionally main.rs or lib.rs depending if the project is a binary or a library.
When we run cargo generate with the --lib flag we already set the variable crate_type.
Now we have all the necessary info to include:

• main.rs if crate_type == bin
• lib.rs if crate_type == lib

If we implement "conditional file includes" with features instead, in this case I would need to add an extra feature to include main.rs or lib.rs. So for example if I want to include lib.rs, the command would be something like cargo generate ... --lib --feature my-library-feature

What syntax should we use in cargo-generate.toml to include files conditionally by using custom variables instead of features?

I don't know, it could end up be really complex.
What about the following simple solution instead?

Simple solution

The simplest solution would be to exclude empty files from the generated project, as proposed in this cookiecutter comment.

So for example my main.rs file could be something like:

{% if crate_type == "bin" %}
fn main() {
    println!("Hello, world!");
}
{% endif %}

Then, if I run cargo generate ... --lib, the if condition is not satisfied, so main.rs will be empty, therefore the file will not be included in the generated project.

Recursively, empty folder could be removed from the generated project, too.

What if I actually want empty files?

Sometimes you might need empty files, like for example the .gitkeep file.
In this case we could specify empty files that you want to keep in the generated project as an array in the Cargo.toml.
For example:

[template]
empty_files = [".gitkeep"]

Positive note on features

I like the fact that with features you could specify different set of placeholders as taurr showed, but again, I don't think they should be necessary to include files conditionally.

[taurr]

@MarcoIeni, @sassman

I liked the above solution so much, I made PR #431 .

it basically allows everything described by @MarcoIeni above, while also allowing us to conditionally ignore files and have conditional placeholders.

In order to facilitate this, I made use of the liquid engine and a little toml.

Example:

[template]
cargo_generate_version = ">=0.9.0"
# include/exclude/ignore also allowed here

[placeholders]
# common placeholders goes here

[conditional.'{% if crate_type == "bin" %}true{% endif %}']
ignore = [ "lib.rs" ]
# include/exclude also allowed here

[conditional.'{{var|truthy}}'.placeholders]
# placeholders for when {{var}} is considered false goes here

The inclusion of ignore in cargo-generate.toml makes .genignore obsolete.

From what I can tell, this works really well, and allows for a lot of flexibility in regards to what files are ignored/included/excluded and what variables will be queried.

[MarcoIeni]

Great, thanks!

conditional files

So files are not ignored if the file is empty, but if they are listed in the ignore list, like in:

[conditional.'{% if crate_type == "bin" %}true{% endif %}']
ignore = [ "lib.rs" ]

right?

Second question: Can you list also directories in ignore = or just files? If directories are not included we could open a separate issue.

Third question:
Do we want to use liquid syntax? Maybe something like this looks better?
Or maybe using liquid has some advantages (maybe it does some work for us)?

[[conditional_ignore]]
condition = { key = "crate_type", value = "bin" }
ignore = [ "lib.rs" ]

[[conditional_ignore]]
condition = { key = "crate_type", value = "lib" }
ignore = [ "main.rs" ]

[[conditional_ignore]]
condition = { key = "my_bool_var", value = true }
ignore = [ "subdir/foo.rs" ]

conditional placeholders

[conditional.'{{var|truthy}}.placeholders]
# placeholders for when {{var}} is considered false goes here

As I commented in the PR, the problem I see is that this doesn't support string comparison in the condition.
But I think it's better if we discuss "conditional placeholders" in a separate issue in order to avoid confusion.

EDIT: probably {{var|truthy}} is just syntactic sugar, so we can still use the '{% if crate_type == "bin" %}true{% endif %}' syntax also for placeholders, so my point is not correct.

[taurr]

1. Having a file/directory in the ignore section works exactly the same as putting it in the .genignore file. Haven't tried with a dir, but it's the same code doing the ignoring! (need to find out)

2. in this case I prefer the liquid syntax. Though it's verbose, it's easy to expand upon, and it doesn't introduce yet a new syntax

3. String comparisons are indeed possible. Only requirement if that the expression should render into something in order for it to be considered true. In case the 2 filters are used it just becomes possible to render something meaning false (namely 0, false or nothing)

EDIT: For the first question, you are correct; cargo-generate does not ignore empty files. It only ignored those that are explicitly in the .genignore file.

[MarcoIeni]

Thanks for the clarifications.
I am fine with using the liquid syntax :)

[MarcoIeni]

What do you think of surrounding the parts of toml we want to add conditionally with the liquid condition? Something like this:

{% if crate_type == "bin" %}
ignore = [ "lib.rs" ]
# include/exclude also allowed here
{% endif %}

{% if var == "true" %}
[placeholders.my_var]
# my_var data, which is included when {{var}} is considered true
{% endif %}

[taurr]

That leaves a problem, as in order to render the toml file we would need to read and parse it first, which we can't as it's not legal toml until after the render!

The file still contains settings that are, or may become, needed before we can resolve the conditions - an example is placeholders for the variables that the conditions use. So, we would need to read the toml file in order to resolve it.

TBH, I think the only way to do that would require us to write our own toml parser/alternate syntax. Doing so would also make sure we can not use any of the nice toml syntax highligters etc. that exist for all larger editors - so no, I do not recommend this route atm.

[taurr]

@MarcoIeni
I just added a few extra filters to the mix. They make it a little simpler to make conditions on crate_type.

[conditional.'{{crate_type|is_bin}}']
# section used when expanding with option --bin

[conditional.'{{crate_type|is_lib}}']
# section used when expanding with option --lib

[conditional.'{{crate_type|is_macro|falsy}}']
# filters can be mixed... this section is when crate_type != "proc-macro"

at least to me, when having these filters, this is just as readable as the stuff we have to write in Cargo.toml to do conditional dependencies.

[MarcoIeni]

Nice! What about is_macro? macro It's not ducumented in the readme.

[taurr]

Yeah, I might have forgotten that one - there's really no way of setting cratetype` to macro, and it might have been around this time I started to doubt my own experiment, wishing for a more natural fit for the domain of evaluating expressions.