Scaffold - tuist templating

[fortmarek]

Need/problem

Following up on swiftUI template PR from here, it makes sense to make something more generic, so adding such new templates is easy and can be read dynamic from one directory. It can also enable users to define their own templates, in addition to the basic ones offered by tuist.

Motivation

Improving upon projectDescription helpers, this is a logical next step to improve creating a new framework/project, etc. Not only can the users share the description throughout their project, users could with this feature share what the structure of a new component looks (i.e. pre-generated files, folders, you name it)

Detailed design

We will need to introduce a new command: tuist scaffold name_of_template Name
This will create a new eg framework (depends on your template) of name Name.
In addition to that, it will be useful to add tuist scaffold list, so users can immediately see what templates are available to them. These templates should be probably made available to tuist init, too, with added option such as tuist init --template name_of_template.

Templates will be of two types, each type handled a bit differently - tuist templates and custom templates (but it should be possible to envoke both of these with the same tuist scaffold command)

1. tuist templates: These templates will be dependent on tuist version, so tuist scaffold produces the same result for everyone in the team. For this reason, I think ~/.tuist/Version/version_number/Templates/... is an ideal place.
2. custom templates: I think it makes sense to place them alongside ProjectDescriptionHelpers in ${PROJECT_FOLDER}/Tuist/Templates. If there is a need to share these custom templates, we will need to think of how to port these easily to another project, or event make it possible to add custom system-wide templates - but we should focus on the core of this feature and leave this as a future possibility.

As for the template files themselves, I'd like to keep them Swift-generated, but if they become more complicated, we should probably resort to Stencil that makes such a task much more manageable.

Drawbacks

Since this feature is optional, I think the current Tuist users should not be affected in any way. But it is something that we will need to maintain if eg there are breaking (or just deprecation) Swift changes - therefore, I'd try to keep our provided templates as simple as possible to make this task easier.

It will also make .tuist directory a little bit more complicated, but in my view it is worth it for this use case.

Alternatives

I thought about keeping templates in the current project directory, but this approach makes the distinction between custom and tuist templates more clear. Also, changing local tuist version would mean changing the templates in the current dir, too, which would be a nightmare, so the current approach is much better.

How we teach this

As an extension of ProjectDescriptionHelpers, I think it can be incorporated into our documentation quite easy; it is something we should try to show off alongside each other to make a point of how easy it is to create a new feature, etc.

Unresolved questions

• What templates do we want to have?
• Are there any arguments that will need to be passed into the files in given templates (which would make the implementation a little bit trickier to keep it dynamic)?
• I am not really decided on scaffold name is the most memorable? Maybe something like tuist new framework Name would read a little better? But on the other hand that could clash with tuist init, but would love to know your thoughts.

[pepicrft]

Hey @fortmarek, first of all, thanks for coming up with such a detail proposal. Let me dump my thoughts on some of the points that you touch on:

In addition to that, it will be useful to add tuist scaffold list, so users can immediately see what templates are available to them.

Agree. In fact, I'd require users to include a description with their templates that we can show next to the identifier when listing them:

The following templates are available for scaffolding:
identifier_a    Use this template to create new projects that represent feature frameworks
identifier_b    Use this template to create a watch application

These templates should be probably made available to tuist init, too, with added option such as tuist init --template name_of_template.

That sounds a good idea!

Templates will be of two types, each type handled a bit differently - tuist templates and custom templates (but it should be possible to envoke both of these with the same tuist scaffold command)

Agree. We could bundle some templates with Tuist.

tuist templates: These templates will be dependent on tuist version, so tuist scaffold produces the same result for everyone in the team. For this reason, I think ~/.tuist/Version/version_number/Templates/... is an ideal place.

Would you then bundle those alongside the binaries and the project description framework?

custom templates: I think it makes sense to place them alongside ProjectDescriptionHelpers in ${PROJECT_FOLDER}/Tuist/Templates. If there is a need to share these custom templates, we will need to think of how to port these easily to another project, or event make it possible to add custom system-wide templates - but we should focus on the core of this feature and leave this as a future possibility.

I agree with the directory here. Inside that Tuist/Templates directory, each subdirectory could represent a template where the name of the directory is the identifier, and it contains a Template.swift with the definition of the template.

But it is something that we will need to maintain if eg there are breaking (or just deprecation) Swift changes - therefore, I'd try to keep our provided templates as simple as possible to make this task easier.

I'd include a suite of acceptance tests that make sure that templates generate files that are valid. The experience of using generated code that does not work is very frustrating.

It will also make .tuist directory a little bit more complicated, but in my view it is worth it for this use case.

I wouldn't worry about this because we just have a /Templates directory with each version.

As an extension of ProjectDescriptionHelpers, I think it can be incorporated into our documentation quite easy; it is something we should try to show off alongside each other to make a point of how easy it is to create a new feature, etc.

Scaffolding can have its own section in the documentation. We could explain why the feature is very useful to extend projects, how to use them, and some examples of templates that are vendored with Tuist.

What templates do we want to have?

• iOS app with Watch app
• Standalone watch app
• macOS app with frameworks.

Are there any arguments that will need to be passed into the files in given templates (which would make the implementation a little bit trickier to keep it dynamic)?

We can start with tuist scaffold template_identifier Name for now, and add support for passing variables in the future. What do you think?

I am not really decided on scaffold name is the most memorable? Maybe something like tuist new framework Name would read a little better? But on the other hand that could clash with tuist init, but would love to know your thoughts.

Rails uses generate but that's a verb that we are already using for the generation of Xcode projects. I like new too, but that's commonly used to mean that you are creating something for the first time. Do you think scaffold is not that common?

[kwridan]

I like the detailed proposal @fortmarek 👍

Something like this would be a great addition to Tuist.

I wonder what the difference between tuist init and tuist scaffold would be? They seem fairly similar as they both add files from templates. Would adding the --template option to init per your suggestion alleviate the need for a dedicated command?

Could the files currently generated via tuist init be converted into a template that follows the proposed structure?

[fortmarek]

Thanks for the comments @kwridan and @pepibumur! I'll try to answer or expand on the points you have raised:

Agree. In fact, I'd require users to include a description with their templates that we can show next to the identifier when listing them

That sounds like a good idea, definitely something we should implement!

Inside that Tuist/Templates directory, each subdirectory could represent a template where the name of the directory is the identifier, and it contains a Template.swift with the definition of the template.

What kind of definition? As in Project.swift config file or what exactly would be defined there?

I agree with the directory here. Inside that Tuist/Templates directory, each subdirectory could represent a template where the name of the directory is the identifier, and it contains a Template.swift with the definition of the template.

Agree!

We can start with tuist scaffold template_identifier Name for now, and add support for passing variables in the future. What do you think?

Yep, probably better to wait for the use case to emerge before starting to implement it.

Could the files currently generated via tuist init be converted into a template that follows the proposed structure?

Implementation-wise I think it is very much possible to add it to tuist init, but it might get a little bit trickier if we want to merge it completely with scaffold command - you'd need to differ between tuist init and tuist scaffold functionality since tuist init adds TuistConfig.swift and different files. Or tuist init without --template provided could do the same as it does currently and with --template option it would serve the scaffold functionality solely, but then it could result in some confusion on the user's side since the same command would do two different things basically.

So, as for the naming, I'd probably leave it with scaffold since it's a name that will result in the least confusion in the end (and make the implementation more straightforward)

[pepicrft]

So, as for the naming, I'd probably leave it with scaffold since it's a name that will result in the least confusion in the end (and make the implementation more straightforward)

Agree with this one. Rails for instance uses init to bootstrap a project with some conventions that just works, and then generate (the equivalent to our scaffold), to initialize components for the project.

[fortmarek]

Hey, I am trying to start work on this and I'd need some input.
At first I thought that we would be able to generate the files for the given template in Swift, but that may not be as easy because we want to let the users define their own templates.
There are two options how we can go about it (that I have thought of):
In Templates/MyTemplate directory there would be the whole structure of the code - the problem is I can have a template that has a directory called name_from_user_input - how do I rename that? We can probably define a String that we will find and always rewrite to the user input; but I don't like that without generating the files in code we will lose some flexibility - and adding additional parameters in the future would not be exactly nice (but still possible - we can take Stencil as our example).
The other option is to have in Templates/MyTemplate a Swift script eg generate.swift. There the user that defines the template can generate whatever he desires - he would just need to consume the argument with which the script will be invoked (ie ./generate.swift name).

Is there a better solution to this that I am not thinking of - and if not, which of these solutions makes more sense to you?

[pepicrft]

Hey @fortmarek, thanks for posting your concerns here.

In regards to where the templates should be defined, I think they should be defined under the Tuist/ directory. There could be a Tuist/Templates directory where each directory represents a template. The can define as a convention that the name of the template is the name of the folder.

Then we have to decide for a format for the template. These are the 2 options that you mention:

• Use a templating system like Stencil such that we copy the files under the directory and then substitute the variables.
• Have a manifest file for the template and provide a declarative interface to define the generation logic.

The first one is clearly more flexible and if you enter a template directory you can see at a glance what's in it. What I'd do though is require every template to have a Template.swift file where they indicate what attributes are required to scaffold using that template. For instance, if the template says that name is a string and is required, the user would have to call the scaffold command like:

tuist scaffold template_name --name MyFeature

Otherwise the command would fail. The format of the template model could be something along the lines of:

// Template.swift

let template = Template(
  attributes: [
    .required("name", default: nil),
  ]
)

This is just an idea, feel free to iterate until you find an API that is concise and readable.

[fortmarek]

Thanks for the comment @pepibumur

I definitely like the second option more and I have started to play with it, but I am still unsure how to actually then generate those files - as I have laid out in the previous comment, we can trigger a script that would be at the same place as Template.swift. I wonder if that can be fit into the declarative interface itself which would be the most clean solution, something along the lines of:

// Template.swift

let template = Template(
  attributes: [
    .required("name", default: nil),
  ], 
  directories: ["MyDirectory", "\(.argument(for: "name"))Tests"],
  files: [File(path: "MyDirectory/file.swift", contents: "Contents of this file")]
)

where \(.argument(for: "name") would be then replaced by the provided argument (could be static func on String) and therefore translate to NameTests in the example case.

You can also easily declare all directories and files with their respective contents with slick Swift interface.

Am I missing some implementation hurdle?

The only thing I can think of is that the arguments, as it stands, are limited to Strings - passing Bool would require some additional logic - but it might be possible.

[TimurBK]

Couple of things that might be useful:

• Arguments to be able to customise template. As an example: module can contain the functionality itself, demo app to run that will include that functionality, various tests(unit/ui/snapshot/performance/etc), mocks, accessibility etc. But when implementing non-UI things(i.e. networking) there's no sense to include snapshot tests and accessibility for example so it would be nice not to generate them and their files at all. While this could be solved by having 2 templates, in case of fixes/improvements in common parts you'll have to fix both.

• Option to specify path where the generated code will end up. Easier to always be inside root folder and then specify relative paths rather than cd'ing around.

[fortmarek]

Thanks for the input! 🙂

As for the first point, you will be able to shared code using a similar pattern as project description helpers - right now I call it TemplateDescriptionHelpers. That will enable to share configuration between templates. While that means that you will still need two templates since they will be able to share most of the configuration, I think that's a good trade-off.

Ad 2) yes, something I definitely plan to implement, thanks for raising this up, I'll add it to the checklist in the pr

[pepicrft]

As for the first point, you will be able to shared code using a similar pattern as project description helpers - right now I call it TemplateDescriptionHelpers. That will enable to share configuration between templates. While that means that you will still need two templates since they will be able to share most of the configuration, I think that's a good trade-off.

I was going to say the same. Instead of having another type of helpers, I'd reuse ProjectDescriptionHelpers. You'll be able to extract in those the reusable parts from your templates. For instance, you'll be able to define a Swift function that acts as a factory of templates:

func myTemplate(name: String, withUnitTests: Bool) -> Template {
  // Initialize the template
}

And then you can just do:

// MyTemplateWithUnitTests.swift
import ProjectDescriptionHelpers

let template = myTemplate(name: "Bar", witUnitTests: true)

That's the beauty of using Swift over YAML or JSON, that it enables this type of reusability by leveraging Swift modules.

[fortmarek]

Right now I have TemplateDescriptionHelpers and TemplateDescription as I felt like it's something a little bit different from ProjectDescription but I did not think about the fact that we'd lose a little bit of reusability.

Do you think it makes sense to merge TemplateDescription into ProjectDescription then?

[pepicrft]

Do you think it makes sense to merge TemplateDescription into ProjectDescription then?

I think so. The fewer of those, the easier it'll be to maintain them. In the end they expose models to describe things. Having them separated doesn't make that much difference for the user, but complicates the maintenance in our end.

[fortmarek]

Already implemented