Prompts with help

[jakubka]

This is yet another try at introducing a new cookiecutter.json format which will support specifying help text for individual parameters which will be then printed when prompting.

There've been several attempts to do that (#249, #30, #381, #433), but all now seem outdated for one reason or another, so I am opening this new issue. I think old ones should be closed to reduce confusion. I've gone through discussions on the above issues/PRs and picked up the good parts.

Ultimately, I would love cookiecutter to be able to prompt for parameters similarly to how sphinx-quickstart does:

New cookiecutter.json format

Backwards compatibility has to be considered when designing a new cookiecutter.json format. First, the current version of cookiecutter should handle the new format. Second, new cookiecutter should support old format.

I propose this new format:

{
    // old way of specifying parameters is to keep backwards compatibility
    // of the new format with the old cookiecutter executables
    // which do not support the new '_parameters' section yet
    // it will be overwritten by '_parameters' by the new cookiecutter
    "directory_name": "default_directory_name",

    // '_parameters' section will be ignored by the old cookiecutter executables
    // for new cookiecutter '_parameters' section will be the main way of specifying parameters
    // old proposals prefer the name '_context' for this section, but I think that '_parameters' is more self-explaining. hmm?
    "_parameters": {
        "directory_name": {
            "default_value": "default_directory_name",
            "prompt": "Name of the directory which will contain the generated project",
            "help": "This will be the root of the new project. bla bla."
        }
    },

    // '_required_cookiecutter_version' is not strictly required for now,
    // but it will make our lives much easier in the future
    // it should be a separate PR for sure, so just including it here to show this option
    "_required_cookiecutter_version": "2.1.3"
}

This format will fully handle the backwards compatibility. I acknowledge that writing cookiecutter.json will be more difficult from now on, but I think that is an acceptable tradeoff. After all, cookiecutter.json is written once, but used many many times, so I prefer to optimize for the end user experience.

What do you think?

If main contributors are happy with the proposal, I can take care of the code and documentation changes.

[hackebrot]

Thank you @jakubka for submitting this proposal. 🙇

I like the fact that cookiecutter.json would work on older cookiecutter versions due to the way it handles _parameters.

That being said, it is not something that would be feasible in the long run as we add more features and also for templates that use numerous variables, such as cookiecutter-django.

The proposed solution covers:

• default values
• prompts
• help messages

However there is more that we have to take into consideration:

• nested parameters
• input validation
• dependencies
• sub templates
• variable types

Although we don’t need to add all of the above in one go, it’s clear that this won’t really work with just using the JSON file. I think we need a proper plugin system to handle with these requirements. Please feel free to give feedback on that 👍

Again I appreciate you taking the time to submit this request.

[jakubka]

Thanks for the feedback @hackebrot! Much appreciated.

Do we have more details about such plugin system somewhere? Do you mean some kind of extension to the current hooks model or something bigger? Has it been discussed?

My initial thoughts are that plugin system is over complicated solution for something we want to achieve. I see that there are requests for other features, but how important are they compared to the ones which can be achieved with just having JSON? From your list it looks to me that JSON is good enough for the majority of the items. If we try to develop a perfect solution, we might end up with nothing. I would personally prefer to add features by extending the JSON format, simply because it's the simpler solution.. After the limits of the single JSON config file are hit, a cookiecutter2 with completely different design can be released.

But maybe I imagine something bigger and more complicated than you have in mind. Please elaborate. :)

[jakubka]

@hackebrot any chance you can take a look? thanks!

[zooba]

(cc @huguesv)

Since using a dict for the default value is already supported, why not just use more fields on those? [EDIT: I see that PR isn't actually merged... my mistake]

We are adding a GUI interface to Visual Studio for cookiecutter (see here for some early screenshots) and would love to be able to use rich information about each field. Nested parameters and templates are less interesting to us, but we're really keen to have value selectors (e.g. pop open a file browser, or a database connection string generator, etc.).

We'll happily go our own way with extensions and underscore-prefixed fields if we need to, but we'd obviously much rather have everyone using the same fields so that developers don't have to write multiple entries.

[jakubka]

@zooba the proposal in #360 isn't forward compatible. Meaning that already released cookiecutter versions would fail with this new format. That was the reasoning behind the format I proposed in this discussion.

That being said, maybe keeping forward compatibility isn't important and we can safely choose the more elegant solution as proposed in the linked PR. This is something I can't decide.

Btw I think inclusion in an official VS plugin is a great opportunity for cookiecutter to become even more mainstream. Great to hear! :) cc @audreyr

[Cadair]

Why not add a section to the JSON just for help prompts that can be ignored by old versions and is independent of the values.

{
    // Still use this format for specifying variables
    "directory_name": "default_directory_name",

    // extra help dictionary maps to above structure. 
    // This could have nested dicts to handle nested prompts.
    "_help": {
        "directory_name": {
            "prompt": "Name of the directory which will contain the generated project",
            "help": "This will be the root of the new project. bla bla."
        }
    },
}

[astrofrog]

I think something like what @Cadair is suggesting would be great in the short term. While I agree a plugin system would be great, at the basic level, being able to have more verbose questions is a pretty important feature, and probably not something that should be provided just by a plugin?

[jakubka]

@Cadair your suggestion is practically the same as the original one, except I propose more generic name _parameters to specify additional metadata of the parameters. You used _help for the same purpose.

In future we can very easily add more possible options to the options in _parameters... How is having _help better than _parameters?

Anyway, it looks like this proposal is being superseded by #848 which takes it much further.

[Cadair]

Yes it is practically the same as yours but still using the original default value specification. _help or _parameters is all basically the same.

As for #848 it seems that that is justifiably on hold. However, I really feel that at the bare minimum cookiecutter needs a way of printing a help prompt during the question process.

[nicktimko]

Has this been subsumed by some other issue and/or implemented elsewhere? #848 also looks stalled, but also looks gigantic.

Right now the workaround I have is to print some help text after the user enters all their variables via a pre_gen_project hook and explain why that variable you entered is invalid. That's kinda frustrating if there are a bunch.

Another workaround I concocted is like the below (and just do nothing with helpX's)

{
  "help1": "The maintainers should be GitHub usernames. Enter to continue...",
  "primary_maintainer": "",
  "secondary_maintainer": "",
  "help2": "The 'app_name' must be a valid Python module name. Enter to continue...",
  "app_name": "example"
}

---

help1 [The maintainers should be GitHub usernames. Enter to continue...]:        
primary_maintainer []: nicktimko                                                 
secondary_maintainer []:                                                         
help2 [The 'app_name' must be a valid Python module name. Enter to continue...]: 
app_name [example]:                                                              

Are there any better alternatives meanwhile?

[limdauto]

I suppose adding a +1 here isn't productive. However, I'd like to add a workaround for people who stumble upon this to implement their own solution for now. In our project, which builds a framework on top of cookiecutter among other things, what we do is have an optional prompts.yml file next to cookiecutter.json which contains more information like the human-friendly title, text, regex validation, etc. for a corresponding field in cookiecutter.json. We then loop through each of these prompts and render the extra information along side the default values coming from cookiecutter by using cookiecutter's lower-level functions like read_user_variable or render_variable

[simkimsia]

Thanks for leaving a comment here, @limdauto else i would not have heard of Kedro.

I'm going to ask questions about Kedro over here

[havardox]

Any progress on this?

[mik3h0]

I believe this has been addressed