Add an interactive feature to putup

[FlorianWilhelm]

Using putup many people are often not aware of the many options it has. This results in projects that lack features like pre-commit and tox that people would actually choose if they knew about. One way to resolve this is to add an --interactive or -i feature to putup that asks the user about each option with sane defaults. Cookiecutter is doing something similar in its input mode.

[abravalheri]

If users run on an interactive mode, we could also ask them about file conflicts when updating...

[alextremblay]

I would also very much like to see an interactive mode added :)

[abravalheri]

In order to be able to implement something like that, I believe we need to split 2 concepts that are currently bundled together in the implementation of the cli:

(a) The specification of PyScaffold parameters
(b) The specification of putup parameters

Currently, we can already use PyScaffold without putup , since create_project already accepts a dictionary as input, which is independent of ARGV. However, there is currently no way the extensions can specify the parameters of create_project in a way they can be used outside argparse.

I would like to discuss bellow some of the approaches for tackling this:
(Warning pseudo-code ahead)

1. We can introspect argparse's parse object.
   Pros: No change required in the existing extensions and docs
   Cons: We would need to access argparse private APIs, which is NotGood^TM
   (I am basically including this option for the sake of completeness, but I am not a big fan)

2. We can create a declarative API on top of the existing Extension class. Something like:

   class Parameter:
       # cli_flags, action, arity, const, default , type, choices, required, help, metavar, dest
       # something like NameTuple with more or less the same options as argparse, but using kwargs and allowing optional arguments
   
   class Namespace(Extension):
       namespace: Parameter(arity=1, help="put your project inside a namespace")

   Pros: Very clean. All parameter definitions stick together with the extensions and we can rename the arguments into something generic (e.g. nargs => arity) so the abstraction is not leaky.
   Cons: Breaking changes. We have to change all the extensions and docs

3. We can do something in between, and instead of injecting an instance of argparse.ArgumentParser as the first element of Extension.augment_cli, inject an instance of a custom class that works more or less like a proxy to argparse, but also stores the information about the parameters somewhere to be accessed by the internals of PyScaffold.

   # Somewhere inside the api package
   class ParameterSpec:
      def add_argument(self, name or flags...[, action][, nargs][, const][, default][, type][, choices][, required][, help][, metavar][, dest]):
      # same API as argparse, so we don't have to change much, but internally stores the arguments and make them accessible
      self.specs.append((args, kwargs))
   
   # Somewhere else, likely cli.py
   def create_cli(specs, **opts):
       parser = argparse.ArgumentParser(**opts)
       for arg, kwargs in specs:
            parser.add_argument(*args, **kwargs)
       return parser

   Pros: Requires less changes to the existing code/docs. Backward compatible-ish. Allows creating more methods (e.g. modify_argument, delete_argument), that could be useful (e.g. an --interactive flag would require to modify mandatory arguments into optional, so we can launch the extension frontend from putup without triggering exceptions from argparse)
   Cons: Can lead to a leaky abstraction. We are probably not fully proxying argparse APIs, so it can be frustrating for extension writers.

[abravalheri]

In addition to my previous comment, we should also discuss how we would like to implement this feature:

1. Ad-hoc manner: we can simply inspect ARGV for the presence of --interactive or -i and instead of calling the regular putup's cli, call the interactive front-end.
2. Built-in into argparse: when --interactive is passed we replace run_scaffold by a different function into the command parameter of the parser.
3. Something more independent of argparse and more generic: right now cli.main calls:

   utils.check_setup_tools_version()
   opts = parse_args(args)
   opts = process_opts(opts)
   opts['command'](opts)

   we could make a separated pipeline out of it (as we do with the actions), and create entrypoints for that so extension can register something for that stage.
   Cons: Might be too complex and overpowering.
   (BTW, it might be worth checking if we can move process_opts to be an action, so create_project can also take advantage of it).

[abravalheri]

My opinion is that the proxy to argparse (3rd option first comment) + using the command parameter (2nd option second comment), can be just good enough, so we don't make things too complex and we don't break much stuff, paying the price of having a bit of leaky abstractions.

Not necessarily we have to implement this feature for 4.0 (and I believe it should be an extension in a separated package), but we should take advantage of moving to 4.0 (#235) to work on the changes of API and associated docs required to make this feature implementable.

P.S.: If we separate the concepts of arguments for PyScaffold and arguments for putup, it would be good to rename the entrypoint pyscaffold.cli to something different (again to avoid having leaky abstractions).

[FlorianWilhelm]

@abravalheri Thanks for all the input, gonna need some time to digest this. Would it be simpler if we just not allow the interactive feature for extensions? I mean extensions could even contradict each other, right? An interactive feature would feel as we would offer the mixture of different extensions which might actually break things.

A compromise would be to have interactive features only for official extension and rather use some registering way of adding this. Just some thoughts... will need some more time to wrap my head around it.

[abravalheri]

Hi @FlorianWilhelm thank you for the feedback. Let me reply your comments inline:

Would it be simpler if we just not allow the interactive feature for extensions?

We can always always avoid mixing the interactive feature with the extensions. However, I don't think this simplifies a lot. The problem here is double bookkeeping:

• We have a way of telling argparse what are the parameters and which are the possibilities for each one.
• If we don't want to touch argparse, we need a separated way of telling the interactive feature the same information. So we end up with 2 separated mechanism that do the same thing.

Imagine for example the --license flag and all the options it allows. We would need to do it twice for argparse and --interactive. So that is basically why I am proposing to proxy argparse and store the information for future inspection (argparse is very secretive about its internals, and private methods are the default).

I mean extensions could even contradict each other, right? An interactive feature would feel as we would offer the mixture of different extensions which might actually break things.

Well, that is actually true for putup CLI anyway... If the person installs 2 extensions that define the same flag, putup will break.

[SarthakJariwala]

Hi! Thanks for making and sharing this amazing package! I also thought an interactive version of putup would be helpful, so I decided to go ahead and make it as a separate package relying on PyScaffold- https://github.com/SarthakJariwala/PyScaffold-Interactive

Currently users can:

• Setup a new python project (using PyScaffold) and interactively add details like project name, author name, email, url, description
• Choose from a selection of licenses
• Configure automated testing using tox
• Setup continuous integration using Travis-CI
• In addition, all of PyScaffold's native commands are also available: putup --help - So users can update/manage using putup

[More features like support for other extensions that PyScaffold supports can be easily added in the future]

I know an interactive version is (was?) in the pipeline for v4.0 but in the meantime if PyScaffold users would like an interactive usage, you can share this with them and hope this helps them!

Happy to hear feedback/suggestions! 😃

[FlorianWilhelm]

@SarthakJariwala Cool! Thanks for sharing pyscaffold-interactive with us. This bridges the time until we have it included in PyScaffold directly. The development has slowed down a bit lately and some bigger decisions have to be taken due to the move to pyproject.toml and other things we need to consider for version 4.0.

[CarliJoy]

@FlorianWilhelm could you define the use case a bit more clear here?
Because if a user has troubles with the --help command then maybe a command line tool is not the way to go at all?

Maybe it is better & easier if we provide a GUI.
There is actually a package to help with this:
https://github.com/chriskiehl/Gooey

What do you think about this solution?

Because if I understand gooey correctly they already "hacked" argparse...

[FlorianWilhelm]

Hi @CarliJoy. The actual use-case is that lazy users, which do not want to read about PyScaffold's docs, might want to just type putup -i and they will be guided through every option it has. Personally, I also like interactive modes in other tools and commands.

Thanks for pointing out https://github.com/chriskiehl/Gooey, never heard of it before. Since PyScaffold addresses developers and I would argue that almost 100% of Python developers can at least use the command-line a bit, especially when they want to build an own Python package, I would see a GUI as a bit of an overkill.

[CarliJoy]

@FlorianWilhelm aggreed. The advantage of Gooey would be that is already does some tricks reading argparser configuration.
I guess in an interactive version of putup also extensions should be considered?
It would nice if such a feature is added that extensions would need to define only one user interface and do not need to hack the interactive mode as well.