Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add an interactive feature to putup #191

Closed
FlorianWilhelm opened this issue Jul 11, 2018 · 16 comments
Closed

Add an interactive feature to putup #191

FlorianWilhelm opened this issue Jul 11, 2018 · 16 comments
Labels
needs discussion This topic should be discussed further
Milestone

Comments

@FlorianWilhelm
Copy link
Member

Using putup many people are often not aware of the many options it has. This results in projects that lack features like pre-commit and tox that people would actually choose if they knew about. One way to resolve this is to add an --interactive or -i feature to putup that asks the user about each option with sane defaults. Cookiecutter is doing something similar in its input mode.

@FlorianWilhelm FlorianWilhelm added the needs discussion This topic should be discussed further label Jul 11, 2018
@abravalheri
Copy link
Collaborator

abravalheri commented Jul 12, 2018

If users run on an interactive mode, we could also ask them about file conflicts when updating...

@alextremblay
Copy link

I would also very much like to see an interactive mode added :)

@abravalheri
Copy link
Collaborator

In order to be able to implement something like that, I believe we need to split 2 concepts that are currently bundled together in the implementation of the cli:

(a) The specification of PyScaffold parameters
(b) The specification of putup parameters

Currently, we can already use PyScaffold without putup , since create_project already accepts a dictionary as input, which is independent of ARGV. However, there is currently no way the extensions can specify the parameters of create_project in a way they can be used outside argparse.

I would like to discuss bellow some of the approaches for tackling this:
(Warning pseudo-code ahead)

  1. We can introspect argparse's parse object.
    Pros: No change required in the existing extensions and docs
    Cons: We would need to access argparse private APIs, which is NotGoodTM
    (I am basically including this option for the sake of completeness, but I am not a big fan)

  2. We can create a declarative API on top of the existing Extension class. Something like:

    class Parameter:
        # cli_flags, action, arity, const, default , type, choices, required, help, metavar, dest
        # something like NameTuple with more or less the same options as argparse, but using kwargs and allowing optional arguments
    
    class Namespace(Extension):
        namespace: Parameter(arity=1, help="put your project inside a namespace")

    Pros: Very clean. All parameter definitions stick together with the extensions and we can rename the arguments into something generic (e.g. nargs => arity) so the abstraction is not leaky.
    Cons: Breaking changes. We have to change all the extensions and docs

  3. We can do something in between, and instead of injecting an instance of argparse.ArgumentParser as the first element of Extension.augment_cli, inject an instance of a custom class that works more or less like a proxy to argparse, but also stores the information about the parameters somewhere to be accessed by the internals of PyScaffold.

    # Somewhere inside the api package
    class ParameterSpec:
       def add_argument(self, name or flags...[, action][, nargs][, const][, default][, type][, choices][, required][, help][, metavar][, dest]):
       # same API as argparse, so we don't have to change much, but internally stores the arguments and make them accessible
       self.specs.append((args, kwargs))
    
    # Somewhere else, likely cli.py
    def create_cli(specs, **opts):
        parser = argparse.ArgumentParser(**opts)
        for arg, kwargs in specs:
             parser.add_argument(*args, **kwargs)
        return parser

    Pros: Requires less changes to the existing code/docs. Backward compatible-ish. Allows creating more methods (e.g. modify_argument, delete_argument), that could be useful (e.g. an --interactive flag would require to modify mandatory arguments into optional, so we can launch the extension frontend from putup without triggering exceptions from argparse)
    Cons: Can lead to a leaky abstraction. We are probably not fully proxying argparse APIs, so it can be frustrating for extension writers.

@abravalheri
Copy link
Collaborator

In addition to my previous comment, we should also discuss how we would like to implement this feature:

  1. Ad-hoc manner: we can simply inspect ARGV for the presence of --interactive or -i and instead of calling the regular putup's cli, call the interactive front-end.
  2. Built-in into argparse: when --interactive is passed we replace run_scaffold by a different function into the command parameter of the parser.
  3. Something more independent of argparse and more generic: right now cli.main calls:
    utils.check_setup_tools_version()
    opts = parse_args(args)
    opts = process_opts(opts)
    opts['command'](opts)
    we could make a separated pipeline out of it (as we do with the actions), and create entrypoints for that so extension can register something for that stage.
    Cons: Might be too complex and overpowering.
    (BTW, it might be worth checking if we can move process_opts to be an action, so create_project can also take advantage of it).

@abravalheri
Copy link
Collaborator

abravalheri commented Jul 15, 2019

My opinion is that the proxy to argparse (3rd option first comment) + using the command parameter (2nd option second comment), can be just good enough, so we don't make things too complex and we don't break much stuff, paying the price of having a bit of leaky abstractions.

Not necessarily we have to implement this feature for 4.0 (and I believe it should be an extension in a separated package), but we should take advantage of moving to 4.0 (#235) to work on the changes of API and associated docs required to make this feature implementable.

P.S.: If we separate the concepts of arguments for PyScaffold and arguments for putup, it would be good to rename the entrypoint pyscaffold.cli to something different (again to avoid having leaky abstractions).

@FlorianWilhelm
Copy link
Member Author

@abravalheri Thanks for all the input, gonna need some time to digest this. Would it be simpler if we just not allow the interactive feature for extensions? I mean extensions could even contradict each other, right? An interactive feature would feel as we would offer the mixture of different extensions which might actually break things.

A compromise would be to have interactive features only for official extension and rather use some registering way of adding this. Just some thoughts... will need some more time to wrap my head around it.

@abravalheri
Copy link
Collaborator

Hi @FlorianWilhelm thank you for the feedback. Let me reply your comments inline:

Would it be simpler if we just not allow the interactive feature for extensions?

We can always always avoid mixing the interactive feature with the extensions. However, I don't think this simplifies a lot. The problem here is double bookkeeping:

  • We have a way of telling argparse what are the parameters and which are the possibilities for each one.
  • If we don't want to touch argparse, we need a separated way of telling the interactive feature the same information. So we end up with 2 separated mechanism that do the same thing.

Imagine for example the --license flag and all the options it allows. We would need to do it twice for argparse and --interactive. So that is basically why I am proposing to proxy argparse and store the information for future inspection (argparse is very secretive about its internals, and private methods are the default).

I mean extensions could even contradict each other, right? An interactive feature would feel as we would offer the mixture of different extensions which might actually break things.

Well, that is actually true for putup CLI anyway... If the person installs 2 extensions that define the same flag, putup will break.

@SarthakJariwala
Copy link

Hi! Thanks for making and sharing this amazing package! I also thought an interactive version of putup would be helpful, so I decided to go ahead and make it as a separate package relying on PyScaffold- https://github.com/SarthakJariwala/PyScaffold-Interactive

pyscaffold_interactive

Currently users can:

  • Setup a new python project (using PyScaffold) and interactively add details like project name, author name, email, url, description
  • Choose from a selection of licenses
  • Configure automated testing using tox
  • Setup continuous integration using Travis-CI
  • In addition, all of PyScaffold's native commands are also available: putup --help - So users can update/manage using putup

[More features like support for other extensions that PyScaffold supports can be easily added in the future]

I know an interactive version is (was?) in the pipeline for v4.0 but in the meantime if PyScaffold users would like an interactive usage, you can share this with them and hope this helps them!

Happy to hear feedback/suggestions! 😃

@FlorianWilhelm
Copy link
Member Author

@SarthakJariwala Cool! Thanks for sharing pyscaffold-interactive with us. This bridges the time until we have it included in PyScaffold directly. The development has slowed down a bit lately and some bigger decisions have to be taken due to the move to pyproject.toml and other things we need to consider for version 4.0.

@CarliJoy
Copy link
Contributor

@FlorianWilhelm could you define the use case a bit more clear here?
Because if a user has troubles with the --help command then maybe a command line tool is not the way to go at all?

Maybe it is better & easier if we provide a GUI.
There is actually a package to help with this:
https://github.com/chriskiehl/Gooey

What do you think about this solution?

Because if I understand gooey correctly they already "hacked" argparse...

@FlorianWilhelm
Copy link
Member Author

Hi @CarliJoy. The actual use-case is that lazy users, which do not want to read about PyScaffold's docs, might want to just type putup -i and they will be guided through every option it has. Personally, I also like interactive modes in other tools and commands.

Thanks for pointing out https://github.com/chriskiehl/Gooey, never heard of it before. Since PyScaffold addresses developers and I would argue that almost 100% of Python developers can at least use the command-line a bit, especially when they want to build an own Python package, I would see a GUI as a bit of an overkill.

@abravalheri abravalheri added this to the v4.0 milestone Dec 23, 2020
@CarliJoy
Copy link
Contributor

@FlorianWilhelm aggreed. The advantage of Gooey would be that is already does some tricks reading argparser configuration.
I guess in an interactive version of putup also extensions should be considered?
It would nice if such a feature is added that extensions would need to define only one user interface and do not need to hack the interactive mode as well.

@CarliJoy
Copy link
Contributor

BTW: I tried to get Gooey to work with pyscaffold and run into some problems. Let's see if I can fix them...

@CarliJoy
Copy link
Contributor

Okay tried some stuff with gooey and the GUI works but seems maintance intensive.
Also running cli and gui code does not work, as one has to add an "widget" option to the argparse.add_argument call which does not allow running vanilla argparse anymore.

@abravalheri i saw you came quite far with your interactive approach extending argparse. I like this approach very much. Can I help you somehow?

@abravalheri
Copy link
Collaborator

Hi @CarliJoy sorry for keeping the conversation about the interactive feature scattered among a few issues and PR.
The latest conversations seem to point out the most "cost effective" would be rename and adopt #333 to provide "user interaction", inspired by the way git rebase -i works. That implementation is intended to be backward compatible with the CLI (indeed we use the same parser). I still have to debug it properly and polish the edges before merging, you are more than welcome to help (thank you very much!). The more eyes trying it and seeing what does not work, the better.

@FlorianWilhelm
Copy link
Member Author

This issue is resolved by an interactive/editable modus that was added in version 4.0, which works a bit lit git rebase -i. Let's see how users react to it :-)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
needs discussion This topic should be discussed further
Projects
None yet
Development

No branches or pull requests

5 participants