CLI flag to only run certain sections?

[PacketPerception]

Would it make sense to add a flag to only run the clean, link, or shell sections of the config? Sometimes I dont want the shell commands to run so I just comment them out. This could be implemented using flags (dotbot -c/--clean -l/--link -s/--shell), or possibly as "sub-commands" (dotbot {clean,link,shell} or dotbot run {clean,link,shell}) and the default would still be to perform all three tasks.

I also thought it might be useful to add another optional argument to shell configurations to prompt the user if they want the shell command to run or not, something like "confirm: true". Not really sure that would be useful if I could just tell dotbot to only clean and link.

[Vaelatern]

Problem is, sometimes people re-order certain commands to run other commands before. In more complicated setups, this means link commands, shell commands, then some link commands, are run in various orders, and things might fail if the shell commands before them were not run.

A possible solution for you would be to export your shell commands to scripts in your repository that, before they run, confirm you want to run.

[PacketPerception]

I don't agree that your scenario would introduce any problems, as I don't think the default behavior should change at all. It would just be extra, optional, arguments.

Currently, I'm doing what you suggested, creating external scripts that prompt for confirmation. This already requires settings extra arguments (stdin: true and stdout: true), so adding a confirm option would make the configuration file simpler and not force users to clutter there repos with extra scripts that simply confirm whether or not they want the command to run.

[Vaelatern]

So it's not actually CLI arguments you are looking for, just an option in YAML to prompt before running?

[PacketPerception]

For the confirm option yeah. I see how I should have made that more clear >D

On Thursday, June 18, 2015, Toyam Cox notifications@github.com wrote:

So it's not actually CLI arguments you are looking for, just an option in
YAML to prompt before running?

—
Reply to this email directly or view it on GitHub
#44 (comment).

[Vaelatern]

I'll poke at the source code and see if I understand the structure enough to do that.

[PacketPerception]

Nice of you to do so! I'd recommend looking at the executor/commandrunner.py script, adding a check for item.get('confirm', False) is True (to follow the convention already in place), then ask for confirmation and just continue to the next command if no, otherwise run it.

[anishathalye]

What's the use case for this? What kinds of commands might you want to only run sometimes? And when would you want to leave the decision up to the user instead of automating the decision making (e.g. rewriting "fetch and install app" to an idempotent operation like "if application is not installed, fetch application from internet and install it").

At least in my use, my config has been idempotent. I think it's a good idea to try to structure stuff like this, so that you can always safely run ./install without worrying about how it's going to interact with past executions of ./install.

[PacketPerception]

I agree that it should be idempotent, my use case is I have multiple
commands that will check for updates and perform them if necwssary (think
Vim Vundle updates) that take 30-49seconds each. It's really only while
working on/expanding my dot files where I will run the link section
multiple times does it become an annoyance. So I just comment out my shell
section for the time being.

On Sunday, June 21, 2015, Anish Athalye notifications@github.com wrote:

What's the use case for this? What kinds of commands might you want to
only run sometimes? And when would you want to leave the decision up to the
user instead of automating the decision making (e.g. rewriting "fetch and
install app" to an idempotent operation like "if application is not
installed, fetch application from internet and install it").

At least in my use, my config has been idempotent. I think it's a good
idea to try to structure stuff like this, so that you can always safely run
./install without worrying about how it's going to interact with past
executions of ./install.

—
Reply to this email directly or view it on GitHub
#44 (comment).

[anishathalye]

Hmm, so if this is primarily used while editing the install config, I wonder if this is enough of a necessity to justify adding this feature?

[PacketPerception]

It's one of those cases where it's potentially a simple addition, with no change to default behavior, that might benefit a lot of people with the same use case. On the other hand, absolutely not necessary. Hence why I figured I'd open up discussion before just implementing it (figured it would take more than 5 minutes like the OrderedDict stuff did ;p)

[anishathalye]

"with no change to default behavior" -- that's a good point.

What's the best API for this? I don't really like -c/--clean, -l/--link, -s/--shell. I think the subcommand version is a bit better.

Maybe if we kind of combined the two and made a single general purpose flag so you could run something like --only link shell? We could use the nargs functionality in argparse.

[anishathalye]

Okay, I think if we do this, it'll be in Dotbot 2.0. So it has to fit nicely into that design. I'll add this issue to the "features under consideration" of #35.

[mainrs]

I was just looking into this too! I think it would be great to test out various steps in isolation. For example only testing that the linking works properly or that custom shell steps work as intended before going on.
A flag like --only would be good enough I guess.
How would it behave with multiple entries under the same name though. Like, having a shell, link, shell, link kind of layout. Would it execute every instance? Maybe adding a second argument, --only-depth for a starting at 1 and representing the number of directives it should execute in order until it reaches the maximum depth.

[anishathalye]

With --only, it would be filtering by the type of command. So if you have shell, link, shell, link, it'll execute every instance.

I think --only-depth is a complicated API. Is it needed?

[mainrs]

In which sense complicated? I didn't take a look at the source tbh but the first think that came into my mind was that dotbot does load the whole yaml at once right? So you only get the directives that match the --only. And then while executing each directive, have a counter increment. As long as the counter is smaller then --only-depth, it will continue to handle more entries of the same directive.

It can be avoided by just creating some kind of temporary install.conf.yaml file based on the params give to install for example. So install --only --only-depth=1 could all be handled in bash.

It just would make testing a little but easier on my site as I have split up my yaml into multiple files and cat them together so I can easily manipulate them during runtime.

[anishathalye]

Not in terms of implementation -- I think it's a confusing API design.

[mainrs]

ah got you. That was the first think that came into my mind. Maybe there are better/more approachable ways to achieve the same.

[mohkale]

Is this going to be merged sometime soon?

I've got a plugin I wrote to install packages, and I don't want to have to recheck whether their installed every time I want to update the symlinks for my dotfiles. At the moment I'm just planning to setup an environment variable which if defined makes the plugin skip all of this, but that's a fragile and far from ideal solution.

Have you thought about somehow exposing the argument parsing framework for the CLI to plugins? I'm not sure how one would go about it, but it would be very nice :).

[anishathalye]

There's no PR for this issue yet, so it's not just a matter of merging, someone needs to write the code. I think I might implement the --only / --except arguments tonight.

Have you thought about somehow exposing the argument parsing framework for the CLI to plugins?

This is not on the roadmap. I think most plugins should take arguments via defaults: or with per-data arguments (e.g. the per-link options that link supports). Exposing CLI arguments to plugins seems a bit complicated (also, it would be a shared namespace, which is also complicated).

[mohkale]

I see, thnx for the update 👍.

[anishathalye]

Feedback welcome in #214

[mohkale]

Yep, it appears to be working:

Some points though:

• Could you set the metavar for --only and --except to 'DIRECTIVE' or 'ACTION'. Just to make the help message a little nicer.
• Could you include only and except in the context passed to plugins? I've got a plugin which spawns dispatcher on config files in subdirectories, and as it stands there's no way to make those sub configs run only some or all except certain directives. (I'd actually prefer if the plugins handle method was passed the dispatcher instance so I didn't have to keep instantiating dispatcher (which then instantiates every plugin) but I understand if that's not possible).