Support multiple --exec <cmd> instances

[terlar]

Sometimes I run a command in several different matching directories with --exec, but I would like to print each directory for each command that is executed without having to add an intermediate shell just to print the path.

Perhaps adding an option that would print the matched file-name as well as executing the command in --exec. I tried with multiple --exec but it is not allowed, the only workaround I found was to have an intermediate shell.

Here is an example (print the git-status in all matching git directories:

fd -HFtd .git -x git -C '{//}' status -s

I can imagine that there are other use cases where you would want to print the matched file you execute the -x command for.

[sharkdp]

Thank you very much for the feedback.

I definitely see how that could be useful, yes. Maybe supporting multiple --execs would be a good idea in general. To support this use case here, users could simply use -x echo -x whatever.

Edit: Unfortunately, users would need to use -x echo \; -x whatever. Otherwise, this would be parsed as a single command (echo -x whatever)

[josephgimenez]

I actually ran into this while testing fd today. I love the utility btw!

As a workaround for right now, I noticed you can have -x call the bash interpreter where then you can use multiple {} placeholders like:

$> fd -e rs -x bash -c "grep main {} && echo {}"

fn main() {
main.rs

[tavianator]

@asyncsrc Note that that will break if you have filenames with shell punctuation like apostrophes. This is safe though:

fd -e rs -x bash -c 'grep main "$1" && echo "$1"' bash {}

[sasacocic]

Any progress on this? This would be really useful I think.

[sharkdp]

I tried to look into this today, but I think we currently run into a limitation of clap (clap-rs/clap#1026) which prevents us from using two instances of --exec/-x.

[sudonym1]

I looked into clap with an eye on adding this feature, and was surprised by the overall size of the project. While I agree that it would be better if clap just did what you needed, there is enough info in the MatchedArg to rebuild multiple commands:

let exec_args = matches.args.get("exec").unwrap();
println!("{:?}", exec_args);

let mut last = exec_args.indices[0] - 1;
for it in exec_args.vals.iter().zip(exec_args.indices.iter()) {
    let (arg, idx) = it;
    if *idx == last + 1 {
        print!("{} ", arg.to_string_lossy());
    } else {
        print!(", {} ", arg.to_string_lossy());
    }
    last = *idx;
}

Of course all of the printing, unwrapping, and lossy utf-8'ing are for demo purposes only. If you are ok with this general approach, and ok helping out a rust newb, I can take a stab at adding this feature.

[sharkdp]

I tried your snippet, but I don't understand how that would help:

fd --exec echo 'running command for file' --exec rm -i`
MatchedArg { occurs: 1, indices: [2, 3, 4, 5, 6], vals: ["echo", "running command for file", "--exec", "rm", "-i"] }
echo running command for file --exec rm -i

[sudonym1]

The command line you specified doesn't include ';' to terminate the first exec instance. There is no gap in the indices because clap never terminates the first --exec. If you had instead specified fd --exec echo 'running command for file' ';' --exec rm -i, then the indices would have been [2, 3, 6, 7], and the separate exec commands could have been detected by the gap between 3 and 6.

That said, I am not sure that always requiring the ';' is good UI. In fact having to escape a semicolon is one of the issues I had with gnu find as a novice. I have some thoughts here that might be worth going into in a separate issue.

[sharkdp]

The command line you specified doesn't include ';' to terminate the first exec instance.

oh, I see. Yeah, not a fan of the semicolon thing either. Especially since it has to be escaped - completely agreed.

In fd, I actually never use it and just put the --exec last. Most of the time, {} can also be skipped.

You might be right though. We might end up requiring the semicolon. Otherwise, --exec or -x could be an option to the first command.

I have some thoughts here that might be worth going into in a separate issue.

Please do.

[sudonym1]

Are you ok with this general approach for supporting multiple commands? I will enter a separate issue RE semicolons.

[sharkdp]

Are you ok with this general approach for supporting multiple commands?

Really depends on how "hacky" this would get. I'd rather wait for clap to fix this than have a solution that adds a lot of code and could possibly result in future bug reports. For example, even if unlikely to be used that way, we would want to support something like --exec cmd1 --opt1a --opt1b \; --some-other-fd-option --exec cmd2 --opt2a --opt2b.

[sudonym1]

I don't think it will get too messy. For example in the command line you give, the exec indices would be something like [4, 5, 6, 10, 11, 12] and the associated values would be ["cmd1", "--opt1a", "--opt1b", "cmd2", "--opt2a", "--opt2b"]. The gap between 6 and 10 would still correctly indicate the separate exec commands.

FWIW, I went through the issue in the clap project, and looked at some possible solutions on that side as well. There isn't really a minor change that can be done which will result in a sensible api. Currently clap is treating multiple == true flags sort of like positional arguments. Either there would have to be some new option to replace multiple or the MatchedArg struct would need to be extended to include the indices of the start of each found flag. I think a replacement for the multiple option provides a better API, but it is the larger change.

How about this, I will implement just the argument handling as I described here and submit a PR. If you like it, I will move forward, if not I will go make a PR for clap. Sound good?

[sharkdp]

How about this, I will implement just the argument handling as I described here and submit a PR. If you like it, I will move forward, if not I will go make a PR for clap. Sound good?

yes - thank you!

[indigoviolet]

@sharkdp Does it make sense to support just printing the filename to handle the common use case of echoing the file (ie the -print option of find), since the general solution of multiple --execs seems to be stymied? Would it be useful to open a separate issue for that?

[terlar]

I think that would make sense, that is proposed in the original issue as well, but I created this with multiple exec as I thought that would cover more use cases, but after a lot of discussion it turns out that is a quite complex beast in terms of interface.

I originally didn’t see the need if this was possible:

fd search -x echo -x dostuff 

But the problem I set out to solve was the print case. However since this issue is on the topic of multiple exec it shouldn’t be closed if print is added. So if we want to track this in an issue that can be closed, I think a new one should be created.

[sharkdp]

I tried to look into this today, but I think we currently run into a limitation of clap (clap-rs/clap#1026) which prevents us from using two instances of --exec/-x.

The upstream issue has actually been fixed. Looks like this going to be part of a clap (3, probably?) release soon.

[sharkdp]

In the meantime, clap 3 has been released. This should make it possible to implement that feature.

[Phantop]

A small question with this. Is it possible to use && or something equivalent? A lot of the time I want to process the files in some way and then remove the originals, but only if the processing succeeds, so having that degree of control is rather important.

[tmccombs]

I believe it actually always behavea like &&. If a command fails the rest of the commands aren't executed.