Running "standard" unix commands

[cloudhead]

I'm testing a tool that requires for example creating a git repository and cd'ing into it, something like:

mkdir foo
cd foo
git init .
my-cmd
...

It seems like it's possible to "register" some of these commands using register_bin, but some commands like cd are actual shell built-ins. What is the correct way to write this test in such a way that it can be read as realistic example as well?

[epage]

I've been considering allowing people to register functions as bins. My main intention was so users can have cross-platform behavior with their commands but we could extend the idea further by passing some state, like the CWD, to the function and allowing it to mutate the state.

[cloudhead]

Yeah, that would be helpful. Another issue is with the CWD, since I need to use a different tempdir for each test ideally, so they don't step on each other; and I don't think I can do that currently, unless I have one case per TestCases.

[epage]

A challenge with that is that we can only have one file system snapshot per trycmd/md file. I was also concerned about forcing people to put everything inside a single code fence.

This can be worked around by creating a directory for each code fence.

[epage]

Something that will be frustrating is if every test author has to write their own set of these functions.

It might be nice for us to create crates on top of trycmd that provide these and register them in bulk.

Ideas include

• trycmd-uutils
• trycmd-git
  • At minimum, quickly initializing repos, both a single-commit repo from a directory and integrating https://github.com/gitext-rs/git-fixture/
• trycmd-cargo
  • Run registries and easily populate them

One of the things I'm thinking about with this is answering the question "what would have prevented cargo-add from having to switch from trycmd to snapbox"

[cloudhead]

Yes, that's a good idea. Another way to handle it could be to have an "unknown command" handler. So eg. it might look like this:

test_cases.handle_unknown_command(|name, args| Command::new(name).args(args).wait())

The closure would be called whenever TryCmd can't find a command, and it would default to outputting an error or warning.

[epage]

I lean towards explicitly registering each one

• Less likely to make mistakes with overly broad ignores
• Easier to build up reusable pieces

[jpmckinney]

Maybe related to this idea, but I'd like to be able to show examples in my documentation that use standard input. My usual way to document this is something like:

$ echo "some simple input" | mycommand -
the output

So users can copy-paste it and get the same result (and then modify as needed). It would be great to be able to test that these examples work using trycmd.

In addition to this issue, I see #101 and #102, but all these issues are a bit more complicated than providing stdin (the simplest command being to echo).

[epage]

@jpmckinney could you create another issue for piping?

[jpmckinney]

Will do! Edit: #172

[figsoda]

We can also use something like duckscript as a cross-platform shell

[tv42]

register_bin / register_bins now exists. It, however, doesn't cover things quite well enough:

I'm trying to write a testable README for a command that processes files. I'd love to have file redirection work:

$ echo Hello, world >demo-file
$ my-command --frob demo-file
Frobnicated demo-file.

The easiest way might be to be able a variant of register_bin that runs the passed string (without shlex) via /bin/sh -c.

[tv42]

Since I really miss cram, I'm very tempted to make a PR that adds a "shell mode" where all command lines that don't look like binaries or environment variable setting are just passed to /bin/sh -c. Or, perhaps even better, add target/debug and target/release to PATH and just let the shell deal with. This would also solve echo foo | myapp (#172), and other variations like myapp <foo, myapp | grep bar.

The biggest speed bump is that currently the code eagerly parses the binary name out of the command line, whereas with shell mode you'd be better off letting commands remain as a string, and parse at the time of needing to execute something. There's not really a singular Bin for echo foo | myapp, or mycmd1 | mycmd2.

Would you be willing to entertain such a change? It could be a toggle that's disabled by default, to remain in the current registered-binaries-only world.

[tv42]

I've read through the trycmd code base and done an experimental spike on the code changes needed. I feel like supporting "real shell" scripts properly is too much at odds with the TOML-based test code in trycmd. I think "real shell" support would be better off living in a separate project.

[cloudhead]

FWIW I've moved to a custom testing framework built on snapbox directly. There were too many small things I needed that would have been complicated to add to trycmd.

[jpmckinney]

@cloudhead Can you share what you're doing with snapbox? snapbox is part of this repository, so it's still relevant :)

[cloudhead]

Sure: this is the alternative to trycmd that we built: https://github.com/radicle-dev/heartwood/blob/master/radicle-cli-test/src/lib.rs
And an example test file: https://github.com/radicle-dev/heartwood/blob/master/radicle-cli/examples/rad-patch-via-push.md?plain=1

It supports stuff like setting env vars, expecting failure etc., and is very simple.