Run multiple cases in a toml file

[epage]

*.trycmd files support this but not *.toml. This discrepancy can push someone to *.trycmd files when they might be best served by the extra control of *.toml files.

Open Issues

• How do we handle expected stdout / stderr?
  • We currently read these from the file system which makes them global to the *.toml file.
  • If we add support for inline stdout, stderr, keep in mind
    • binary support
    • TRYCMD=overwrite support, see TRYCMD=overwrite support for *.trycmd files #23 (we are already using a format-preserving TOML library)

[pwinckles]

Couldn't you model it something like this, combing a few of the test cases in this project as an example:

bin.name = "bin-fixture"

[basic]

[sandbox]
fs.sandbox = true

[sandbox.env.add]
write = "file.txt = Goodbye\nWorld"
cat = "file.txt"

[stderr-to-stdout]
stderr-to-stdout = true

[stderr-to-stdout.env.add]
stdout = """
Hello
World!
"""
stderr = """
Goodnight
Moon!
"""

I haven't tried deserializing that, but I would expect that it'd look something like:

struct TomlFile {
    global: OneShot,
    cases: HashMap<String, OneShot>,
}

It might be a better idea to put the test cases in an array of tables though, [[cases]].

So, if there are no cases are defined the current behavior is used. If cases are defined, then the OneShot objects are resolved by defaulting their values to whatever is in the global context, applying any values set in the case, and using the case's name as the name of the case (that is to say, if the case basic is in the file tests.toml, then it will look for basic.in, basic.out, etc).

[epage]

Since this issue was created, we've added inline stdout/stderr support (just too many disparate files to juggle in large cases like cargo-edit).

Now its just a matter of moving forward with implementing this. We'd need to use an Array of Tables for the cases since they are order-dependent. We'd also want the fs table to be global since the assumption is the fs is shared between runs.

So the data model would be to just expose the TryCmd struct. I think the only remaining question is if we should actually do:

pub enum TrycmdSpec {
    OneShot(OneShot),
    MultiStep(TryCmd),
}

I'm assuming so.

So after that, its just getting around to implementing this.