Testing against result stored in a file

[jonclayden]

When the expected result of a test is relatively complex (in the case of the package I'm currently working on, a numeric array), it can be awkward to write it as a literal within an expect_that call. It would be convenient to store such results in a file instead and have an expectation that checks against that file. To avoid code duplication, it would also be helpful if that same test code would be used to create the file if it doesn't yet exist.

I knocked up the following as a proof of principle.

matches_file <- function (file, compare = equals, label = NULL, ...)
{
    if (file.exists(file)) {
        reference <- readRDS(file)
        compare <- match.fun(compare)
        if (is.null(label))
            label <- paste("reference from", file)
        compare(reference, label=label, ...)
    } else {
        return (function(actual) {
            saveRDS(actual, file)
            expectation(TRUE, "should never fail", "saved to file")
        })
    }
}

This is then called like

expect_that(shapeKernel(c(5,5),type="box"), matches_file("box_kernel.rds"))

I'm happy to put this, or a modification, in a pull request if it would be helpful. The only issue I can see with this as it stands is that you need to check by hand what goes into the files the first time. Once that's verified then the tests will ensure the correct result on future runs. Comments?

[rmflight]

Curious, why not save the result, and merely load it prior to doing the new calculation, and then do the comparison? Although it takes a couple extra lines in the test suite, it is more explicit in my mind.

[gaborcsardi]

What I usually do in these cases, is calculating an MD5 (or other) hash of the object using the digest package. Then you can just compare the hashes. Saves you from having to store the files.

[jonclayden]

@rmflight The difference is in the generation of the reference output in the first place. This ties the two together to avoid code duplication. Otherwise, how should one systematically generate reference output that matches the tests?
@gaborcsardi Fair enough, but that limits the information you can get from failures.

[hadley]

See also #38

[jonclayden]

Sure, I saw that, but it seemed like that was a much more extensive change. Do you regard this out of scope as well, then? If so I can always just define this expectation for my tests, but I thought it was pretty simple and might be useful for others...

[hadley]

I'd be willing to review a pull request that implemented this. Maybe equals_reference()?