Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Testing against result stored in a file #148

Closed
jonclayden opened this Issue Jun 10, 2014 · 6 comments

Comments

Projects
None yet
4 participants
@jonclayden
Copy link
Contributor

jonclayden commented Jun 10, 2014

When the expected result of a test is relatively complex (in the case of the package I'm currently working on, a numeric array), it can be awkward to write it as a literal within an expect_that call. It would be convenient to store such results in a file instead and have an expectation that checks against that file. To avoid code duplication, it would also be helpful if that same test code would be used to create the file if it doesn't yet exist.

I knocked up the following as a proof of principle.

matches_file <- function (file, compare = equals, label = NULL, ...)
{
    if (file.exists(file)) {
        reference <- readRDS(file)
        compare <- match.fun(compare)
        if (is.null(label))
            label <- paste("reference from", file)
        compare(reference, label=label, ...)
    } else {
        return (function(actual) {
            saveRDS(actual, file)
            expectation(TRUE, "should never fail", "saved to file")
        })
    }
}

This is then called like

expect_that(shapeKernel(c(5,5),type="box"), matches_file("box_kernel.rds"))

I'm happy to put this, or a modification, in a pull request if it would be helpful. The only issue I can see with this as it stands is that you need to check by hand what goes into the files the first time. Once that's verified then the tests will ensure the correct result on future runs. Comments?

@rmflight

This comment has been minimized.

Copy link

rmflight commented Jun 10, 2014

Curious, why not save the result, and merely load it prior to doing the new calculation, and then do the comparison? Although it takes a couple extra lines in the test suite, it is more explicit in my mind.

@gaborcsardi

This comment has been minimized.

Copy link
Member

gaborcsardi commented Jun 10, 2014

What I usually do in these cases, is calculating an MD5 (or other) hash of the object using the digest package. Then you can just compare the hashes. Saves you from having to store the files.

@jonclayden

This comment has been minimized.

Copy link
Contributor Author

jonclayden commented Jun 10, 2014

@rmflight The difference is in the generation of the reference output in the first place. This ties the two together to avoid code duplication. Otherwise, how should one systematically generate reference output that matches the tests?
@gaborcsardi Fair enough, but that limits the information you can get from failures.

@hadley

This comment has been minimized.

Copy link
Member

hadley commented Jun 13, 2014

See also #38

@jonclayden

This comment has been minimized.

Copy link
Contributor Author

jonclayden commented Jun 14, 2014

Sure, I saw that, but it seemed like that was a much more extensive change. Do you regard this out of scope as well, then? If so I can always just define this expectation for my tests, but I thought it was pretty simple and might be useful for others...

@hadley

This comment has been minimized.

Copy link
Member

hadley commented Sep 17, 2014

I'd be willing to review a pull request that implemented this. Maybe equals_reference()?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.