Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Testing against result stored in a file #148

Closed
jonclayden opened this issue Jun 10, 2014 · 6 comments
Closed

Testing against result stored in a file #148

jonclayden opened this issue Jun 10, 2014 · 6 comments

Comments

@jonclayden
Copy link
Contributor

@jonclayden jonclayden commented Jun 10, 2014

When the expected result of a test is relatively complex (in the case of the package I'm currently working on, a numeric array), it can be awkward to write it as a literal within an expect_that call. It would be convenient to store such results in a file instead and have an expectation that checks against that file. To avoid code duplication, it would also be helpful if that same test code would be used to create the file if it doesn't yet exist.

I knocked up the following as a proof of principle.

matches_file <- function (file, compare = equals, label = NULL, ...)
{
    if (file.exists(file)) {
        reference <- readRDS(file)
        compare <- match.fun(compare)
        if (is.null(label))
            label <- paste("reference from", file)
        compare(reference, label=label, ...)
    } else {
        return (function(actual) {
            saveRDS(actual, file)
            expectation(TRUE, "should never fail", "saved to file")
        })
    }
}

This is then called like

expect_that(shapeKernel(c(5,5),type="box"), matches_file("box_kernel.rds"))

I'm happy to put this, or a modification, in a pull request if it would be helpful. The only issue I can see with this as it stands is that you need to check by hand what goes into the files the first time. Once that's verified then the tests will ensure the correct result on future runs. Comments?

@rmflight
Copy link

@rmflight rmflight commented Jun 10, 2014

Curious, why not save the result, and merely load it prior to doing the new calculation, and then do the comparison? Although it takes a couple extra lines in the test suite, it is more explicit in my mind.

Loading

@gaborcsardi
Copy link
Contributor

@gaborcsardi gaborcsardi commented Jun 10, 2014

What I usually do in these cases, is calculating an MD5 (or other) hash of the object using the digest package. Then you can just compare the hashes. Saves you from having to store the files.

Loading

@jonclayden
Copy link
Contributor Author

@jonclayden jonclayden commented Jun 10, 2014

@rmflight The difference is in the generation of the reference output in the first place. This ties the two together to avoid code duplication. Otherwise, how should one systematically generate reference output that matches the tests?
@gaborcsardi Fair enough, but that limits the information you can get from failures.

Loading

@hadley
Copy link
Member

@hadley hadley commented Jun 13, 2014

See also #38

Loading

@jonclayden
Copy link
Contributor Author

@jonclayden jonclayden commented Jun 14, 2014

Sure, I saw that, but it seemed like that was a much more extensive change. Do you regard this out of scope as well, then? If so I can always just define this expectation for my tests, but I thought it was pretty simple and might be useful for others...

Loading

@hadley
Copy link
Member

@hadley hadley commented Sep 17, 2014

I'd be willing to review a pull request that implemented this. Maybe equals_reference()?

Loading

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
4 participants