-
Notifications
You must be signed in to change notification settings - Fork 315
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Vignette on using testthat outside the context of a package #659
Comments
I think our current advice is to make a package. In the near future we might be able to recommend making an analysis package instead. Either way, it's just matter of running a helper function and then putting your R scripts in |
@jennybc This is potentially related to the approach of testrmd? I think having just a |
Coming from industry I feel there is place for tests in a poject which is more complex than an rmd file but not a package. Although I certainly should make more packages I do not think everything should be a package. Although I know that technically you need very little for a project to be a package for me a package has clear purpose, more limited scope than a bigger research/analysis project and main intended use is after installation. On the other hand the quality assurance brought by testing is needed very early on. Some projects will be abandoned before reaching a package phase others could be split into several small packages but it is not clear from the beginning. I also don't necessarily want to think about what functions to export, when to increase versions etc. Regarding advice: imo most of it applies equally to packages and not packages:
This last point what I missed guidance about the most, how to process test summary results automatically, like for a CI: single function call to determine whether all tests has passed. Do you think such thing has a place in testthat? I am curious about the mentioned concept of an analysis package, is there any resource I can read? |
For context, @hadley, there was a conversation in rOpenSci Slack re: how to use testthat outside of a package and I encouraged the participants to weigh in here. |
I would also echo the desire to use testthat outside of a package context. Many of my analyses are not in packages (unnecessary overhead) but could greatly benefit from testing. |
Re testing for 'analysis packages' and non-packages, there's so much tooling around R packages that I think using that approach actually reduces overhead. For instance, r-travis will automatically install dependencies if it sees them listed in a DESCRIPTION file. For example, here's a template I use to have travis test that any .Rmd files in a student's repo can be knit successfully: https://github.com/cboettig/compendium . |
I think it's suboptimal to think about packages as adding overhead - they add conventions, which with appropriate tooling, reduces overhead. At some level, all a package means is to put your R code in a directory called |
One major hassle is that subdirectories of Sourcing the files manually works for both |
My task is to assemble an environment with many R packages from Anaconda, and some from CRAN. I would like to run tests that check for the presence of certain packages, and later maybe some very basic tests that selected packages work well together. That's to detect when something is dropped accidentally or some incompatibility is introduced through package updates in the future. I have that setup established for Python, now I want to achieve the same for R. But being new to R, I don't even know how to run testthat tests without devtools or RStudio. There is no code of my own to put in a package... do I have to create an empty dummy package, just to put the tests in there? |
As another data point for this, my team has a situation where we have a small git repository that uses R with plumber to create an API endpoint that runs a R keras model. Our team is used to doing testing for Java and we need to figure out how to get the automatic testing to work with R. testthat seems like a good solution, however turning the repository into an R package feels like a lot of overhead. This is especially true with all the non-R stuff that has to live in the repo (the dockerfile, the jenkinsfile, the linux stuff to get plumber to work with https etc). |
@jnolis: It's actually quite simple to use the API, once you know how to do it: library('testthat')
testthat::test_dir('tests/testthat/') This runs all the tests in directory I'm calling an R script Rscript --verbose run-tests.r Hope this helps. |
I was looking for information on this because I have an analysis script which is getting complicated and has the potential to keep growing. I thought that this was a good first step to ensuring reproducible results and making me be purposeful in the file structure. I found a nice blog on this ( https://katherinemwood.github.io/post/testthat/) which I took as a guide and implemented but I think it would be nice to have a recommended "light touch" framework. Why I'm not making a package now:
|
I'm going to close this issue, because I am not convinced that creating a package "adds overhead". All it requires is that you put your R files in one directory, your test files in another, and have a file that describes what packages are needed to run your code. Nevertheless, if you believe strongly that a package is too much overhead, I think it is still possible to use testthat by calling |
It's definitely possible to use testthat by calling |
To build on that, things like having helper functions that have to live in specific folders to be loaded by test_file may be unintuitive for people who aren't used to making packages (like me). Right now all our my team's docker containers have a test.R file that sources some scripts, creates some helper functions, then runs tests all within the file. If the tests fail the build stops. It seems like I should be able to do something more elegant by using a function like test_file, but it's hard for me to tell what the trade offs are, or if it's even possible without a large restructuring of a code base (that still needs to work with docker). If I'm having a lot of trouble trying to understand this, I suspect there are lots of other people who would benefit too! |
I found this useful for setting up unit testing for an analysis project library(usethis)
library(testthat)
# Create test files and directories
use_directory("./tests/")
file_create("./tests/test-my-analysis-1.R")
file_create("./tests/test-my-analysis-2.R")
# run a test file
test_file("./tests/test-my-analysis-1.R")
# run all test files in a directory (filenames must start with "test-", otherwise it won't work)
test_dir("./tests/") |
I've had success running Should As an aside, I use |
I taught "writing functions" today and showed a bit of testthat. Which overalls feels like a good thing.
But I feel there's little guidance on how to use testthat on functions that are not (yet) in a package.
Do you make a
.R
file that sources the target functions and proceeds to execute lots of expectations, possibly gathered into tests? How do you trigger testing? It's tempting to tell someone that their need to do this suggests the function and its tests should be in a package. Once you've given that advice though, you still need to do something today. But exactly what?Another use case extracted from a student question: An organization has lots of existing scripts. Not necessarily using functions, not necessarily very modular, etc etc. It is what it is. How can you use testthat to start getting more control and visibility into the output of these scripts? Especially, is the output changing as a result of people tinkering with the scripts or upgrading R or upgrading packages? I suspect there's a workflow here using reference objects and reference files.
The text was updated successfully, but these errors were encountered: