Feature request: Run a single named test within a file

[dvg-p4]

Currently, the smallest unit of tests that can be run with this package is a file, with testthat::test_file(). However, for our use case each testing file (we've already split up into about 10 of them) takes 5-10 minutes to run, so the dev loop for tests at the end of a file is painfully slow. Therefore I propose something like testthat::test_single(file, test_name) to run a single named test within a file, skipping over all tests whose names do not match test_name.

[kevinushey]

This would also be useful for front-ends which might want to provide UI for running individual tests (either interactively or potentially separately in a new R process)

[hadley]

What should happen to other non-test code in the file. Should it get run?

What happens if there are multiple tests that happen to have the same name?

[kevinushey]

What should happen to other non-test code in the file. Should it get run?

I think so. The way I imagine it, the whole file is run in the "regular" way, but test_that() calls would be skipped if they aren't in the list of requested tests.

What happens if there are multiple tests that happen to have the same name?

I guess this could be made an error?

[hadley]

Some maybe something like this (untested):

library(rlang)

run_one_test <- function(path, name, env = parent.frame()) {
  
  exprs <- parse(path, keep.source = TRUE)
  run <- FALSE
  
  for (expr in exprs) {
    if (is_call(expr, "test_that", n = 2)) {
      if (!is_string(test_that[[2]]))
        next
    
      test_name <- as.character(test_that[[2]])
      if (test_name != name)
        next
      
      if (run) {
        cli::cli_abort("Multiple tests with specified name")
      }
      run <- TRUE
    }
    
    eval(expr, envir = env)
  }
  
  if (!run) {
    cli::cli_abort("Failed to find test with name")
  }

  invisible()
}

[kevinushey]

I think it'd be simpler to have test_file() accept some tests or labels argument, scope that to a binding like the$test_labels, and then have test_that() check against that the$test_labels when it decides whether or not it should be run. Error checking / counts could also happen there.

It also ensures that 'single' tests get run in the same way as a whole file, respecting all of the other setup that testthat does when running a test file in the regular way.

[hadley]

Yeah, sorry I wasn't suggesting this as the real implementation, just something that concretely expressed the logic so we could be sure we were both saying the same thing.