Dynamically assign variable to object to use both in setup and in unit tests

[alessap]

Dear All,

In our CI pipeline we currently test that an API (built with plumber) can be succesfully run in a sub process and run the tests defined by the developer. We follow the same framework as in plumbpkg: https://github.com/sol-eng/plumbpkg/blob/master/tests/testthat/test-plumber.R

My question is how can we set the port dynamically? One needs first to make sure that the port on the server running the CI tests is available and then use the same value in the tests. How can this be achieved? Here below is an example of tests that succeed if the port variable is explicitely written, but the following code where port is defined as a global variable fails:

port <<- 1879 # or a function that finds an available port
alive_url <<- glue::glue("http://localhost:{port}/alive")

testthat::setup({
  process <<- callr::r_bg(
    function() {
      r <- plumber::plumb(file = system.file("plumber", "api1", "plumber.R", package = "mycustom.api"))
      r$run(host = "0.0.0.0", port = port, swagger = FALSE)
    }
  )
})

Sys.sleep(5)

testthat::teardown({
  process$kill()
})

testthat::test_that("Process is alive", {
  testthat::expect_true(process$is_alive())
})

test_that("alive_url returns 200", {
  r <- httr::GET(alive_url)
  expect_equal(r$status_code, 200)
})

Output here:

── 1. Failure: Process is alive (@test-plumber.R#32)  ────────────────────
process$is_alive() isn't true.

── 2. Error: alive_url returns 200 (@test-plumber.R#38)  ─────────────────
Failed connect to localhost:1879; Connection refused
Backtrace:
 1. httr::GET(alive_url)
 2. httr:::request_perform(req, hu$handle$handle)
 4. httr:::request_fetch.write_memory(req$output, req$url, handle)
 5. curl::curl_fetch_memory(url, handle = handle)

══ testthat results  ═════════════════════════════════════════════════════
[ OK: 1 | SKIPPED: 0 | WARNINGS: 0 | FAILED: 2 ]
1. Failure: Process is alive (@test-plumber.R#32) 
2. Error: alive_url returns 200 (@test-plumber.R#38) 

There might be something I misunderstand about testthat::setup as it seems that global variable set outside it are not accessible. Can anyone help with this issue?

Best regards,
Alessandro

[alessap]

I have just found this issue #422 (believe me I have looked at length before I have created this issue) that points to use helpers-* files in testthat. I will try this and see if it solves my problem.

[gaborcsardi]

You cannot refer to a variable in the main process, from a function that runs in the sub-process, you need to pass it as an argument. Then callr will copy it to the subprocess. I.e.:

port <- 1879 # or a function that finds an available port
alive_url <- glue::glue("http://localhost:{port}/alive")
process <- NULL

testthat::setup({
  process <<- callr::r_bg(
    function(port) {
      r <- plumber::plumb(file = system.file("plumber", "api1", "plumber.R", package = "mycustom.api"))
      r$run(host = "0.0.0.0", port = port, swagger = FALSE)
    },
    args = list(port = port)
  )
})

Btw. <<- will set objects in the global environment, that's probably not what you want for port, etc. You only want to set them for one test file, I assume.
Btw.2. binding to "0.0.0.0" might not be a good idea, then the app is reachable from the internet.

[hadley]

You should probably also read https://testthat.r-lib.org/articles/test-fixtures.html, which has our latest advice on setup()/teardown().