Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Testthat reporting warnings that do not appear in code #958

Closed
jeroen opened this issue Nov 4, 2019 · 6 comments
Closed

Testthat reporting warnings that do not appear in code #958

jeroen opened this issue Nov 4, 2019 · 6 comments

Comments

@jeroen
Copy link
Member

jeroen commented Nov 4, 2019

Minimal reprex

from @gaborcsardi

test_that("x", {
  op <- options(warn = -1)  
  warning("not see this")
  options(op)
  expect_true(TRUE)
})

Real world case

If you run this in R, there is no warning:

out <- available.packages(repos = 'https://cloud.r-project.org', type = 'mac.binary.el-capitan')

However if you run this code anywhere in testthat:

Screen Shot 2019-11-04 at 8 44 45 PM

This warning should never appear because R sets options(warn = -1L) internally.

@gaborcsardi
Copy link
Member

gaborcsardi commented Nov 4, 2019

Even better reprex:

code <- quote(test_that("x", {
  op <- options(warn = -1)  
  warning("not see this")
  options(op)
  expect_true(TRUE)
}))

cat(deparse(code), sep = "\n", file = tmp <- tempfile())

testthat::test_file(tmp)
#> ✔ |  OK F W S | Context
#> |   0       | filebd63127caaed|   1   1   | filebd63127caaed
#> ───────────────────────────────────────────────────────────────────────────
#> filebd63127caaed:3: warning: x
#> not see this
#> ───────────────────────────────────────────────────────────────────────────
#> 
#> ══ Results ════════════════════════════════════════════════════════════════
#> Duration: 0.2 s
#> 
#> OK:       1
#> Failed:   0
#> Warnings: 1
#> Skipped:  0

Created on 2019-11-04 by the reprex package (v0.3.0)

@gaborcsardi
Copy link
Member

But there is no warning coming out here:

withCallingHandlers(
  testthat::test_that("x", {
    op <- options(warn = -1)  
    warning("not see this")
    options(op)
    testthat::expect_true(TRUE)
  }),
  warning = function(w) { print(w); browser() }
)

Created on 2019-11-04 by the reprex package (v0.3.0)

@hadley
Copy link
Member

hadley commented Nov 4, 2019

Probably needs special handling around here: https://github.com/r-lib/testthat/blob/windows-check/R/test-that.R#L120-L123

@gaborcsardi
Copy link
Member

gaborcsardi commented Nov 4, 2019

But how does the warning get to testthat at all? I guess I am missing the fine bits of the options(warn = -1) implementation...

@jeroen
Copy link
Member Author

jeroen commented Nov 4, 2019

I am guessing the default R condition handler for warnings checks with options(warn) on how to proceed, but testthat overrides this with a custom condition handler that doesn't do this?

jeroen added a commit that referenced this issue Nov 4, 2019
jeroen added a commit that referenced this issue Nov 4, 2019
@gaborcsardi
Copy link
Member

How can you define a default condition handler? I am clearly missing something about the condition system. :(

@hadley hadley closed this as completed in fa75bd9 Nov 5, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants