Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Testthat reporting warnings that do not appear in code #958

Closed
jeroen opened this issue Nov 4, 2019 · 6 comments

Comments

@jeroen
Copy link
Member

@jeroen jeroen commented Nov 4, 2019

Minimal reprex

from @gaborcsardi

test_that("x", {
  op <- options(warn = -1)  
  warning("not see this")
  options(op)
  expect_true(TRUE)
})

Real world case

If you run this in R, there is no warning:

out <- available.packages(repos = 'https://cloud.r-project.org', type = 'mac.binary.el-capitan')

However if you run this code anywhere in testthat:

Screen Shot 2019-11-04 at 8 44 45 PM

This warning should never appear because R sets options(warn = -1L) internally.

@gaborcsardi

This comment has been minimized.

Copy link
Member

@gaborcsardi gaborcsardi commented Nov 4, 2019

Even better reprex:

code <- quote(test_that("x", {
  op <- options(warn = -1)  
  warning("not see this")
  options(op)
  expect_true(TRUE)
}))

cat(deparse(code), sep = "\n", file = tmp <- tempfile())

testthat::test_file(tmp)
#> ✔ |  OK F W S | Context
#> |   0       | filebd63127caaed|   1   1   | filebd63127caaed
#> ───────────────────────────────────────────────────────────────────────────
#> filebd63127caaed:3: warning: x
#> not see this
#> ───────────────────────────────────────────────────────────────────────────
#> 
#> ══ Results ════════════════════════════════════════════════════════════════
#> Duration: 0.2 s
#> 
#> OK:       1
#> Failed:   0
#> Warnings: 1
#> Skipped:  0

Created on 2019-11-04 by the reprex package (v0.3.0)

@gaborcsardi

This comment has been minimized.

Copy link
Member

@gaborcsardi gaborcsardi commented Nov 4, 2019

But there is no warning coming out here:

withCallingHandlers(
  testthat::test_that("x", {
    op <- options(warn = -1)  
    warning("not see this")
    options(op)
    testthat::expect_true(TRUE)
  }),
  warning = function(w) { print(w); browser() }
)

Created on 2019-11-04 by the reprex package (v0.3.0)

@hadley

This comment has been minimized.

Copy link
Member

@hadley hadley commented Nov 4, 2019

@gaborcsardi

This comment has been minimized.

Copy link
Member

@gaborcsardi gaborcsardi commented Nov 4, 2019

But how does the warning get to testthat at all? I guess I am missing the fine bits of the options(warn = -1) implementation...

@jeroen

This comment has been minimized.

Copy link
Member Author

@jeroen jeroen commented Nov 4, 2019

I am guessing the default R condition handler for warnings checks with options(warn) on how to proceed, but testthat overrides this with a custom condition handler that doesn't do this?

jeroen added a commit that referenced this issue Nov 4, 2019
jeroen added a commit that referenced this issue Nov 4, 2019
@gaborcsardi

This comment has been minimized.

Copy link
Member

@gaborcsardi gaborcsardi commented Nov 5, 2019

How can you define a default condition handler? I am clearly missing something about the condition system. :(

@hadley hadley closed this in fa75bd9 Nov 5, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
3 participants
You can’t perform that action at this time.