Consider suppressing deprecation warnings in expect_warning(, NA)

[hadley]

Since this is causing revdep failures

[hadley]

And in expect_silent()

[nealrichardson]

Maybe suppress only on CRAN?

[hadley]

I don't think we need to because the deprecations will still bubble up in other tests (just not the tests where you're asserting there are no warnings). But I think we'd probably couple this with #1679, where we'd start to steer people away from saying there are no warnings whatsoever in favour of asserting there are no warnings of a specific type/message.

[hadley]

Hmmmm, but there are advantages to just suppressing the warnings on CRAN: then we're guaranteeing that a deprecation will never break R CMD check. But then your local checks will return different results to CRAN, which we try to avoid as much as possible (snapshot tests not withstanding).

[nealrichardson]

But then your local checks will return different results to CRAN

To me this feels more like a variation of skip_on_cran(), where I have checks that I want to be alerted about early but don't want to cause my package to get (r)ejected, though I understand your concern.

[hadley]

We've determined this is actually but lifecycle's responsibility: r-lib/lifecycle#134

[hadley]

We've now decided this is not lifecycle's responsibility: r-lib/lifecycle#144

[hadley]

We've decided that it's simpler for lifecycle to be consistent, and generate deprecations regardless of where the code is run. So this means that testthat has to be a bit smarter, because we don't want a test like this:

test_that("no warnings", {
  expect_warning(foo(), NA)
}

to fail just because foo() calls a function that is now deprecated.

The deprecation warnings are no longer captured by the catch all expectation like expect_warning(code, NA) and expect_no_warning(code) ignore deprecation warnings so that they bubble up and appear in your tests. (They will still be included in snapshots.) This means that deprecation won't cause your package to break in CI or on CRAN, but when you're next working on it interactively, you'll be compelled to eliminate the warnings.

In most cases, we'll use soft deprecation in tidyverse packages which mean that users of your package won't see the warnings either. So all up, only developers who are actively working on the package will see them. This might be slightly too little pain (since if you're not actively developing your package, you might never see the deprecations), but I think we can address that problem separately, possibly by some additional tooling that lets us alert package developers via Github.