Print progress only if enough time has passed. #701
Conversation
| ⠹ | 1 2 | Expectations | ||
| ⠸ | 1 3 | Expectations | ||
| ⠼ | 1 4 | Expectations | ||
| ⠴ | 2 4 | Expectations | ||
| ✖ | 2 4 | Expectations |
There was a problem hiding this comment.
Do you think it is possible to keep these lines? Because otherwise the tests are not really testing the output. Maybe by setting min_time?
Timing based tests are not too great in general, but of course in this case we can't really avoid them. We can have another set of (the same?) tests that set min_time to something very large, so that the output is omitted on very slow systems as well.
R/reporter-progress.R
Outdated
| # Do not print if not enough time has passed since we last printed. | ||
| if ((time - self$last_time)[[3]] < self$min_time) { | ||
| return() | ||
| } |
There was a problem hiding this comment.
I would always print a line at the beginning of a new context/file, so that you at least know which test file is running. I.e. this:
⠏ | 0 | Expectations
Otherwise, if the first test case takes very long, you'll have no output from the file for a long time.
jimhester
force-pushed
the
delay_printing
branch
2 times, most recently
from
ffcfcab
to
bf0ab72
Compare
|
Ok I changed how this was implemented slightly, there is now a separate |
|
It also now should always update when there is a new context. |
R/reporter-progress.R
Outdated
| @@ -37,11 +39,13 @@ ProgressReporter <- R6::R6Class("ProgressReporter", | |||
| initialize = function(show_praise = TRUE, | |||
| max_failures = getOption("testthat.progress.max_fails", 10L), | |||
| min_time = 0.1, | |||
| update_interval = 0, | |||
There was a problem hiding this comment.
Should the default be 0.1 here? Otherwise it is never throttled.
There was a problem hiding this comment.
Yeah, I changed the wrong value, thanks!
R/reporter-progress.R
Outdated
| @@ -18,6 +18,8 @@ ProgressReporter <- R6::R6Class("ProgressReporter", | |||
| show_praise = TRUE, | |||
| min_time = 0.1, | |||
| start_time = NULL, | |||
| last_update = NULL, | |||
| update_interval = 0.1, | |||
There was a problem hiding this comment.
It is a small thing, but I would set this to NULL here. It is set by initialize anyway, and then this magic number is misleading.
jimhester
force-pushed
the
delay_printing
branch
from
bf0ab72
to
a3425a3
Compare
|
Looks good, but the CI is failing. :( |
jimhester
force-pushed
the
delay_printing
branch
from
a3425a3
to
e8c1fbe
Compare
|
I missed setting |
R/reporter-progress.R
Outdated
| @@ -78,13 +82,19 @@ ProgressReporter <- R6::R6Class("ProgressReporter", | |||
| }, | |||
|
|
|||
| show_status = function(complete = FALSE) { | |||
| time <- proc.time() | |||
There was a problem hiding this comment.
No it is vestigial, thanks for catching it.
R/reporter-progress.R
Outdated
|
|
||
| time <- proc.time() | ||
| if (!is.null(self$last_update) && | ||
| (time - self$last_update)[[3]] < self$update_interval) { |
There was a problem hiding this comment.
Maybe just save proc.time()[[3]] to make a little simpler?
jimhester
force-pushed
the
delay_printing
branch
from
e8c1fbe
to
8b11d9e
Compare
The following WARNING currently happens to any package built on R-devel with vignettes > checking serialized R objects in the sources ... WARNING > Found file(s) with version 3 serialization: > ‘build/vignette.rds’ > Such files are only readable in R >= 3.5.0. > Recreate them with R < 3.5.0 or save(version = 2) or saveRDS(version = > 2) as appropriate
[ci-skip]
|
Thank you both for the reviews! |
Currently progress is printed after every test, which can cause a significant portion of the total test time taken to be used by printing if each test runs quickly. You also get a distracting flashing cursor when the updates occur too frequently.
As a real world example r-lib/fs@master...sanitize adds a bunch of tests which happen very fast, the total time taken for all tests in the fs package with the current master of testthat is ~5.973 seconds. With this PR the total time drops to is 4.254 seconds, a considerable change considering it is just test printing overhead.
This PR currently reuses the min_time parameter, which seems to work ok, 1/10 of a second is a reasonable amount of time between updates. But if necessary we could add an additional option or hardcode it to some other value.