Prevent error when all failed autotest checks were of type "no_test"

[simpar1471]

The issue

In autotest_single_yaml(), there is a mechanism which checks whether failed tests were meant to be run. This looked like this:

# ... other code
if (test)
    reports <- reports [which (!reports$type == "no_test"), ]
# ... other code

This is great when some or all elements in !reports$type == "no_test" are TRUE - the reports data frame is cut down to not include any tests which failed but weren't meant to be tested anyway.

In situations where all values in !reports$type == "no_test" are FALSE, the subsetting procedure makes a data frame with 0 rows and 10 columns. This empty data frame causes an error to be thrown later in autotest's order_at_rows function:

order_at_rows <- function (x) {
    type_order <- c ("error", "warning", "diagnostic", "message", "dummy", "no_test")
    # ERROR HAPPENS HERE!
    index <- data.frame(index = seq (nrow (x)), type = match (x$type, type_order))
    index <- index[order (index$type), ]
    x <- x[index$index, ]
    rownames(x) <- NULL
    return(x)
}

The fix

To prevent this error I've added some logic which, in autotest_single_yaml(), sets reports back to NULL if all(!reports$type == "no_test") is TRUE.

# ... other code
if (test) {
  no_test <- reports$type == "no_test"
  if (all(no_test)) {
    reports <- NULL
  } else {
    reports <- reports[which(!no_test), ]
    rownames(reports) <- NULL
    }
  }
# ... other code

Checks on local machine

I don't think I've broken anything that wasn't leading to warnings already.

devtools::test()

#> i Testing autotest
#> v | F W  S  OK | Context
#> v |         12 | autotest object
#> v |          5 | local source package [5.8s]
#> v |   2     30 | stats-package [120.5s]
#> ---------------------------------------------------------------------------------
#> Warning (test-statspkg.R:99:5): autotest rnorm
#> NaNs produced
#> Backtrace:
#>      x
#>   1. +-testthat::expect_message(...) at test-statspkg.R:99:5
#>   2. | \-testthat:::quasi_capture(enquo(object), label, capture_messages)
#>   3. |   +-testthat (local) .capture(...)
#>   4. |   | \-base::withCallingHandlers(...)
#>   5. |   \-rlang::eval_bare(quo_get_expr(.quo), quo_get_env(.quo))
#>   6. \-autotest::autotest_package(...)
#>   7.   +-base::rbind(...) at autotest/R/autotest-functions.R:259:9
#>   8.   | \-base::rbind(deparse.level, ...)
#>   9.   \-autotest::autotest_yaml(...) at autotest/R/autotest-functions.R:259:9
#>  10.     \-base::lapply(...) at autotest/R/autotest-functions.R:59:5
#>  11.       \-autotest (local) FUN(X[[i]], ...)
#>  12.         \-autotest:::autotest_single_yaml(...) at autotest/R/autotest-functions.R:59:5
#>  13.           +-base::rbind(reports, autotest_vector(test_obj, test_data)) at autotest/R/autotest-functions.R:139:13
#>  14.           | \-base::rbind(deparse.level, ...)
#>  15.           +-autotest:::autotest_vector(test_obj, test_data) at autotest/R/autotest-functions.R:139:13
#>  16.           \-autotest:::autotest_vector.autotest_obj(test_obj, test_data) at autotest/R/test-vector.R:3:5
#>  17.             +-base::rbind(ret, test_double_noise(x, test_data = test_data)) at autotest/R/test-vector.R:49:13
#>  18.             | \-base::rbind(deparse.level, ...)
#>  19.             +-autotest:::test_double_noise(x, test_data = test_data) at autotest/R/test-vector.R:49:13
#>  20.             \-autotest:::test_double_noise.autotest_obj(x, test_data = test_data) at autotest/R/input-double.R:74:5
#>  21.               \-autotest:::double_noise(x) at autotest/R/input-double.R:97:5
#>  22.                 +-base::tryCatch(...) at autotest/R/input-double.R:132:5
#>  23.                 | \-base (local) tryCatchList(expr, classes, parentenv, handlers)
#>  24.                 |   \-base (local) tryCatchOne(expr, names, parentenv, handlers[[1L]])
#>  25.                 |     \-base (local) doTryCatch(return(expr), name, parentenv, handler)
#>  26.                 +-withr::with_seed(seed, do.call(x$fn, x$params)) at autotest/R/input-double.R:132:5
#>  27.                 | \-withr::with_preserve_seed(...)
#>  28.                 +-base::do.call(x$fn, x$params) at autotest/R/input-double.R:132:5
#>  29.                 \-stats::qnorm(...)
#> 
#> Warning (test-statspkg.R:99:5): autotest rnorm
#> NaNs produced
#> Backtrace:
#>      x
#>   1. +-testthat::expect_message(...) at test-statspkg.R:99:5
#>   2. | \-testthat:::quasi_capture(enquo(object), label, capture_messages)
#>   3. |   +-testthat (local) .capture(...)
#>   4. |   | \-base::withCallingHandlers(...)
#>   5. |   \-rlang::eval_bare(quo_get_expr(.quo), quo_get_env(.quo))
#>   6. \-autotest::autotest_package(...)
#>   7.   +-base::rbind(...) at autotest/R/autotest-functions.R:259:9
#>   8.   | \-base::rbind(deparse.level, ...)
#>   9.   \-autotest::autotest_yaml(...) at autotest/R/autotest-functions.R:259:9
#>  10.     \-base::lapply(...) at autotest/R/autotest-functions.R:59:5
#>  11.       \-autotest (local) FUN(X[[i]], ...)
#>  12.         \-autotest:::autotest_single_yaml(...) at autotest/R/autotest-functions.R:59:5
#>  13.           +-base::rbind(reports, autotest_vector(test_obj, test_data)) at autotest/R/autotest-functions.R:139:13
#>  14.           | \-base::rbind(deparse.level, ...)
#>  15.           +-autotest:::autotest_vector(test_obj, test_data) at autotest/R/autotest-functions.R:139:13
#>  16.           \-autotest:::autotest_vector.autotest_obj(test_obj, test_data) at autotest/R/test-vector.R:3:5
#>  17.             +-base::rbind(ret, test_double_noise(x, test_data = test_data)) at autotest/R/test-vector.R:49:13
#>  18.             | \-base::rbind(deparse.level, ...)
#>  19.             +-autotest:::test_double_noise(x, test_data = test_data) at autotest/R/test-vector.R:49:13
#>  20.             \-autotest:::test_double_noise.autotest_obj(x, test_data = test_data) at autotest/R/input-double.R:74:5
#>  21.               \-autotest:::double_noise(x) at autotest/R/input-double.R:97:5
#>  22.                 +-base::tryCatch(...) at autotest/R/input-double.R:132:5
#>  23.                 | \-base (local) tryCatchList(expr, classes, parentenv, handlers)
#>  24.                 |   \-base (local) tryCatchOne(expr, names, parentenv, handlers[[1L]])
#>  25.                 |     \-base (local) doTryCatch(return(expr), name, parentenv, handler)
#>  26.                 +-withr::with_seed(seed, do.call(x$fn, x$params)) at autotest/R/input-double.R:132:5
#>  27.                 | \-withr::with_preserve_seed(...)
#>  28.                 +-base::do.call(x$fn, x$params) at autotest/R/input-double.R:132:5
#>  29.                 \-stats::qnorm(...)
#> ---------------------------------------------------------------------------------
#> v |          4 | testthat expectation [88.8s]
#> v |         20 | text parsing
#> v |          8 | test types
#> v |         14 | yaml [4.2s]
#> 
#> == Results ======================================================================
#> Duration: 221.6 s
#> 
#> [ FAIL 0 | WARN 2 | SKIP 0 | PASS 93 ]

[codecov]

Codecov Report

Attention: 1 lines in your changes are missing coverage. Please review.

Comparison is base (7f76988) 78.44% compared to head (e94d5d4) 78.43%.

❗ Current head e94d5d4 differs from pull request most recent head f93ef5f. Consider uploading reports for the commit f93ef5f to get more accurate results

Files	Patch %	Lines
R/autotest-functions.R	83.33%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main      #87      +/-   ##
==========================================
- Coverage   78.44%   78.43%   -0.01%     
==========================================
  Files          29       29              
  Lines        4319     4322       +3     
==========================================
+ Hits         3388     3390       +2     
- Misses        931      932       +1     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.