Run Tests is now very verbose, showing one line per test #1561

DavisVaughan · 2022-01-31T22:41:52Z

If i hit Run Tests on a file with testthat 3.1.2, it shows me output for every individual test in the file rather than one summary at the end. (RStudio 2021.09.2 Build 382)

==> Testing R file using 'testthat'

ℹ Loading vctrs
Warning message:
package 'testthat' was built under R version 4.1.2 

══ Testing test-slice-interleave.R ═════════════════════════════════════════════

[ FAIL 0 | WARN 0 | SKIP 0 | PASS 0 ]
[ FAIL 0 | WARN 0 | SKIP 0 | PASS 1 ]
[ FAIL 0 | WARN 0 | SKIP 0 | PASS 2 ]
[ FAIL 0 | WARN 0 | SKIP 0 | PASS 3 ]
[ FAIL 0 | WARN 0 | SKIP 0 | PASS 4 ]
[ FAIL 0 | WARN 0 | SKIP 0 | PASS 5 ]
[ FAIL 0 | WARN 0 | SKIP 0 | PASS 6 ]
[ FAIL 0 | WARN 0 | SKIP 0 | PASS 7 ]
[ FAIL 0 | WARN 0 | SKIP 0 | PASS 8 ]
[ FAIL 0 | WARN 0 | SKIP 0 | PASS 9 ]
[ FAIL 0 | WARN 0 | SKIP 0 | PASS 10 ]
[ FAIL 0 | WARN 0 | SKIP 0 | PASS 11 ]
[ FAIL 0 | WARN 0 | SKIP 0 | PASS 12 ]
[ FAIL 0 | WARN 0 | SKIP 0 | PASS 13 ]
[ FAIL 0 | WARN 0 | SKIP 0 | PASS 14 ]
[ FAIL 0 | WARN 0 | SKIP 0 | PASS 15 ] Done!

Test complete

Gabor:

It is not expected, the IDE probably had some assumptions about testthat output, which do not hold any more.

DavisVaughan · 2022-09-12T15:43:39Z

@gaborcsardi did you know cli 3.4.0 fixes this?

gaborcsardi · 2022-09-12T15:52:50Z

I would not think that cli fixed this, but FWIW, I do not see it with a fairly recent RStudio build.

DavisVaughan · 2022-09-12T16:03:19Z

If I swap to cli 3.3.0 it breaks again, which is why I thought they'd be related

dpprdan · 2022-09-12T16:08:07Z

I see

ℹ Loading desc

══ Testing test-built.R ════════════════════════════════════════════════════════

[ FAIL 0 | WARN 0 | SKIP 0 | PASS 0 ]
[ FAIL 0 | WARN 0 | SKIP 0 | PASS 1 ]
[ FAIL 0 | WARN 0 | SKIP 0 | PASS 2 ] Done!

Test complete

with cli 3.3.0 and latest stable RStudio

Session info

sessioninfo::session_info()
#> ─ Session info ───────────────────────────────────────────────────────────────
#>  setting  value
#>  version  R version 4.2.1 (2022-06-23 ucrt)
#>  os       Windows 10 x64 (build 19044)
#>  system   x86_64, mingw32
#>  ui       RTerm
#>  language en
#>  collate  German_Germany.utf8
#>  ctype    German_Germany.utf8
#>  tz       Europe/Berlin
#>  date     2022-09-12
#>  pandoc   2.19.2 @ C:/PROGRA~1/Pandoc/ (via rmarkdown)
#> 
#> ─ Packages ───────────────────────────────────────────────────────────────────
#>  package     * version date (UTC) lib source
#>  brio          1.1.3   2021-11-30 [1] CRAN (R 4.2.0)
#>  cli           3.3.0   2022-04-25 [1] CRAN (R 4.2.0)
#>  digest        0.6.29  2021-12-01 [1] CRAN (R 4.2.0)
#>  evaluate      0.16    2022-08-09 [1] CRAN (R 4.2.1)
#>  fansi         1.0.3   2022-03-24 [1] CRAN (R 4.2.0)
#>  fastmap       1.1.0   2021-01-25 [1] CRAN (R 4.2.0)
#>  fs            1.5.2   2021-12-08 [1] CRAN (R 4.2.0)
#>  glue          1.6.2   2022-02-24 [1] CRAN (R 4.2.0)
#>  highr         0.9     2021-04-16 [1] CRAN (R 4.2.0)
#>  htmltools     0.5.3   2022-07-18 [1] CRAN (R 4.2.1)
#>  knitr         1.40    2022-08-24 [1] CRAN (R 4.2.1)
#>  lifecycle     1.0.1   2021-09-24 [1] CRAN (R 4.2.0)
#>  magrittr      2.0.3   2022-03-30 [1] CRAN (R 4.2.0)
#>  pillar        1.8.1   2022-08-19 [1] CRAN (R 4.2.1)
#>  pkgconfig     2.0.3   2019-09-22 [1] CRAN (R 4.2.0)
#>  purrr         0.3.4   2020-04-17 [1] CRAN (R 4.2.0)
#>  R.cache       0.16.0  2022-07-21 [1] CRAN (R 4.2.1)
#>  R.methodsS3   1.8.2   2022-06-13 [1] CRAN (R 4.2.0)
#>  R.oo          1.25.0  2022-06-12 [1] CRAN (R 4.2.0)
#>  R.utils       2.12.0  2022-06-28 [1] CRAN (R 4.2.1)
#>  R6            2.5.1   2021-08-19 [1] CRAN (R 4.2.0)
#>  reprex        2.0.2   2022-08-17 [1] CRAN (R 4.2.1)
#>  rlang         1.0.5   2022-08-31 [1] CRAN (R 4.2.1)
#>  rmarkdown     2.16    2022-08-24 [1] CRAN (R 4.2.1)
#>  rstudioapi    0.14    2022-08-22 [1] CRAN (R 4.2.1)
#>  sessioninfo   1.2.2   2021-12-06 [1] CRAN (R 4.2.0)
#>  stringi       1.7.8   2022-07-11 [1] CRAN (R 4.2.1)
#>  stringr       1.4.1   2022-08-20 [1] CRAN (R 4.2.1)
#>  styler        1.7.0   2022-03-13 [1] CRAN (R 4.2.0)
#>  testthat    * 3.1.4   2022-04-26 [1] CRAN (R 4.2.0)
#>  tibble        3.1.8   2022-07-22 [1] CRAN (R 4.2.1)
#>  utf8          1.2.2   2021-07-24 [1] CRAN (R 4.2.0)
#>  vctrs         0.4.1   2022-04-13 [1] CRAN (R 4.2.0)
#>  withr         2.5.0   2022-03-03 [1] CRAN (R 4.2.0)
#>  xfun          0.32    2022-08-10 [1] CRAN (R 4.2.1)
#>  yaml          2.3.5   2022-02-21 [1] CRAN (R 4.2.0)
#> 
#>  [1] C:/Users/Daniel/AppData/Local/R/win-library/4.2
#>  [2] C:/Program Files/R/R-4.2.1/library
#> 
#> ──────────────────────────────────────────────────────────────────────────────

dpprdan · 2022-09-12T16:53:31Z

... and the same with cli 3.4.0, so the new cli version doesn't fix this for me. 🤷🏻‍♂️

Session info

sessioninfo::session_info()
#> ─ Session info ───────────────────────────────────────────────────────────────
#>  setting  value
#>  version  R version 4.2.1 (2022-06-23 ucrt)
#>  os       Windows 10 x64 (build 19044)
#>  system   x86_64, mingw32
#>  ui       RTerm
#>  language EN
#>  collate  German_Germany.utf8
#>  ctype    German_Germany.utf8
#>  tz       Europe/Berlin
#>  date     2022-09-12
#>  pandoc   2.18 @ C:/Program Files/RStudio/bin/quarto/bin/tools/ (via rmarkdown)
#> 
#> ─ Packages ───────────────────────────────────────────────────────────────────
#>  package     * version date (UTC) lib source
#>  brio          1.1.3   2021-11-30 [1] CRAN (R 4.2.0)
#>  cli           3.4.0   2022-09-08 [1] CRAN (R 4.2.1)
#>  digest        0.6.29  2021-12-01 [1] CRAN (R 4.2.0)
#>  evaluate      0.16    2022-08-09 [1] CRAN (R 4.2.1)
#>  fansi         1.0.3   2022-03-24 [1] CRAN (R 4.2.0)
#>  fastmap       1.1.0   2021-01-25 [1] CRAN (R 4.2.0)
#>  fs            1.5.2   2021-12-08 [1] CRAN (R 4.2.0)
#>  glue          1.6.2   2022-02-24 [1] CRAN (R 4.2.0)
#>  highr         0.9     2021-04-16 [1] CRAN (R 4.2.0)
#>  htmltools     0.5.3   2022-07-18 [1] CRAN (R 4.2.1)
#>  knitr         1.40    2022-08-24 [1] CRAN (R 4.2.1)
#>  lifecycle     1.0.2   2022-09-09 [1] CRAN (R 4.2.1)
#>  magrittr      2.0.3   2022-03-30 [1] CRAN (R 4.2.0)
#>  pillar        1.8.1   2022-08-19 [1] CRAN (R 4.2.1)
#>  pkgconfig     2.0.3   2019-09-22 [1] CRAN (R 4.2.0)
#>  purrr         0.3.4   2020-04-17 [1] CRAN (R 4.2.0)
#>  R.cache       0.16.0  2022-07-21 [1] CRAN (R 4.2.1)
#>  R.methodsS3   1.8.2   2022-06-13 [1] CRAN (R 4.2.0)
#>  R.oo          1.25.0  2022-06-12 [1] CRAN (R 4.2.0)
#>  R.utils       2.12.0  2022-06-28 [1] CRAN (R 4.2.1)
#>  R6            2.5.1   2021-08-19 [1] CRAN (R 4.2.0)
#>  reprex        2.0.2   2022-08-17 [1] CRAN (R 4.2.1)
#>  rlang         1.0.5   2022-08-31 [1] CRAN (R 4.2.1)
#>  rmarkdown     2.16    2022-08-24 [1] CRAN (R 4.2.1)
#>  rstudioapi    0.14    2022-08-22 [1] CRAN (R 4.2.1)
#>  sessioninfo   1.2.2   2021-12-06 [1] CRAN (R 4.2.0)
#>  stringi       1.7.8   2022-07-11 [1] CRAN (R 4.2.1)
#>  stringr       1.4.1   2022-08-20 [1] CRAN (R 4.2.1)
#>  styler        1.7.0   2022-03-13 [1] CRAN (R 4.2.0)
#>  testthat    * 3.1.4   2022-04-26 [1] CRAN (R 4.2.0)
#>  tibble        3.1.8   2022-07-22 [1] CRAN (R 4.2.1)
#>  utf8          1.2.2   2021-07-24 [1] CRAN (R 4.2.0)
#>  vctrs         0.4.1   2022-04-13 [1] CRAN (R 4.2.0)
#>  withr         2.5.0   2022-03-03 [1] CRAN (R 4.2.0)
#>  xfun          0.32    2022-08-10 [1] CRAN (R 4.2.1)
#>  yaml          2.3.5   2022-02-21 [1] CRAN (R 4.2.0)
#> 
#>  [1] C:/Users/Daniel/AppData/Local/R/win-library/4.2
#>  [2] C:/Program Files/R/R-4.2.1/library
#> 
#> ──────────────────────────────────────────────────────────────────────────────

dpprdan · 2022-09-12T16:58:11Z

the IDE probably had some assumptions about testthat output, which do not hold any more.

Does this mean that this should/needs to be fixed in RStudio, @gaborcsardi?

gaborcsardi · 2022-09-12T17:08:19Z

I would think so, yes. Try the latest RStudio release.

DavisVaughan · 2022-09-12T17:17:08Z

It is possible you need a daily build https://dailies.rstudio.com/

dpprdan · 2022-09-12T17:50:41Z

Sorry, no luck. I get the same output with the latest daily (and cli 3.4.0)

  RStudio 2022.11.0-daily+173 "Elsbeth Geranium" Daily (76585f6cb30fff5acca8555a3634e1241fef7a40, 2022-09-09) for Windows
  Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) RStudio/2022.11.0-daily+173 Chrome/102.0.5005.167 Electron/19.0.10 Safari/537.36

dpprdan · 2022-09-12T17:56:18Z

I would think so, yes.

It's not filed as an issue there, is it?

gaborcsardi · 2022-09-12T18:26:21Z

It's not filed as an issue there, is it?

IDK.

hadley · 2022-09-19T20:00:53Z

This now mostly works for me. The remainder should be fixed by #1633

dpprdan · 2022-09-22T11:15:37Z

This now mostly works for me. The remainder should be fixed by #1633

Could you please elaborate @hadley? Because it does not on my machine. I am still seeing with RStudio Desktop 2022.11.0-daily+199, Win10, testthat from #1633 (but also today's main @ 81dfbed):

==> Testing R file using 'testthat'


══ Testing test-testthat.R ═════════════════════════════════════════════════════

[ FAIL 0 | WARN 0 | SKIP 0 | PASS 0 ]
[ FAIL 0 | WARN 0 | SKIP 0 | PASS 1 ]
[ FAIL 0 | WARN 0 | SKIP 0 | PASS 2 ] Done!

Test complete

Do you mean "testthat is fine, got to be fixed in rstudio/rstudio#11969"?

Session info

RStudio 2022.11.0-daily+199 "Elsbeth Geranium" Daily (b08a343e7263eef6864e9236bbf1302197702605, 2022-09-21) for Windows
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) RStudio/2022.11.0-daily+199 Chrome/102.0.5005.167 Electron/19.0.10 Safari/537.36

sessioninfo::session_info()
#> ─ Session info ───────────────────────────────────────────────────────────────
#>  setting  value
#>  version  R version 4.2.1 (2022-06-23 ucrt)
#>  os       Windows 10 x64 (build 19044)
#>  system   x86_64, mingw32
#>  ui       RTerm
#>  language EN
#>  collate  German_Germany.utf8
#>  ctype    German_Germany.utf8
#>  tz       Europe/Berlin
#>  date     2022-09-22
#>  pandoc   2.19.2 @ C:/Program Files/RStudio/resources/app/bin/quarto/bin/tools/ (via rmarkdown)
#> 
#> ─ Packages ───────────────────────────────────────────────────────────────────
#>  package     * version    date (UTC) lib source
#>  brio          1.1.3      2021-11-30 [1] CRAN (R 4.2.0)
#>  cli           3.4.0      2022-09-08 [1] CRAN (R 4.2.1)
#>  digest        0.6.29     2021-12-01 [1] CRAN (R 4.2.0)
#>  evaluate      0.16       2022-08-09 [1] CRAN (R 4.2.1)
#>  fansi         1.0.3      2022-03-24 [1] CRAN (R 4.2.0)
#>  fastmap       1.1.0      2021-01-25 [1] CRAN (R 4.2.0)
#>  fs            1.5.2      2021-12-08 [1] CRAN (R 4.2.0)
#>  glue          1.6.2      2022-02-24 [1] CRAN (R 4.2.0)
#>  highr         0.9        2021-04-16 [1] CRAN (R 4.2.0)
#>  htmltools     0.5.3      2022-07-18 [1] CRAN (R 4.2.1)
#>  knitr         1.40       2022-08-24 [1] CRAN (R 4.2.1)
#>  lifecycle     1.0.2      2022-09-09 [1] CRAN (R 4.2.1)
#>  magrittr      2.0.3      2022-03-30 [1] CRAN (R 4.2.0)
#>  pillar        1.8.1      2022-08-19 [1] CRAN (R 4.2.1)
#>  pkgconfig     2.0.3      2019-09-22 [1] CRAN (R 4.2.0)
#>  purrr         0.3.4      2020-04-17 [1] CRAN (R 4.2.0)
#>  R.cache       0.16.0     2022-07-21 [1] CRAN (R 4.2.1)
#>  R.methodsS3   1.8.2      2022-06-13 [1] CRAN (R 4.2.0)
#>  R.oo          1.25.0     2022-06-12 [1] CRAN (R 4.2.0)
#>  R.utils       2.12.0     2022-06-28 [1] CRAN (R 4.2.1)
#>  R6            2.5.1      2021-08-19 [1] CRAN (R 4.2.0)
#>  reprex        2.0.2      2022-08-17 [1] CRAN (R 4.2.1)
#>  rlang         1.0.5      2022-08-31 [1] CRAN (R 4.2.1)
#>  rmarkdown     2.16       2022-08-24 [1] CRAN (R 4.2.1)
#>  rstudioapi    0.14       2022-08-22 [1] CRAN (R 4.2.1)
#>  sessioninfo   1.2.2      2021-12-06 [1] CRAN (R 4.2.0)
#>  stringi       1.7.8      2022-07-11 [1] CRAN (R 4.2.1)
#>  stringr       1.4.1      2022-08-20 [1] CRAN (R 4.2.1)
#>  styler        1.7.0      2022-03-13 [1] CRAN (R 4.2.0)
#>  testthat    * 3.1.4.9000 2022-09-22 [1] Github (r-lib/testthat@a871362)
#>  tibble        3.1.8      2022-07-22 [1] CRAN (R 4.2.1)
#>  utf8          1.2.2      2021-07-24 [1] CRAN (R 4.2.0)
#>  vctrs         0.4.1      2022-04-13 [1] CRAN (R 4.2.0)
#>  withr         2.5.0      2022-03-03 [1] CRAN (R 4.2.0)
#>  xfun          0.32       2022-08-10 [1] CRAN (R 4.2.1)
#>  yaml          2.3.5      2022-02-21 [1] CRAN (R 4.2.0)
#> 
#>  [1] C:/Users/Daniel/AppData/Local/R/win-library/4.2
#>  [2] C:/Program Files/R/R-4.2.1/library
#> 
#> ──────────────────────────────────────────────────────────────────────────────

hadley · 2022-09-22T12:03:45Z

It works for me 😞

dpprdan · 2022-09-22T13:59:44Z

Then we'll ship your machine. 😉

hadley · 2023-03-29T23:55:53Z

This wasn't working for me for a bit, but it seems to be working once more. This is a bit of a mystery to me.

kevinushey · 2023-07-10T21:42:49Z

Ok, I've figured out what's going on here. tl;dr is that this is an issue with how RStudio launches the R process for running tests, but only on Windows, combined with how cli tries to detect if it's running in an RStudio sub-process.

cli tries to compare RSTUDIO_SESSION_PID against the parent process ID to conclude if it's within an R sub-process. However, this check fails if we try to launch R within a cmd.exe process, since cli then sees the cmd.exe process rather than the R session.

As a workaround, one can set:

Sys.setenv(R_CLI_DYNAMIC = TRUE)

For posterity, here's what cli concludes if you run str(cli:::rstudio$detect()) within a test run by Run Tests:

List of 7
 $ type              : chr "rstudio_subprocess"
 $ dynamic_tty       : logi FALSE
 $ ansi_tty          : logi FALSE
 $ ansi_color        : logi FALSE
 $ num_colors        : int 1
 $ hyperlink         : logi FALSE
 $ has_canonical_mode: logi FALSE

In other words, cli sees that the parent process is the cmd.exe terminal "hosting" the R process, rather than the RStudio session itself. I'm not sure why this doesn't affect other platforms, though -- maybe some other fall-through logic helps things to work on Linux / macOS?

Either way, I think this ultimately needs to be fixed on the RStudio side. But at least the workaround is simple...

hadley · 2023-09-18T13:30:04Z

@gaborcsardi is then anything we can do in cli in the interim?

bdforbes · 2023-11-09T08:59:02Z

@kevinushey I've run Sys.setenv(R_CLI_DYNAMIC = TRUE) as you suggest, and can confirm that during Run Tests I get the same output as you state for cli::rstudio$detect(), but I'm still not seeing any colouring or hyperlinks in the Tests pane. Am I mistaken about how your workaround should be working?

I'm on RStudio 2023.09.1 Build 393, cli 3.6.1 and testthat 3.2.0.

EDIT: Looking more closely at the rstudio$detect() output, I realise it's telling me it won't colour text or use hyperlinks. Where is it exactly that I should run Sys.setenv(R_CLI_DYNAMIC = TRUE)? I've tried running it in the main R session, as well as putting it in my test file.

kevinushey · 2023-11-09T18:11:36Z

It should be sufficient to set that in your current R environment, since that would be inherited by child processes as well...

Are you able to debug more to see what cli:::rstudio$detect() is doing? See also:

> cli:::rstudio$.internal$detect_new
function (rspid, clear_cache)
{
    mypid <- Sys.getpid()
    new <- get_data()
    if (mypid == rspid) {
        return(get_caps(new, type = "rstudio_console"))
    }
    parentpid <- get_parentpid()
    pane <- Sys.getenv("RSTUDIO_CHILD_PROCESS_PANE")
    if (pane == "")
        return(detect_old(clear_cache))
    new$type <- if (rspid == parentpid) {
        if (pane == "job") {
            "rstudio_job"
        }
        else if (pane == "build") {
            "rstudio_build_pane"
        }
        else if (pane == "render") {
            "rstudio_render_pane"
        }
        else if (pane == "terminal" && new$tty && new$envs["ASCIICAST"] !=
            "true") {
            "rstudio_terminal"
        }
        else {
            "rstudio_subprocess"
        }
    }
    else if (pane == "terminal" && new$tty && new$envs[["ASCIICAST"]] !=
        "true") {
        "rstudio_terminal"
    }
    else {
        "rstudio_subprocess"
    }
    get_caps(new)
}
<environment: 0x11932b240>
> cli:::rstudio$.internal$detect_old
function (clear_cache = FALSE)
{
    cache <- TRUE
    new <- get_data()
    new$type <- if (new$envs[["RSTUDIO"]] != "1") {
        "not_rstudio"
    }
    else if (new$gui == "RStudio" && new$api) {
        "rstudio_console"
    }
    else if (!new$api && basename(new$args[1]) == "RStudio") {
        cache <- FALSE
        "rstudio_console_starting"
    }
    else if (new$gui == "Rgui") {
        "not_rstudio"
    }
    else if (new$tty && new$envs[["ASCIICAST"]] != "true") {
        "rstudio_terminal"
    }
    else if (!new$tty && new$envs[["RSTUDIO_TERM"]] == "" &&
        new$envs[["R_BROWSER"]] == "false" && new$envs[["R_PDFVIEWER"]] ==
        "false" && is_build_pane_command(new$args)) {
        "rstudio_build_pane"
    }
    else if (new$envs[["RSTUDIOAPI_IPC_REQUESTS_FILE"]] != "" &&
        grepl("rstudio", new$envs[["XPC_SERVICE_NAME"]])) {
        "rstudio_job"
    }
    else if (new$envs[["RSTUDIOAPI_IPC_REQUESTS_FILE"]] != "" &&
        any(grepl("SourceWithProgress.R", new$args))) {
        "rstudio_job"
    }
    else {
        "rstudio_subprocess"
    }
    installing <- Sys.getenv("R_PACKAGE_DIR", "")
    if (cache && installing == "")
        data <<- new
    get_caps(new)
}

bdforbes · 2023-11-09T21:49:27Z

I see in detect_new() that it doesn't explicitly check for the Tests pane. Is that perhaps the issue here? I'm running tests outside of a package, by opening a specific test R script and clicking the "Run Tests" button. That opens the Tests pane, not the Build pane.

Interestingly, I did notice a slight change in output behaviour before and after running Sys.setenv(R_CLI_DYNAMIC = TRUE).

Before running it, the test summary line, e.g. [ FAIL 1 | WARN 0 | SKIP 0 | PASS 1 ], is always on a new line.
After running it, it updates within the same line.

kevinushey · 2023-11-09T22:10:40Z

Hmm, my version of RStudio does set these environment variables, even when clicking the Run Tests button:

RSTUDIO_CHILD_PROCESS_PANE
                        build
RSTUDIO_CLI_HYPERLINKS
                        true

These are for individual test files that would normally be run via testthat. Note also that I see test output within the Build pane.

bdforbes · 2023-11-09T22:21:03Z

Sorry, correction: it is indeed reading as the Build pane. However, at this line:

https://github.com/r-lib/cli/blob/eeeb260f0e24006940fe30ed4afb29d0a848a5a7/R/aab-rstudio-detect.R#L91C29-L91C29

The RStudio PID is not matching the retrieved parent PID, hence it is treating it as rstudio_subprocess. I don't know how to debug any further than that, any tips?

kevinushey · 2023-11-09T22:22:58Z

Ah, yeah, then that's the same diagnosis as in #1561 (comment), so would require some changes in cli to accommodate.

bdforbes · 2023-11-09T22:27:02Z

Ok, thanks. I thought the workaround you stated might fix it but now seems clear that cli still wouldn't be able to figure out it does have ANSI capabilities in this context. In the meantime I'll just run the test files manually using test_file, which is fine.

Thanks for your help!

olivroy · 2024-03-05T19:55:23Z

@kevinushey I did a little bit more investigation. It seems that RStudio doesn't initialize properly some env vars that are crucial for good functioning output.

RSTUDIO_CLI_HYPERLINKS = "true"
RSTUDIO = "1"

it seems that there is a problem in RStudio directly. During the test_active file I created this file to investigate which envvars are initialized.

Maybe some code in the IDE could be modified to enforce the envvars are properly set.
maybe in :
https://github.com/rstudio/rstudio/blob/cb7f275bfe6f290272b6eae6e6667852c1d51d59/src/gwt/src/org/rstudio/studio/client/workbench/views/buildtools/BuildPresenter.java#L363-L370

I tried to make a fix in cli, but it didn't really make sense as the important missing env vars problem is from RStudio..

The criteria used by cli is

is_rstudio <- function() Sys.getenv("RSTUDIO") == "1"

and returns FALSE for me in the test active file setting.

In a package, I used the following for my investigation

test_that("test in pane", {
  print({
    "\n"
    Sys.getenv("RSTUDIO")
    Sys.getenv("RSTUDIO_CLI_HYPERLINKS ")
    Sys.getenv()
  })
  expect_equal(rstudio_detect()$type, "rstudio_build_pane")
})

The relevant env vars I see are the following.

CLICOLOR_FORCE 1
RSTUDIO_CONSOLE_COLOR 256
RSTUDIO_CONSOLE_WIDTH 80
RSTUDIO_DESKTOP_EXE
RSTUDIO_PANDOC
RSTUDIO_PROGRAM_MODE
RSTUDIO_SESSION_PORT
RSTUDIO_USER_IDENTITY
RS_LOCAL_PEER
RS_LOG_LEVEL WARN
RS_RPOSTBACK_PATH
R_BROWSER false
R_PDFVIEWER false
R_CLI_DYNAMIC TRUE
R_CLI_HAS_HYPERLINK_IDE_HELP true
R_CLI_HAS_HYPERLINK_IDE_RUN true
R_CLI_HAS_HYPERLINK_IDE_VIGNETTE true

I would guess that in the regular testthat (ctrl + shift + t), the 2 variables I mentioned above are present to ensure proper detection.

Hope this can help resolve the issue (I opened an issue in the repo a while back. (rstudio/rstudio#13088)

--

Testing the active file works as expected with hyperlinks functioning

test_that("test in pane", {
  withr::local_envvar(list(
    "RSTUDIO" = "1",
    "RSTUDIO_CLI_HYPERLINKS" = "true"
    ))
  expect_equal(rstudio_detect()$type, "rstudio_build_pane")
  expect_false(TRUE)
})

kevinushey · 2024-03-09T21:48:52Z

@olivroy if the fix for this would need to happen in RStudio, could you please file this issue on the RStudio issue repository so it doesn't get lost?

olivroy · 2024-03-11T16:41:51Z

@kevinushey already done with rstudio/rstudio#13088, which is probably a duplicate of rstudio/rstudio#11969.

Now resolved on RStudio's end 2024.07

DavisVaughan changed the title ~~Test File is now very verbose, showing one line per test~~ Run Tests is now very verbose, showing one line per test Feb 1, 2022

dpprdan mentioned this issue Sep 17, 2022

Verbose testthat output rstudio/rstudio#11969

Closed

4 tasks

hadley closed this as completed Sep 19, 2022

hadley reopened this Mar 24, 2023

hadley added this to the 3.1.8 milestone Mar 24, 2023

hadley mentioned this issue Mar 24, 2023

The text "Done!" is not showing in the workbench results #1707

Closed

hadley closed this as completed Mar 29, 2023

hadley reopened this Jul 10, 2023

hadley mentioned this issue Sep 22, 2023

No colors in test active file output #1846

Closed

olivroy mentioned this issue Sep 22, 2023

No hyperlinks or colors when using run tests on the active file rstudio/rstudio#13088

Closed

4 tasks

hadley closed this as completed Mar 11, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Run Tests is now very verbose, showing one line per test #1561

Run Tests is now very verbose, showing one line per test #1561

DavisVaughan commented Jan 31, 2022

DavisVaughan commented Sep 12, 2022 •

edited

Loading

gaborcsardi commented Sep 12, 2022

DavisVaughan commented Sep 12, 2022

dpprdan commented Sep 12, 2022

dpprdan commented Sep 12, 2022

dpprdan commented Sep 12, 2022

gaborcsardi commented Sep 12, 2022

DavisVaughan commented Sep 12, 2022 •

edited

Loading

dpprdan commented Sep 12, 2022

dpprdan commented Sep 12, 2022

gaborcsardi commented Sep 12, 2022

hadley commented Sep 19, 2022

dpprdan commented Sep 22, 2022

hadley commented Sep 22, 2022

dpprdan commented Sep 22, 2022

hadley commented Mar 29, 2023

kevinushey commented Jul 10, 2023

hadley commented Sep 18, 2023 •

edited

Loading

bdforbes commented Nov 9, 2023 •

edited

Loading

kevinushey commented Nov 9, 2023

bdforbes commented Nov 9, 2023

kevinushey commented Nov 9, 2023

bdforbes commented Nov 9, 2023

kevinushey commented Nov 9, 2023

bdforbes commented Nov 9, 2023

olivroy commented Mar 5, 2024 •

edited

Loading

kevinushey commented Mar 9, 2024

olivroy commented Mar 11, 2024 •

edited

Loading

Run Tests is now very verbose, showing one line per test #1561

Run Tests is now very verbose, showing one line per test #1561

Comments

DavisVaughan commented Jan 31, 2022

DavisVaughan commented Sep 12, 2022 • edited Loading

gaborcsardi commented Sep 12, 2022

DavisVaughan commented Sep 12, 2022

dpprdan commented Sep 12, 2022

dpprdan commented Sep 12, 2022

dpprdan commented Sep 12, 2022

gaborcsardi commented Sep 12, 2022

DavisVaughan commented Sep 12, 2022 • edited Loading

dpprdan commented Sep 12, 2022

dpprdan commented Sep 12, 2022

gaborcsardi commented Sep 12, 2022

hadley commented Sep 19, 2022

dpprdan commented Sep 22, 2022

hadley commented Sep 22, 2022

dpprdan commented Sep 22, 2022

hadley commented Mar 29, 2023

kevinushey commented Jul 10, 2023

hadley commented Sep 18, 2023 • edited Loading

bdforbes commented Nov 9, 2023 • edited Loading

kevinushey commented Nov 9, 2023

bdforbes commented Nov 9, 2023

kevinushey commented Nov 9, 2023

bdforbes commented Nov 9, 2023

kevinushey commented Nov 9, 2023

bdforbes commented Nov 9, 2023

olivroy commented Mar 5, 2024 • edited Loading

kevinushey commented Mar 9, 2024

olivroy commented Mar 11, 2024 • edited Loading

DavisVaughan commented Sep 12, 2022 •

edited

Loading

DavisVaughan commented Sep 12, 2022 •

edited

Loading

hadley commented Sep 18, 2023 •

edited

Loading

bdforbes commented Nov 9, 2023 •

edited

Loading

olivroy commented Mar 5, 2024 •

edited

Loading

olivroy commented Mar 11, 2024 •

edited

Loading