Progress bar responsiveness behind callr processes

[wlandau]

Related to #755. It seems that progress bars update less frequently when they run behind callr processes.

out <- callr::r(
  \() {
    library(cli)
    clean <- function() {
      options(cli.num_colors = 256, cli.dynamic = TRUE)
      cli_progress_bar("Cleaning data", total = 100)
      for (i in 1:100) {
        Sys.sleep(5/100)
        cli_progress_update()
      }
      cli_progress_done()
    }
    clean()
  },
  show = TRUE,
  stderr = "2>&1"
)

It would be nice if they updated with the same frequency and timing as in the local process.

clean <- function() {
  options(cli.num_colors = 256, cli.dynamic = TRUE)
  cli_progress_bar("Cleaning data", total = 100)
  for (i in 1:100) {
    Sys.sleep(5/100)
    cli_progress_update()
  }
  cli_progress_done()
}
clean()

[gaborcsardi]

Updates from a subprocess are much more costly, but you can set the CLI_TICK_TIME env var to configure how frequently they should happen. By default it is 200 (ms) in the main process.

[wlandau]

Thanks! CLI_TICK_TIME works if I set it before cli is loaded. In my use case, 200ms is fine.

[wlandau]

Updates from a subprocess are much more costly

Is this because of polling? Could an event-driven approach increase responsiveness and efficiency in general? Just wondering because @shikokuchuo's nanonext package provides scalability protocols and synchronization primitives from NNG, and it has become a Suggests: package for httr2.

[gaborcsardi]

I am not sure what you mean. What does NNG use for waiting for IO? I bet it uses poll() on Unix or something platform specific like kqueue, which is effectively the same.

poll() does not make programs slow. It makes them fast. What is the alternative, anyway?

The cost is that the message objects have to be sent to another process. I never measured this, but I suspect that most of the cost is running the R code that encodes the messages, and not the actual IO.

(Btw. if the buffer of the socket between the processes is full, then the subprocess will stop until the main process reads out some messages from the buffer.)

EDIT: https://github.com/nanomsg/nng/blob/5b886b57e79992f1d184ca0309757b01e320c552/src/platform/posix/posix_pollq_poll.c#L198