waitress infinite for long calculations (> 30 seconds)

[MalditoBarbudo]

waitress with the infinite = TRUE argument works nicely for calculations under 30 seconds, but when a plot/table/map needs longer calculations progress bar gets full and no progress is shown. Using the example in the documentation for waitress, but changing the sleep step from 3 to 60 seconds shows the problem:

library(shiny)
library(waiter)

ui <- fluidPage(
  use_waitress(),
  actionButton("btn", "render"),
  plotOutput("plot", width = 400)
)

server <- function(input, output){
  
  waitress <- Waitress$new("#btn", theme = "overlay", infinite = TRUE)
  
  output$plot <- renderPlot({
    input$btn
    
    waitress$start()
    
    Sys.sleep(60) # I change the original 3 for a 60 seconds calculation
    
    hist(runif(100))
    waitress$close() # hide when done
  })
  
}

shinyApp(ui, server)

After 30 seconds, the bar reaches the end, and the rest of the time the user has no sense of something is still working on the background.
Is there any way to control how increments are calculated to try to find an optimum calculation for each use case?

Background and study case:

I'm building an app in which users can ask for different rasters to load on a leaflet map. Those rasters have to be retrieved from a postgis postgresql database, and depending on the raster number or type, the operation can last from 20 seconds to several minutes. As I'm using an external function to retrieve the rasters (one I have no control over), infinite would be the perfect solution to inform the user something is working on the background, without knowing beforehand how long it would last. Normal shiny progress (shiny::Progress$new) or waiter::Hostess$new need to set the increments manually, which is no possible in this case.

[JohnCoene]

Yes, something better should definitely be implemented, the JavaScript that does so is here and below.

    var value = 0,
        inc = 0,
        end = 100;

    intervals[opts.name] = setInterval(function(){
      inc = ((end - value) / 50);
      value = Math.ceil(value + inc);
      window.waitress[opts.name].set(value);
     }, 350);

The underlying library has no "infinite" bar so I mimic that with setInterval: every 350 ms the bar is increased by ((end - value) / 50).

1. (100-0)/50 = increase by 50
2. (100 - 50) / 50 = increase by 25
3. (100 - 75) / 50 = increase by 12.5
4. etc

I'm sure better can be done. If you have a suggestion please let me know, I'm more than happy to implement the JavaScript.

[MalditoBarbudo]

Thanks for the fast answer.
Ok, I'm gonna think about an "algorithm" that can be useful in both short and long calculations and let you know.

[MalditoBarbudo]

I'm not proficient with JavaScript so I'm probably wrong, but what I understand is that the increase is not exactly exponential as you said, but in 2 seconds steps at the beginning?

1. (100-0)/50 is equal to 2, not 50
2. (100-2)/50 is then 1.96
3. (100-4)/50 is then 1.92

This goes on and around step 50 it becomes an increment of 1. I made some tests in R, and the times are consistent with what I see on the waitress bar in the example code.

If this is true, maybe I came with a solution that works relatively well in calculations ranging from 3 seconds to 5 minutes (more than that will require a better algorithm):

var value = 0,
inc = 0,
end = 100;

intervals[opts.name] = setInterval(function(){
  inc = ((end - value) / (end + value));
  value = Math.round((value + inc + Number.EPSILON) * 1000) / 1000
  window.waitress[opts.name].set(value);
}, 350);

1. inc changed to ((end - value) / (end + value)). This makes creates small but perceptible increments in the final part of the progress bar.
2. Math.ceil changed to Math.round with a tweak for problematic decimals (see here) and to calculate 3 decimals. Ceiling always round to the unit, making impossible to have infinite bars for more than 30 seconds with a timestep of 0.350 seconds. Also, rounding with 3 decimals allows that small increments (0.01) when reaching the 90% are still added (You can check the R code below to play with this).

As I said, I'm not really proficient with JavaScript (one thing I have on my list, but the list is long hehe), so below you can find the R code I used to test this:

three_seconds_steps <- 1:ceiling(3/0.350)
thirty_seconds_steps <- 1:ceiling(30/0.350)
sixty_seconds_steps <- 1:ceiling(60/0.350)
five_minutes_steps  <- 1:ceiling(300/0.350)

### 3 sec
# reset values
value <- 0
inc <- 0
end <- 100
time_pass <- 0
for (step in three_seconds_steps) {
  inc <- ((end - value) / (end + value))
  value <- round(value + inc, 3)
  time_pass <- time_pass + 0.350
  Sys.sleep(0.350)
  print(paste0('value: ', value, ' | inc: ', inc, ' | time_pass: ', time_pass))
}

### 30 sec
# reset values
value <- 0
inc <- 0
end <- 100
time_pass <- 0
for (step in thirty_seconds_steps) {
  inc <- ((end - value) / (end + value))
  value <- round(value + inc, 3)
  time_pass <- time_pass + 0.350
  Sys.sleep(0.350)
  print(paste0('value: ', value, ' | inc: ', inc, ' | time_pass: ', time_pass))
}

### 60 sec
# reset values
value <- 0
inc <- 0
end <- 100
time_pass <- 0
for (step in sixty_seconds_steps) {
  inc <- ((end - value) / (end + value))
  value <- round(value + inc, 3)
  time_pass <- time_pass + 0.350
  Sys.sleep(0.350)
  print(paste0('value: ', value, ' | inc: ', inc, ' | time_pass: ', time_pass))
}

### 5 min
# reset values
value <- 0
inc <- 0
end <- 100
time_pass <- 0
for (step in five_minutes_steps) {
  inc <- ((end - value) / (end + value))
  value <- round(value + inc, 3)
  time_pass <- time_pass + 0.350
  Sys.sleep(0.350)
  print(paste0('value: ', value, ' | inc: ', inc, ' | time_pass: ', time_pass))
}

[JohnCoene]

Thank you, I just pushed your suggestion and it works well indeed. Please create a pull request to add yourself as contributor to the package (if you want to of course).

[MalditoBarbudo]

Thanks for the quick check and update. I will create the pull request, thank you!

[Dschaykib]

If I understand this correctly, than #67 fixed the problem for hostess mentioned here #65 , but the same issue for waitress is still pending? Is there a chance, that it can be included as well?

[Dschaykib]

ok, my mistake, I had an older version installed. It works also fine with waitress. Thanks!