Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

plyr forces .progress="none" when .parallel=TRUE (although it works with progressr) #70

Open
HenrikBengtsson opened this issue Jan 22, 2020 · 1 comment

Comments

@HenrikBengtsson
Copy link
Owner

@HenrikBengtsson HenrikBengtsson commented Jan 22, 2020

Unfortunately, the plyr package ignores the .progress argument whenever .parallel = TRUE. It does this by forcing .progress = "none", e.g. in plyr::llply() there is:

  if (.parallel && .progress != "none") {
    message("Progress disabled when using parallel plyr")
    .progress <- "none"
  }

This is probably based on the assumption that progress updates do not work in parallel processing. However, that does indeed work when using future + progressr. Here's an example of this when we hack plyr to still use .progress = "progressr" internally:

# Force 'progressr' progress bars in plyr
> trace(plyr:::create_progress_bar, tracer = quote(name <- "progressr"))
Tracing function "create_progress_bar" in package "plyr (not-exported)"
[1] "create_progress_bar"
library(plyr)
library(progressr)
doFuture::registerDoFuture()
future::plan("multisession", workers = 2L)
with_progress({
  y <- llply(1:10, function(x) { Sys.sleep(1); identity(x) }, 
                .parallel = TRUE, .progress = "progressr")
})

which outputs:

|====================                              |  40%

and when done:

Tracing create_progress_bar(.progress) on entry 
Progress disabled when using parallel plyr
HenrikBengtsson added a commit that referenced this issue Jan 22, 2020
@HenrikBengtsson
Copy link
Owner Author

@HenrikBengtsson HenrikBengtsson commented May 5, 2020

Created PR hadley/plyr#301 to fix this in plyr.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
1 participant
You can’t perform that action at this time.