Output for 3.1.1) Parallel for loops with foreach

[ck37]

Hi Chris,

Reviewing this material again, and one thing I noticed is that example 3.1.1 has the cat output within a foreach and says:
"(Note that the printed statements from cat are not showing up in the creation of this document but should show if you run the code.)"

When I run the code I don't get any cat output, which is consistent with what I've gotten in my own foreach loops and read online about foreach discarding printed output. This is for my system at least, which is R 3.2.3 on OSX 10.11 via RStudio 0.99.878. I've seen the same thing on Ubuntu via BCE-fall-2015.

I did find that creating a cluster via doSNOW and specifying the outfile="" option per http://stackoverflow.com/a/26519566/2463900 works though.

Here's the code I used:

#library(parallel) # one of the core R packages
#library(doParallel)
library(doSNOW)
cl <- makeCluster(4, outfile="") # number of cores. Notice 'outfile'
registerDoSNOW(cl)
# library(multicore); library(doMC) # alternative to parallel/doParallel
# library(Rmpi); library(doMPI) # to use Rmpi as the back-end
library(foreach)
library(iterators)

taskFun <- function(){
  mn <- mean(rnorm(10000000))
  return(mn)
}
#nCores <- 4  # to set manually
#registerDoParallel(nCores)
# registerDoMC(nCores) # alternative to registerDoParallel
# cl <- startMPIcluster(nCores); registerDoMPI(cl) # when using Rmpi as the back-end

out <- foreach(i = 1:30, .combine = c) %dopar% {
  cat('Starting ', i, 'th job.\n', sep = '')
  outSub <- taskFun()
  cat('Finishing ', i, 'th job.\n', sep = '')
  outSub # this will become part of the out object
}
print(out)

# We have to explicitly stop a doSNOW() cluster, otherwise the cluster will waste cpu.
stopCluster(cl) 

Looks like it works fine via doParallel too, which doesn't keep the processes maxed out like doSnow does for me.

library(doParallel)
cl <- parallel::makeCluster(4, outfile = "")
registerDoParallel(cl)

Best,
Chris

[paciorek]

Hmm, not sure what the problem is here. When I run the following code (see
bottom) either directly in the R console or via:
R CMD BATCH input.R output.R
I see the printed output at the console and in output.R, respectively.

library(doParallel)
registerDoParallel(4)

out <- foreach(i = 1:30, .combine = c) %dopar% {
# try cat()
cat('Starting ', i, 'th job.\n', sep = '')
outSub <- i
# try print()
print(paste0('Finishing ', i, 'th job.', sep = ''))
outSub
}
print(out)

On Fri, Feb 12, 2016 at 11:08 AM, Chris Kennedy notifications@github.com
wrote:

Hi Chris,

Reviewing this material again, and one thing I noticed is that example
3.1.1 has the cat output within a foreach and says:
"(Note that the printed statements from cat are not showing up in the
creation of this document but should show if you run the code.)"

When I run the code I don't get any cat output, which is consistent with
what I've gotten in my own foreach loops and read online about foreach
discarding printed output. This is for my system at least, which is R 3.2.3
on OSX 10.11 via RStudio 0.99.878. I've seen the same thing on Ubuntu via
BCE-fall-2015.

I did find that creating a cluster via doSNOW and specifying the
outfile="" option per http://stackoverflow.com/a/26519566/2463900 works
though.

Here's the code I used:

#library(parallel) # one of the core R packages#library(doParallel)
library(doSNOW)cl <- makeCluster(4, outfile="") # number of cores. Notice 'outfile'
registerDoSNOW(cl)# library(multicore); library(doMC) # alternative to parallel/doParallel# library(Rmpi); library(doMPI) # to use Rmpi as the back-end
library(foreach)
library(iterators)
taskFun <- function(){
mn <- mean(rnorm(10000000))
return(mn)
}#nCores <- 4 # to set manually#registerDoParallel(nCores)# registerDoMC(nCores) # alternative to registerDoParallel# cl <- startMPIcluster(nCores); registerDoMPI(cl) # when using Rmpi as the back-end
out <- foreach(i = 1:30, .combine = c) %dopar% {
cat('Starting ', i, 'th job.\n', sep = '')
outSub <- taskFun()
cat('Finishing ', i, 'th job.\n', sep = '')
outSub # this will become part of the out object
}
print(out)

Best,
Chris

—
Reply to this email directly or view it on GitHub
#2.

[ck37]

Hmm, trying out your code:

• If I do R CMD BATCH I do see the output in output.R.
• It also works if I run the code in command-line interactive R.
• If I run the code in R GUI or RStudio I don't see the output.

So it seems to be GUI-specific, at least on OSX.