num core > 1 causes issues

[JEFworks]

Multiple users have experienced errors when increasing the number of cores to > 1
https://groups.google.com/forum/#!topic/singlecellstats/Miwy1Jg6PRU
https://groups.google.com/forum/#!topic/singlecellstats/ij2GWC1JLr0

The error may stem from updates to mclapply or bclapply.

[JEFworks]

From Alexander:

I can add from my side, the same code worked with 4 cores (the maximal) on MacOS and it didn't on unix system (ncore set to 4 as well while the max cores is 12).

Here are sessions Infos. 
Unix

> sessionInfo()
R version 3.3.2 (2016-10-31)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: CentOS Linux 7 (Core)

locale:
 [1] LC_CTYPE=fr_FR.UTF-8       LC_NUMERIC=C              
 [3] LC_TIME=fr_FR.UTF-8        LC_COLLATE=fr_FR.UTF-8    
 [5] LC_MONETARY=fr_FR.UTF-8    LC_MESSAGES=fr_FR.UTF-8   
 [7] LC_PAPER=fr_FR.UTF-8       LC_NAME=C                 
 [9] LC_ADDRESS=C               LC_TELEPHONE=C            
[11] LC_MEASUREMENT=fr_FR.UTF-8 LC_IDENTIFICATION=C       

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

MacOS
> sessionInfo()
R version 3.3.2 (2016-10-31)
Platform: x86_64-apple-darwin16.1.0 (64-bit)
Running under: macOS Sierra 10.12.2

locale:
[1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] scde_1.99.4     flexmix_2.3-13  lattice_0.20-34 RSQLite_1.1-1  

loaded via a namespace (and not attached):
 [1] Rcpp_0.12.8               nloptr_1.0.4              RColorBrewer_1.1-2        tools_3.3.2               Lmoments_1.2-3            digest_0.6.10            
 [7] lme4_1.1-12               memoise_1.0.0             tibble_1.2                nlme_3.1-128              mgcv_1.8-16               Matrix_1.2-7.1           
[13] DBI_0.5-1                 parallel_3.3.2            SparseM_1.74              RcppArmadillo_0.7.600.1.0 extRemes_2.0-8            MatrixModels_0.4-1       
[19] locfit_1.5-9.1            stats4_3.3.2              grid_3.3.2                nnet_7.3-12               Biobase_2.34.0            distillery_1.0-2         
[25] Rook_1.1-1                BiocParallel_1.8.1        minqa_1.2.4               limma_3.30.7              car_2.1-4                 edgeR_3.16.5             
[31] pcaMethods_1.66.0         modeltools_0.2-21         MASS_7.3-45               BiocGenerics_0.20.0       splines_3.3.2             assertthat_0.1           
[37] RMTstat_0.3               pbkrtest_0.4-6            brew_1.0-6                quantreg_5.29             rjson_0.2.15              Cairo_1.5-9    

Working with BiocParallel_1.8.1 and parallel_3.3.2

[dat4git]

Hi,

Has anyone figured a work around this issue? I just ran into the same error with a data set of 170 cells x 4000 genes. I've attached the code used, error and session info below. I will try re-running this on a single core with a smaller dataset to see if the error goes away but any advice/help in the meantime would be much appreciated. Please note, I am also using the dev version of scde and a linux machine.

Thanks,
SG

cd <- clean.counts(cts, min.lib.size=1000, min.reads = 1, min.detected = 1)

dim(cd)
[1] 4200 170

o.ifm <- scde.error.models(counts=cd, groups=ngroup, n.cores=10, threshold.segmentation=TRUE, save.crossfit.plots=FALSE, save.model.plots=FALSE, verbose=1)

cross-fitting cells.
number of pairs: 3403
number of pairs: 3741
total number of pairs: 7144
cross-fitting 7144 pairs:
building individual error models.
adjusting library size based on 2000 entries
....
ERROR fitting of 3 out of 83 cells resulted in errors reporting remaining 80 cells
....
Error in row.names<-.data.frame(*tmp*, value = value) :
invalid 'row.names' length

sessionInfo()

R version 3.2.1 (2015-06-18)

Platform: x86_64-unknown-linux-gnu (64-bit)
Running under: Red Hat Enterprise Linux Server release 6.8 (Santiago)

locale:
[1] C

attached base packages:
[1] stats graphics grDevices utils datasets methods base

other attached packages:
[1] scde_1.99.4 flexmix_2.3-13 lattice_0.20-33

loaded via a namespace (and not attached):
[1] Rcpp_0.12.9 edgeR_3.12.1
[3] splines_3.2.1 BiocGenerics_0.16.1
[5] MASS_7.3-45 BiocParallel_1.4.3
[7] rjson_0.2.15 RcppArmadillo_0.7.600.1.0
[9] brew_1.0-6 minqa_1.2.4
[11] distillery_1.0-2 car_2.1-0
[13] Rook_1.1-1 tools_3.2.1
[15] Lmoments_1.2-3 pbkrtest_0.4-2
[17] nnet_7.3-11 parallel_3.2.1
[19] RMTstat_0.3 grid_3.2.1
[21] Biobase_2.28.0 nlme_3.1-122
[23] mgcv_1.8-9 quantreg_5.29
[25] lambda.r_1.1.9 futile.logger_1.4.3
[27] modeltools_0.2-21 MatrixModels_0.4-1
[29] lme4_1.1-10 Matrix_1.2-2
[31] nloptr_1.0.4 RColorBrewer_1.1-2
[33] extRemes_2.0-8 futile.options_1.0.0
[35] limma_3.26.9 pcaMethods_1.60.0
[37] stats4_3.2.1 SparseM_1.74
[39] Cairo_1.5-