Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Potential bug using permutest? #369

Closed
ecologyjh opened this issue Jun 29, 2020 · 9 comments
Closed

Potential bug using permutest? #369

ecologyjh opened this issue Jun 29, 2020 · 9 comments
Assignees
Labels
Milestone

Comments

@ecologyjh
Copy link

ecologyjh commented Jun 29, 2020

I am not an R wizard yet but I just spent 3 hours trying to figure out why my permutest code stopped working. Maybe there's a way to fix this to save other people spending 3 hours on it in the future! Not sure if this would be reproduced on other machines as have not been able to test.

I ran the following rstan code to increase computing power:

rstan_options(auto_write = TRUE)
options(mc.cores = parallel::detectCores())

I then ran betadisper followed by permutest (the code from the example in the permutest.betadisper R documentation returns the same result for me - which can be used to test this) to be returned with the error:

Pstats [,1]:incorrect number of dimensions

When I restarted without running the cores code, there were no issues again. I read somewhere that this was supposed to have been fixed in a previous version - but I still had problems! Hope this is useful.

@gavinsimpson
Copy link
Contributor

gavinsimpson commented Jun 30, 2020

I can't reproduce this with:

library('rstan')
rstan_options(auto_write = TRUE)
options(mc.cores = parallel::detectCores())
library('vegan')
example('permutest.betadisper')

on

session info
Session info ───────────────────────────────────────────────────────────────
 setting  value                          
 version  R version 3.6.3 (2020-02-29)   
 os       Fedora 32 (Workstation Edition)
 system   x86_64, linux-gnu              
 ui       X11                            
 language (EN)                           
 collate  en_US.utf8                     
 ctype    en_US.utf8                     
 tz       America/Regina                 
 date     2020-06-29Packages ───────────────────────────────────────────────────────────────────
 package     * version  date       lib source        
 assertthat    0.2.1    2019-03-21 [1] CRAN (R 3.6.3)
 backports     1.1.8    2020-06-17 [1] CRAN (R 3.6.3)
 callr         3.4.3    2020-03-28 [1] CRAN (R 3.6.3)
 cli           2.0.2    2020-02-28 [1] CRAN (R 3.6.3)
 cluster       2.1.0    2019-06-19 [2] CRAN (R 3.6.3)
 colorspace    1.4-1    2019-03-18 [1] CRAN (R 3.6.3)
 crayon        1.3.4    2017-09-16 [1] CRAN (R 3.6.3)
 desc          1.2.0    2018-05-01 [1] CRAN (R 3.6.3)
 devtools      2.3.0    2020-04-10 [1] CRAN (R 3.6.3)
 digest        0.6.25   2020-02-23 [1] CRAN (R 3.6.3)
 dplyr         1.0.0    2020-05-29 [1] CRAN (R 3.6.3)
 ellipsis      0.3.1    2020-05-15 [1] CRAN (R 3.6.3)
 fansi         0.4.1    2020-01-08 [1] CRAN (R 3.6.3)
 fs            1.4.1    2020-04-04 [1] CRAN (R 3.6.3)
 generics      0.0.2    2018-11-29 [1] CRAN (R 3.6.3)
 ggplot2     * 3.3.1    2020-05-28 [1] CRAN (R 3.6.3)
 glue          1.4.1    2020-05-13 [1] CRAN (R 3.6.3)
 gridExtra     2.3      2017-09-09 [1] CRAN (R 3.6.3)
 gtable        0.3.0    2019-03-25 [1] CRAN (R 3.6.3)
 inline        0.3.15   2018-05-18 [1] CRAN (R 3.6.3)
 lattice     * 0.20-41  2020-04-02 [1] CRAN (R 3.6.3)
 lifecycle     0.2.0    2020-03-06 [1] CRAN (R 3.6.3)
 loo           2.2.0    2019-12-19 [1] CRAN (R 3.6.3)
 magrittr      1.5      2014-11-22 [1] CRAN (R 3.6.3)
 MASS          7.3-51.6 2020-04-26 [1] CRAN (R 3.6.3)
 Matrix        1.2-18   2019-11-27 [2] CRAN (R 3.6.3)
 matrixStats   0.56.0   2020-03-13 [1] CRAN (R 3.6.3)
 memoise       1.1.0    2017-04-21 [1] CRAN (R 3.6.3)
 mgcv          1.8-31   2019-11-09 [2] CRAN (R 3.6.3)
 munsell       0.5.0    2018-06-12 [1] CRAN (R 3.6.3)
 nlme          3.1-148  2020-05-24 [1] CRAN (R 3.6.3)
 permute     * 0.9-5    2019-03-12 [1] CRAN (R 3.6.3)
 pillar        1.4.4    2020-05-05 [1] CRAN (R 3.6.3)
 pkgbuild      1.0.8    2020-05-07 [1] CRAN (R 3.6.3)
 pkgconfig     2.0.3    2019-09-22 [1] CRAN (R 3.6.3)
 pkgload       1.1.0    2020-05-29 [1] CRAN (R 3.6.3)
 prettyunits   1.1.1    2020-01-24 [1] CRAN (R 3.6.3)
 processx      3.4.2    2020-02-09 [1] CRAN (R 3.6.3)
 ps            1.3.3    2020-05-08 [1] CRAN (R 3.6.3)
 purrr         0.3.4    2020-04-17 [1] CRAN (R 3.6.3)
 R6            2.4.1    2019-11-12 [1] CRAN (R 3.6.3)
 Rcpp          1.0.4.6  2020-04-09 [1] CRAN (R 3.6.3)
 remotes       2.1.1    2020-02-15 [1] CRAN (R 3.6.3)
 rlang         0.4.6    2020-05-02 [1] CRAN (R 3.6.3)
 rprojroot     1.3-2    2018-01-03 [1] CRAN (R 3.6.3)
 rstan       * 2.19.3   2020-02-11 [1] CRAN (R 3.6.3)
 scales        1.1.1    2020-05-11 [1] CRAN (R 3.6.3)
 sessioninfo   1.1.1    2018-11-05 [1] CRAN (R 3.6.3)
 StanHeaders * 2.21.0-3 2020-05-28 [1] CRAN (R 3.6.3)
 testthat      2.3.2    2020-03-02 [1] CRAN (R 3.6.3)
 tibble        3.0.1    2020-04-20 [1] CRAN (R 3.6.3)
 tidyselect    1.1.0    2020-05-11 [1] CRAN (R 3.6.3)
 usethis       1.6.1    2020-04-29 [1] CRAN (R 3.6.3)
 vctrs         0.3.1    2020-06-05 [1] CRAN (R 3.6.3)
 vegan       * 2.5-6    2019-09-01 [1] CRAN (R 3.6.3)
 withr         2.2.0    2020-04-20 [1] CRAN (R 3.6.3)

[1] /home/gavin/R/x86_64-redhat-linux-gnu-library/3.6
[2] /usr/lib64/R/library
[3] /usr/share/R/library

@jarioksa
Copy link
Contributor

I can't reproduce this either. Need more info.

@jarioksa
Copy link
Contributor

jarioksa commented Jun 30, 2020

Confirmed: this seems to happen when we use socket clusters in parallel evaluation (parallel::makeCluster). This happens usually only in Windows and therefore @gavinsimpson and I failed to reproduce the error (I tested in two operating systems, but neither was Windows). We can reproduce this by using socket clusters also in non-Windows-systems:

library(vegan)
data(varespec)
dis <- vegdist(varespec)
groups <- factor(c(rep(1,16), rep(2,8)), labels = c("grazed","ungrazed"))
mod <- betadisper(dis, groups)
cl <- parallel::makeCluster(2) ## to force the use of socket cluster in non-Windows systems
permutest(mod, parallel=cl) ## Error in Pstats[, 1] : incorrect number of dimensions

The error condition emerges because we use parRapply with socket clusters, and this drops dimensions of the result, and when we later try to drop the dimension again, we get an error of not having them, but only a a vector.

@jarioksa jarioksa added the bug label Jun 30, 2020
@gavinsimpson gavinsimpson self-assigned this Jun 30, 2020
@gavinsimpson
Copy link
Contributor

Thanks @jarioksa (shows the importance of a fully reproducible example). I'll take a look.

@jarioksa
Copy link
Contributor

jarioksa commented Jun 30, 2020

@gavinsimpson : an obvious solution is to use drop(Pstats) instead of Pstats[,1], but that would fail if Pstats has more than one column (I don't know if that can happen). Then there are dirty ifs (but they may be more robust).

Yes, and perhaps we should find a Windows test bed as well. The rstan was not an issue.

It seems that we have had this bug in vegan since release 2.2-0 (Nov 2014), but no Windows-user had reported problems.

@jarioksa jarioksa added this to the 2.5-7 milestone Nov 12, 2020
@jarioksa
Copy link
Contributor

We are forced to have a new release of vegan (see #382), and this is a release critical bug. How to fix this @gavinsimpson ?

@gavinsimpson
Copy link
Contributor

@jarioksa were you given a timeline for submitting to CRAN? I'll look at this by the weekend unless we need a real ease sooner?

@jarioksa
Copy link
Contributor

@gavinsimpson no deadline. This is a problem in the future R (now R-devel) and may not be very urgent yet. However, I was working for a release, but upon checking issues, noticed this one that I had forgot, and think should be fixed for the release. So you can quite well postpone this to the weekend (and my best greetings to the family).

jarioksa added a commit that referenced this issue Nov 25, 2020
This is a quick fix for issue #369 made only in cran-2.5 branch:
master branch may have a more fundamental fix later, but we needed
this because vegan 2.5-7 must be released for other reasons.
jarioksa added a commit that referenced this issue Nov 25, 2020
jarioksa added a commit that referenced this issue Nov 26, 2020
The previous quick fix f368bbb failed with pairwise=TRUE:
parRapply always drops dimensions, also in pairwise=TRUE which
should return a matrix. It was replaced with parApply that can
return a matrix, but transposed to our need. Now we take care
in the socket cluster block that we always get similar matrices
in all cases (non-parallel, fork cluster, socket cluster). Also
added drop=TRUE when getting the first column to protect us against
Hadleyan changes in R (like recent change in stringsAsFactors default).
@jarioksa
Copy link
Contributor

It looks to me that commit fae3f42 fixes the issues. It is currently only in the cran-2.5 branch and I presume that it will be included in the next vegan release in CRAN. However, it is not merged in the master branch (2.6-0 versions), but if there are no better ideas, it can be cherry-picked some day.

gavinsimpson pushed a commit that referenced this issue Mar 9, 2022
The previous quick fix f368bbb failed with pairwise=TRUE:
parRapply always drops dimensions, also in pairwise=TRUE which
should return a matrix. It was replaced with parApply that can
return a matrix, but transposed to our need. Now we take care
in the socket cluster block that we always get similar matrices
in all cases (non-parallel, fork cluster, socket cluster). Also
added drop=TRUE when getting the first column to protect us against
Hadleyan changes in R (like recent change in stringsAsFactors default).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

3 participants