Bug in p-value calculations in adonis

[fra-gh]

The p-value calculated by adonis is sometimes wrong.
For example, the following code

library("vegan")
  options(digits = 12)
  g = as.factor(c("Control","Control","Group1", "Group2", "Group2"))
  d = as.dist(matrix(c(0,3,7,2,1, 3,0,5,4,1, 7,5,0,2,6, 4,2,4,0,2, 1,1,6,2,0), ncol=5))
  pp = adonis(d ~ g)
  pp

returns

Call:
adonis(formula = d ~ g) 

Permutation: free
Number of permutations: 120

Terms added sequentially (first to last)

          Df SumsOfSqs MeanSqs     F.Model           R2  Pr(>F)
g          2      23.3   11.65 3.584615385 0.7818791946 0.20833
Residuals  2       6.5    3.25             0.2181208054        
Total      4      29.8                     1.0000000000  

The p-value is wrong (only 24 permutations with high f instead of 39). The correct p-value can be computed from the definition to be

(rowSums(t(pp$f.perms >= pp$aov.tab$F.Model[1]))+1) / (length(pp$f.perms) + 1)

which correctly returns

[1] 0.333333333333

It looks like the bug is due to rounding. When comparing f.perms and F.mod, the value of f.perms is rounded but the value of F.mod is not (although it is rounded afterwards). This makes the >= comparison fail in this example because the value of f.perms is rounded down.

[jarioksa]

Confirmed: this is a bug. Fixed in commit f727b58.

Thanks for reporting this.

[fra-gh]

thanks for the very prompt reply. keep up the good work!

[jarioksa]

PR #136 has a better fix for this issue.

[chitrabaniya]

Do we always need to calculate p-value separately or it goes automatically fixed?
I got surprised with different p-values.