anova.cca by="margin" inconsistent with earlier version

[battusboy]

The anova.cca function in version 2.5-2 is giving answers (p-values) that are inconsistent with earlier versions (and don't make sense to me) when using by="margin". In particular, when using by="margin" things are showing up as "significant" that seem erroneous.

Below is code and results using version 2.4-6 and 2.5-2.

library(vegan)

set.seed(2)
dat<-matrix(rnorm(90),ncol=9)
x<-c(rep("A",5),rep("B",5))

mod<-rda(dat~x)

vegan version 2.4-6

set.seed(2);anova(mod)
set.seed(2);anova(mod, by="margin")

 > set.seed(2);anova(mod)
 Permutation test for rda under reduced model
 Permutation: free
 Number of permutations: 999

 Model: rda(formula = dat ~ x)
 Df Variance      F Pr(>F)
 Model     1   1.0949 0.9649  0.518
 Residual  8   9.0779

 > set.seed(2);anova(mod, by="margin")
 Permutation test for rda under NA model
 Marginal effects of terms
 Permutation: free
 Number of permutations: 999

 Model: rda(formula = dat ~ x)
 Df Variance      F Pr(>F)
 x         1   1.0949 0.9649  0.518
 Residual  8   9.0779
 >

vegan version 2.5-2

set.seed(2);anova(mod)
set.seed(2);anova(mod, by="margin")

 > set.seed(2);anova(mod)
 Permutation test for rda under reduced model
 Permutation: free
 Number of permutations: 999

 Model: rda(formula = dat ~ x)
 Df Variance     F Pr(>F)
 Model     1   1.3924 1.031  0.456
 Residual  8  10.8045

 > set.seed(2);anova(mod, by="margin")
 Permutation test for rda under reduced model
 Marginal effects of terms
 Permutation: free
 Number of permutations: 999

 Model: rda(formula = dat ~ x)
 Df Variance     F Pr(>F)
 x         1   1.3924 1.031   0.05 *
   Residual  8  10.8045

  Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

[jarioksa]

You have noticed that it makes no sense to have marginal test with only one variable? It did not occur to me that somebody would do so, and it seems that we do not handle this case gracefully. At the moment the best thing you can do is that you don't have marginal tests when you have only one variable and hence no marginal effects. I'll fix this, though.

[battusboy]

Yes, I’m fully aware it makes no sense to do so. I noticed it when I was explaining to a student how to use the package - in particular what “by=margin” was doing. The result from the previous version made sense since if only one term in is the model, it will be the last thing entered in the model. For me it was all about pedagogy. Thanks for maintaining a great package! Jim

…

On Sep 20, 2018, at 4:08 PM, Jari Oksanen ***@***.***> wrote: You have noticed that it makes no sense to have marginal test with only one variable? It did not occur to me that somebody would do so, and it seems that we do not handle this case gracefully. At the moment the best thing you can do is that you don't have marginal tests when you have only one variable and hence no marginal effects. I'll fix this, though. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#291 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ANYYMZlL8ju0jrhhqwu2pOVRBFRvF_Rpks5uc_WogaJpZM4WylmZ>.

[jarioksa]

We re-wrote completely constrained ordination and permutations tests for vegan 2.5-1, and a tiny detail in the ordination infrastructure caused this bug. The marginal effects are tested by removing each variable in turn and seeing how the full model changes. If there is only one constraining variable, it makes no sense to drop it because the remaining model is unconstrained, and the constraint matrix has zero columns. We did not have any guard of this in 2.4-6 either, but incidentally the model was handled correctly, but 2.5-1 needed a check against such models. Suggested fix is in commit eefb4a7.

[jarioksa]

Closed with commit 6f28cda