RM ANOVA: Estimated Marginal Means gives One Standard Error for all conditions

[mwegrzyn]

Dear Jamovi Team,

using the repeated measures ANOVA and generating a figure/table using "Estimated Marginal Means", I find that only one global SE/CI is computed for all conditions, even when their distributions are vastly different. See the following example:

( Jamovi 0.9.5.17 on Debian, installed using flatpak)

I feel that it is more useful if each condition has its own SE/CI.

Best,
Martin

[jonathon-love]

hi,

so the neat thing about emmeans is that they're based on the model you are fitting. if the model doesn't allow group 1 to have a much bigger SE/CI than group 2, then the emmeans won't either.

in this case, it clearly illustrates that the model doesn't fit the data.

makes sense?

jonathon

[R180]

I think another way to put it is that ANOVA, whether repeated-measures or not, assumes equal variances. So there is only one error variance, pooled across conditions. Therefore as long as n is equal for all conditions, the standard errors [s/sqrt(n)] and confidence intervals will also be equal across all conditions. While this may be a correct way to do things, it's not the scientific practice, at least not in experimental psychology. Unless we are plotting regression functions, we plot the actual means, and confidence intervals based on the actual variance for each condition.

JASP comes closer to supporting this practice (ANOVA plots show group-specific CIs, and repeated-measures ANOVA plots give the user the option to pool or not to pool the variances). For now, I think I'll need to modify the undergrad stats course I'm currently teaching, so that we use SPSS or JASP for plotting (we have not gotten to ANOVA yet, so the pooled variance issue hasn't come up yet).

But as a new feature request: I would suggest that jamovi allow the user to choose whether to have the plots use pooled error variances (which isn't wrong) or un-pooled error variance (which is the general practice).

[jonathon-love]

i'd recommend an approach where the data is first explored and visualised using descriptives, before fitting the model.

While this may be a correct way to do things

i'd probably go further, and say this is the correct way to do things -- but there's more to emmeans than pooled vs unpooled variance. more info here:

https://cran.r-project.org/web/packages/emmeans/vignettes/basics.html

but it is an awkward balancing act for us -- wanting to provide methods that people want, but not wanting to reinforce bad practice (and not wanting to overwhelm the user with options).

jonathon

[R180]

I sympathize with your competing goals. . . .

(a) I'm not sure I agree with the way the emmeans package is documented. From the point of view of an experimental psychologist, emmeans is at its core (without having covariates in the model) just "Calculate means in the normal way, so if you collapse across levels of of a factor, calculate means OF MEANS so that smaller-n means count just as much as larger-n means." It only gets "tricky," or "special" when you have covariates, but that's always the case with ANCOVA.

(b) I'm fine with having a set of "descriptives" plotting routines that are model free, and that might be employed prior to fitting a model. However, jamovi's "descriptives" visualizations are focused on box plots. No means, standard deviations, standard errors, or confidence intervals. So maybe adding some of those things would help. (See the attached example of a kind of descriptives plot I like to make, but wish I could make without having to do so much coding in ggplot2.)

Oh, and the plot below has a repeated-measures factor on the horizontal axis; but I'm not sure jamovi's "descriptives" plots work elegantly for repeated-measures experimental designs.

[jonathon-love]

oh yup.

(a) ... you still do have the 'equal cell weights' option

(b) yeah, that is true. i've been wanting to add more plot options along those lines

(c) yeah, we have discussed adding vanilla descriptives to the RM ANOVA - i'll raise the issue again.

thanks for your input.

jonathon