Improvements to Effect Size (Eta2, Omega2) Indices for ANOVAs

[DominiqueMakowski]

Combining several issues here:

• Support repeated measures ANOVAs (Remove isFALSE #84, sjstats and parameters eta and omega squared functions different? parameters#100):
  • make it work for anova(themixedmodel)
  • make it work for anova(aovlist)
  • Make sure these are correct
• Generalized eta2 and Omega2 (effect size paper #91)
  • https://www.frontiersin.org/articles/10.3389/fpsyg.2013.00863/full
• Adjusted eta2
  • https://journals.sagepub.com/doi/abs/10.1177/2515245919855053?journalCode=ampa

Does anybody know an expert of ANOVAs that could help us going through this?

[IndrajeetPatil]

@DominiqueMakowski Maybe these functions can also be moved to the new effectsize package?

[DominiqueMakowski]

I don't see why not!

[mattansb]

I agree - effectsize seems like the perfect place for all r, d, OR, and vaf measures!

[DominiqueMakowski]

I'll need help 🙉

[IndrajeetPatil]

Until these generalized functions are implemented in effectsize itself, we can probably just move (literally copy-paste) the eta- and partial-omega squared functions to effectsize and re-export them in parameters?
(Of course, this means that effectsize will have to go on CRAN soon, so this can wait if there is a new version of parameters in the offing.)

All the new development for these new generalized functions can then happen in effectsize itself.

[DominiqueMakowski]

Yup, I just wonder whether the eta2 & co functions need stuff from parameters that would need to be transferred too.

For instance in parameters a line added the SE to the object:

effectsize/R/standardize_parameters.R

Line 145 in 28fa79c

# Standardize SE if possible

But I had to deactivate it because it requires standard_error, which is parameters... Not sure how to proceed in such cases. Do we move standard_error as well? But standard_error relies in some cases on the whole model_parameters. So we need to find a better way...

[mattansb]

Perhaps under effectsize some of these functions would be stripped down a bit (e.g., .standardize_parameters_posthoc wouldn't return standardized SEs?).

Additionally, some functions can appear, maybe internally / not-exported, in more than 1 package, if need be (obviously this should be avoided as much as possible).

[DominiqueMakowski]

Additionally, some functions can appear, maybe internally / not-exported, in more than 1 package, if need be (obviously this should be avoided as much as possible).

this conflicts a lot with my "neatness" obsession ^^. more importantly, it becomes hard to maintain and might reflect a clunky organisation (and we are well organized aren't we) . But Maybe standard_error could be moved to insight as it is mainly a "retrieval" function?

[strengejacke]

SE for "classic" standardized parameters is useful for calculating the related CI's around the standardized parameters. That's why I added that code lines...

[strengejacke]

Why do we need standardization methods for effect sizes? Else, can't we just make parameters a dependency of effectsize?

[DominiqueMakowski]

SE for "classic" standardized parameters is useful for calculating the related CI's around the standardized parameters. That's why I added that code lines...

I know, I know, and I believe it is indeed super useful. That's why we need to find a way to make it work

Why do we need standardization methods for effect sizes

Some indices effect sizes are related to standardization

Else, can't we just make parameters a dependency of effectsize?

it's more natural if effectsize is a dep of parameters

I'll think about a solution

[DominiqueMakowski]

after looking a bit more at the code for SE, eta2 omega2 etc. etc. they are really intertwined with the core of the parameters package (relying on the extract_parameters_* internals. Seems hard to move them from here indeed...

Simplest way is, indeed, that parameters becomes a deps of effectsize... but will it be possible then to easily have effectsize imported in parameters? I.e., circular dependencies?

[IndrajeetPatil]

Circular dependencies are not allowed by CRAN, so that’s definitely not an option.

I think the best current solution is for effectsize to rely on parameters.

[mattansb]

If effectsize depends on parameters, that would mean that parameters's functions can't "pull" the effect sizes into the various model_parameters() functions.

But, say we would be able to have all the effect size functions in effectsize with dependency on parameters, this would mean that parameters would have to be re-structured... Not sure that's a bad thing? Right now parameters has a lot of stuff, only some of it seems directly related to a model's parameters:

• Model parameter extraction
• parameter diagnostic stuff that varies by model type
• SEs
• CIs
• Bootstrapping
• Standardized parameters (beta-slopes - maybe more suitable in effectsize?)
• VAF effect sizes (partial/adjusted (partial) eta/omega/epsilon/f squared) (would be in effectsize)
• Stuff relating to feature reduction (maybe more suitable in correlation?)
• Index Conversion (maybe more suitable in effectsize?)
• Table and Value Formatting (maybe more suitable in report?)

[DominiqueMakowski]

I agree that parameters becoming a dep of effectsize is problematic as the former needs to pull effect sizes from effectsize, especially in the current state where parameters is a mid-to-high level package.

One solution would indeed to lower the level of parameters, to re-scope it on functions surrounding parameters extraction and computation.

We could have parameter and performance in the middle of the hierarchy and effectsize on the side, providing functions to complete and/or interpret the output of both. On top of that, report (which is being more and more stripped down of features) would finally find its role as the integrator: it would run model_parameters (in parameters), model_performance (in performance), interpret_parameters and interpret_performance (to be implemented in effectsize) and would merge it all it in a nicely formatted table. Et voilà. What do you think?

(Side note, maybe if we move some stuff (feature reduction) to correlation it might good to rename it, maybe something like "multivariate" or something that would encompass these kind of things.)