RoBMA results interpretation

[tressoldi]

Dear Frantisek,
how can I interpret the results obtained from one of my meta-analysis by using the syntax below?
In particular, I do not know how to interpret the PET and PEESE means. Are their values suggesting a positive mu estimate?

Thanks
Patrizio

RoBMA(y = data$ESg, se = data$SEg, study_names = data$Author,
priors_effect = prior("normal", parameters = list(mean = 0.15,
sd = 0.1)), priors_heterogeneity = prior("invgamma",
parameters = list(shape = 1, scale = 0.15)), priors_effect_null = prior("spike",
parameters = list(location = 0)), priors_bias_null = (prior_none()))

Robust Bayesian meta-analysis
Components summary:
Models Prior prob. Post. prob. Inclusion BF
Effect 18/36 0.500 0.170 0.204
Heterogeneity 18/36 0.500 1.000 Inf
Bias 32/36 0.500 0.990 97.314

Model-averaged estimates:
Mean Median 0.025 0.975
mu -0.005 0.000 -0.109 0.034
tau 0.168 0.168 0.135 0.206
omega[0,0.025] 1.000 1.000 1.000 1.000
omega[0.025,0.05] 1.000 1.000 1.000 1.000
omega[0.05,0.5] 1.000 1.000 1.000 1.000
omega[0.5,0.95] 1.000 1.000 1.000 1.000
omega[0.95,0.975] 1.000 1.000 1.000 1.000
omega[0.975,1] 1.000 1.000 1.000 1.000
PET 0.265 0.000 0.000 1.425
PEESE 2.790 3.607 0.000 5.704

[FBartos]

Hi Patrizio,

the PET and PEESE estimates correspond to the PET and PEESE regression coefficients from PET-PEESE adjustment for publication bias - they are auxiliary parameters of the regression and they are not indicative of positive/negative effect size estimate.

The PET model adjusts the relationship between effect sizes and standard errors - positive PET coefficient means that the standard errors and effect sizes are linearly related.
The PEESE model adjusts for the relationship between effect sizes and standard errors squared - positive PEESE coefficient means that the standard errors and effect sizes are quadratically related.

Both of them are indicators of publication bias / small study effects. The effect size is fully captured by the mu coefficient.
We describe the updated RoBMA-PSMA in more detail in our new preprint: https://psyarxiv.com/kvsp7/

Cheers,
Frantisek

[tressoldi]

Clarified.
My suggestion is to separate the estimates of the effect size and tau from those related to the publication bias tests in the results, with a brief explanation of their meaning.
Thank you.
Patrizio

[FBartos]

Glad it's clear now.

Also thanks for the suggestion. I will think about it. The other important aspect is keeping the summary tables concise. The printed output is already large and details regarding interpretation should be addressed in documentation, vignettes, and papers. I will see whether I come up with something that could help.

How about adding an interpret() function that would provide a textual summary of the output in plain English? E.g.:

The RoBMA-PSMA model ensemble found weak evidence favoring the presence of the effect, BF[10] = 3.15, with the model-averaged effect size estimate Cohen's d = 0.130, 95% CI [0.05, 0.25]. There was strong evidence against the heterogeneity, BF[fr] = 15.10, with the ....

[tressoldi]

I like this type of narrative explanations that, apart giving a correct interpretation of the results, can be copied and pasted in the research papers.

Best
Patrizio

[FBartos]

OK, I will implement something like that. It will however take some time before the new version goes to CRAN since it was just updated.

Cheers,
Frantisek

[FBartos]

Hi Patrizio,

I just pushed an update that adds the interpret() function. You can download the developmental version of the package with:

devtools::install_github("fbartos/RoBMA")

and check it out.

Cheers,
Frantisek

[tressoldi]

Hi Frantisek,
I've downloaded and fitted a meta-analytic model ensemble.
However with the following command:

interpret(fit)
I got
Error: 'interpret' is not an exported object from 'namespace:BayesTools'

[FBartos]

oh, I see the issue, I forgot to tell you that you also need to update the BayesTools package.

devtools::install_github("fbartos/BayesTools")

Let me know if that fixes the issue.

[tressoldi]

Fixed, thank you.

Best
Patrizio