Text in Supp Mat 1 updated for clairty as per reviewer comment

supp_mat/SM1_summary.qmd

@@ -372,11 +372,7 @@ ArticleReviewSummary2<-summarise(article_grouped_reviews2,
 
 ### Number of analyses of different types
 
-As described in the summary statistics section of the manuscript, `r Table1[1,4]` teams submitted `r Table1[1,3]` $Z_r$ model estimates and `r Table1[2,4]` teams submitted `r Table1[2,3]` out of sample predictions for the blue tit dataset.
-Similarly, `r Table1[3,4]` submitted `r Table1[3,3]` $Z_r$ model estimates and `r Table1[4,4]` teams submitted `r Table1[4,3]` out of sample predictions for the *Eucalytpus* dataset.
-The majority of the blue tit analyses specified normal error distributions and were non-Bayesian mixed effects models.
-Analyses of the *Eucalyptus* dataset rarely specified normal error distributions, likely because the response variable was in the form of counts.
-Mixed effects models were also common for *Eucalytpus* analyses (@tbl-Table1).
+As described in the summary statistics section of the manuscript, `r Table1[1,4]` teams submitted `r Table1[1,3]` $Z_r$ model estimates and `r Table1[2,4]` teams submitted `r Table1[2,3]` out of sample predictions for the blue tit dataset. Similarly, `r Table1[3,4]` submitted `r Table1[3,3]` $Z_r$ model estimates and `r Table1[4,4]` teams submitted `r Table1[4,3]` out of sample predictions for the *Eucalytpus* dataset. The majority of the blue tit analyses specified normal error distributions and were non-Bayesian mixed effects models. Analyses of the *Eucalyptus* dataset rarely specified normal error distributions, likely because the response variable was in the form of counts. Mixed effects models were also common for *Eucalytpus* analyses (@tbl-Table1).
 
 ```{r}
 #| label: tbl-Table1
@@ -410,8 +406,7 @@ Table1 %>%
 
 ### Model composition
 
-The composition of models varied substantially (@tbl-Table2) in regards to the number of fixed and random effects, interaction terms and the number of data points used.
-For the blue tit dataset, models used up to `r Table2[1,7]` fixed effects, `r Table2[6,7]` random effects, and `r Table2[3,7]` interaction terms and had sample sizes ranging from `r Table2[7,6]` to `r Table2[7,7]` For the Eucalyptus dataset models had up to `r Table2[9,7]` fixed effects, `r Table2[13,7]` random effects, `r Table2[11,7]` interaction terms and sample sizes ranging from `r Table2[15,6]` to `r Table2[15,7]`.
+The composition of models varied substantially (@tbl-Table2) in regards to the number of fixed and random effects, interaction terms and the number of data points used. For the blue tit dataset, models used up to `r Table2[1,7]` fixed effects, `r Table2[6,7]` random effects, and `r Table2[3,7]` interaction terms and had sample sizes ranging from `r Table2[7,6]` to `r Table2[7,7]` For the Eucalyptus dataset models had up to `r Table2[9,7]` fixed effects, `r Table2[13,7]` random effects, `r Table2[11,7]` interaction terms and sample sizes ranging from `r Table2[15,6]` to `r Table2[15,7]`.
 
 ```{r}
 #| label: tbl-Table2
@@ -448,8 +443,7 @@ Table2 %>%
 
 ### Choice of variables
 
-The choice of variables also differed substantially among models (@tbl-Table3).
-Considering all submitted analyses, the blue tit dataset had `r length(unique(BTVariables$rowname))` candidate variables, which were used in a mean of `r round(Table3[1,3],digits=2)` $Z_r$ analyses (range `r Table3[1,5]`- `r Table3[1,6]`), the *Eucalyptus* dataset had `r length(unique(EucVariables$rowname))` candidate variables which were used in a mean of `r round(Table3[2,3],digits=2)` $Z_r$ analyses (range `r Table3[2,5]`-`r Table3[2,6]`).
+The choice of variables also differed substantially among analyses (@tbl-Table3). The blue tit dataset had a total of `r length(unique(BTVariables$rowname))` candidate variables, each of which were used in at least one analysis. These variables were included in a mean of `r round(Table3[1,3],digits=2)` $Z_r$ analyses (range `r Table3[1,5]`- `r Table3[1,6]`). The *Eucalyptus* dataset had a total of `r length(unique(EucVariables$rowname))` candidate variables, all of which were used in at least one analysis. The variables in the *Eucalyptus* included in a mean of `r round(Table3[2,3],digits=2)` $Z_r$ analyses (range `r Table3[2,5]`-`r Table3[2,6]`).
 
 ```{r}
 #| label: tbl-Table3
@@ -484,17 +478,11 @@ Table3 %>%
 
 ## Effect Size Specification Analysis
 
-We used a specification curve [@simonsohn2015] to look for relationships between $Z_r$ values and several modeling decisions, including the choice of independent and dependent variable, transformation of the dependent variable, and other features of the models that produced those $Z_r$ values (@fig-specr-bt, @fig-specr-euc).
-Each effect can be matched to the model features that produced it by following a vertical line down the figure.
+We used a specification curve [@simonsohn2015] to look for relationships between $Z_r$ values and several modeling decisions, including the choice of independent and dependent variable, transformation of the dependent variable, and other features of the models that produced those $Z_r$ values (@fig-specr-bt, @fig-specr-euc). Each effect can be matched to the model features that produced it by following a vertical line down the figure.
 
 ### Blue tit
 
-We observed few clear trends in the blue tit specification curve (@fig-specr-bt).
-The clearest trend was for the independent variable 'contrast: reduced broods vs. unmanipulated broods' to produce weak or even positive relationships, but never strongly negative relationships.
-The biological interpretation of this pattern is that nestlings in reduced broods averaged similar growth to nestlings in unmanipulated broods, and sometimes the nestlings in reduced broods even grew less than the nestlings in unmanipulated broods.
-Therefore, it may be that competition limits nestling growth primarily when the number of nestlings exceeds the clutch size produced by the parents, and not in unmanipulated broods.
-The other relatively clear trend was that the strongest negative relationships were never based on the independent variable 'contrast: unmanipulated broods vs. enlarged broods'.
-These observations demonstrate the potential value of specification curves.
+We observed few clear trends in the blue tit specification curve (@fig-specr-bt). The clearest trend was for the independent variable 'contrast: reduced broods vs. unmanipulated broods' to produce weak or even positive relationships, but never strongly negative relationships. The biological interpretation of this pattern is that nestlings in reduced broods averaged similar growth to nestlings in unmanipulated broods, and sometimes the nestlings in reduced broods even grew less than the nestlings in unmanipulated broods. Therefore, it may be that competition limits nestling growth primarily when the number of nestlings exceeds the clutch size produced by the parents, and not in unmanipulated broods. The other relatively clear trend was that the strongest negative relationships were never based on the independent variable 'contrast: unmanipulated broods vs. enlarged broods'. These observations demonstrate the potential value of specification curves.
 
 ```{r calc_MA_mod_coefs-2, eval=TRUE, cache = FALSE, eval = TRUE, warning=FALSE, message = FALSE}
 # knitr::read_chunk(here::here("index.qmd"), labels = "calc_MA_mod_coefs")
@@ -691,9 +679,7 @@ cowplot::plot_grid(curve_bt, specs_bt, samp_size_hist_bt,
 
 ### *Eucalyptus*
 
-In the *Eucalyptus* specification curve, there are no strong trends (@fig-specr-euc).
-It is, perhaps, the case that choosing the dependent variable 'count of seedlings 0-0.5m high' corresponds to more positive results and choosing 'count of all *Eucalytpus* seedlings' might find more negative results.
-Choosing the independent variable 'sum of all grass types (with or without non-grass graminoids)' might be associated with more results close to zero consistent with the absence of an effect.
+In the *Eucalyptus* specification curve, there are no strong trends (@fig-specr-euc). It is, perhaps, the case that choosing the dependent variable 'count of seedlings 0-0.5m high' corresponds to more positive results and choosing 'count of all *Eucalytpus* seedlings' might find more negative results. Choosing the independent variable 'sum of all grass types (with or without non-grass graminoids)' might be associated with more results close to zero consistent with the absence of an effect.
 
 ```{r}
 #| label: fig-specr-euc
@@ -796,9 +782,7 @@ cowplot::plot_grid(curve_euc, specs_euc, samp_size_hist_euc,
 
 ### Post-hoc analysis: Exploring the effect of removing analyses with poor peer-review ratings on heterogeneity
 
-The forest plots in @fig-all-forest-plots-Zr compare the distributions of $Z_r$ effects from our full set of analyses with the distributions of $Z_r$ effects from our post-hoc analyses which removed either analyses that were reviewed at least once as being 'unpublishable' or analyses that were reviewed at least once as being 'unpublishable' or requiring 'major revisions'.
-Removing these analyses from the blue tit data had little impact on the overall distribution of the results.
-For the *Eucalytpus* analyses, removing 'unpublishable' analyses meant dropping the extreme outlier 'Brooklyn-2-2-1' which made a substantial difference to the amount of observerd deviation from the meta-analytic mean.
+The forest plots in @fig-all-forest-plots-Zr compare the distributions of $Z_r$ effects from our full set of analyses with the distributions of $Z_r$ effects from our post-hoc analyses which removed either analyses that were reviewed at least once as being 'unpublishable' or analyses that were reviewed at least once as being 'unpublishable' or requiring 'major revisions'. Removing these analyses from the blue tit data had little impact on the overall distribution of the results. For the *Eucalytpus* analyses, removing 'unpublishable' analyses meant dropping the extreme outlier 'Brooklyn-2-2-1' which made a substantial difference to the amount of observerd deviation from the meta-analytic mean.
 
 ```{r}
 #| label: fig-all-forest-plots-Zr