solomonR 0.8.2

A patch release with one fix. If you have used perm_solomon() with an outcome that has tied scores (a binary outcome, counts, ratings, rounded scores), or with very small groups, rerun those analyses.

The fix: ties in the permutation test (#131)

At the participant level, perm_solomon() let rounding error decide whether a permutation whose statistic equals the observed one was counted as at least as extreme. The validity of the p-value is shown for the count of statistics at least as extreme as the observed one (Phipson & Smyth, 2010), so such ties must be counted. When they were missed, the p-value was too small.

• Tied scores, at any sample size. With a binary outcome and 20 participants per group, one test gave p = .017 where the correct value is .056.
• Very small groups. With groups of 4, 4, 3, and 3 and continuous scores, one test gave p = .004 where the correct value is .117.
• Continuous scores with 10 or more per group. A tie is rare. For a contrast within one pretest condition, about 1 analysis in 100 with 999 permutations meets one, and its p-value rises by at most .001.
• An observed statistic of exactly zero now gives a p-value of 1, at the participant and the cluster level.
• Cluster-level tests otherwise give the same results as before.

Ties are now counted at the participant and the cluster level and in plot_perm() by one rule. The fix was reviewed independently three times against exact enumeration and closed-form p-values, and this release was checked against the same references.

What remains

These are tracked in #134.

• Models fitted by iteration (for example logistic or Poisson), where the contrast's cells include a pretest covariate: a tie can still be missed. In the checks made, no p-value was affected by this with the logit or the Poisson log link.
• A pretest or covariate far from zero. This release line does not center them. When the mean is about a thousand times the standard deviation or more, as with a calendar year, ties can still be lost. Results were right in every check with a pretest mean up to 500 times its standard deviation.
• Degenerate cases: an observed labeling that empties an arm, and an outcome that does not vary within a pretest condition.

Everything else

No other function changes from 0.8.1. Within perm_solomon(), the refitted models may take up to 200 iterations where they took 25, which removes convergence warnings that some models gave. The development version, which is working toward v1.0.0, has the same fix.

Install

install.packages(
  "solomonR",
  repos = c(
    JUhalt = "https://juhalt.r-universe.dev",
    CRAN = "https://cloud.r-project.org"
  )
)

Reference

Phipson, B., & Smyth, G. K. (2010). Permutation p-values should never be zero: Calculating exact p-values when permutations are randomly drawn. Statistical Applications in Genetics and Molecular Biology, 9(1), Article 39. https://doi.org/10.2202/1544-6115.1585

solomonR 0.8.1

solomonR 0.8.1

A patch release for v0.8.0.

Bug fix: refit affected SEM analyses

• The bug. In 0.8.0, fit_solomon_sem() and fit_solomon_sem_latent() gave wrong contrasts when the data did not begin with a pretested, treated participant (#119). Their models label the four group means by position, and lavaan orders groups by their first appearance in the data, so the labels could attach to the wrong groups.
• An example. On solomon_example with an unpretested control first, the average treatment effect came out as −2.68 instead of 2.68.
• The fix. The group order is now fixed (#120).
• What to refit. Results from data that began with a pretested, treated participant, including every documented example, are unchanged. Refit any SEM analysis whose data began with another kind of participant.

Also included

• Attribution (#102). The default post hoc tests of fit_solomon_steyn() are credited to Scheffé (1953), whose method they are, as well as to Steyn (2005). ?steyn2005 cites Steyn and Mynhardt (2008), the English report of the study.
• Sources (#103). Lessac and Solomon (1969) is added. The coverage article records the sources Steyn names that were not read: Kerlinger's textbook, whose points are all stated in primary sources the package cites, and Steyn (2001), of which no copy was found.
• CRAN readiness (#103). The README's license and roadmap links now work inside the built package, and DESCRIPTION names the American Psychological Association in full.

Installation

install.packages(
  "solomonR",
  repos = c(
    JUhalt = "https://juhalt.r-universe.dev",
    CRAN = "https://cloud.r-project.org"
  )
)

or

pak::pak("JUhalt/solomonR@v0.8.1")

References

Lessac, M. S., & Solomon, R. L. (1969). Effects of early isolation on the later adaptive behavior of beagles: A methodological demonstration. Developmental Psychology, 1(1), 14–25. https://doi.org/10.1037/h0026778

Scheffé, H. (1953). A method for judging all contrasts in the analysis of variance. Biometrika, 40(1–2), 87–104. https://doi.org/10.1093/biomet/40.1-2.87

Steyn, R. (2005). Self-evaluasie en die vorming van selfdoeltreffendheidspersepsies [Self-evaluation and the forming of self-efficacy perceptions] [Doctoral thesis, University of South Africa]. Unisa Institutional Repository. https://hdl.handle.net/10500/1745

Steyn, R., & Mynhardt, J. (2008). Factors that influence the forming of self-evaluation and self-efficacy perceptions. South African Journal of Psychology, 38(3), 563–573. https://doi.org/10.1177/008124630803800310

solomonR 0.8.0

solomonR 0.8.0

solomonR 0.8.0 brings together the work planned for v0.5.0 through v0.8.0, and the stable interface (#83) and the first CRAN-readiness changes (#84) planned for v0.9.0. Versions 0.5.0, 0.6.0, and 0.7.0 were not released separately. The full notes are in NEWS.

Highlights

Outcomes and designs

• Binary and count outcomes (#43, #44, #62).
  • Risk differences, risk ratios, and odds ratios, or rate ratios and rate differences, for each Solomon contrast, with a negative-binomial option for overdispersed counts.
  • The historical chi-square and Fisher path, fisher_solomon().
  • Marginal risk and rate contrasts for clustered fits (#64), marginal_solomon().
• Designs with several treatments (#45).
  • fit_solomon_glm(control = , contrasts = ) analyzes all 2(k + 1) groups in one model, with omnibus tests and Holm-adjusted comparisons.
  • fit_solomon_steyn() carries out Steyn's (2009) sequence of tests.
  • The analysis of several treatments is experimental: its simulation study did not meet the pre-specified rule for error control with three treatments and 10 participants per group.
• Clustered designs (#46, #19).
  • validate_solomon(cluster = ) detects clusters confounded with conditions.
  • perm_solomon() permutes whole clusters, with an exact p-value when every allocation can be enumerated.
• Nonrandomized designs (#58).
  • baseline_solomon() compares the pretested arms at baseline.
  • report_solomon(), told the design was not randomized, reports differences between groups rather than treatment effects, and names the threats that random assignment would have controlled.
• Several posttest occasions (#57).
  • fit_solomon_mmrm() is a mixed model for repeated measures that estimates the Solomon contrasts at each occasion and the change in sensitization. It is experimental.
  • "Worked Example: Repeated Posttests" reproduces a published study with three occasions (jordaan2014).
• Missing posttests (#82). fit_solomon_mi() and tipping_point_solomon() provide multiple imputation and a delta-adjusted tipping-point analysis. Both are experimental.

History, teaching, and reporting

• History (#48, #50, #51, #78).
  • The 1988, 1990, and 1995 versions of the historical test sequence, fit_solomon_classic(flow = ).
  • A replication of their published error rates.
  • Solomon's (1949) own analysis, fit_solomon_1949(), and his data, solomon1949.
• Published data sets (#54). mai2020, with individual data, and the group statistics elkarkri2025a, jordaan2014, kvalem1996, lana1959, solomon1949, and steyn2005. Each reproduces its source's published analyses, and mai2020 is followed through a full worked example.
• Reporting and synthesis (#52, #53).
  • report_solomon() drafts APA 7 results text with method-specific references.
  • solomon_from_summary() reanalyzes published group statistics.
  • solomon_effect_sizes() gives meta-analytic effect sizes.
• Teaching and planning (#49, #79, #81).
  • A planning article with literature-based planning values.
  • simulate_solomon() simulates data with known effects, for classroom exercises.
  • analysis_plan_solomon() writes a preregistration-ready analysis plan, with a study template.
• SEM (#55). An SEM article, and measurement invariance for latent models, invariance_solomon(). Both latent-variable functions are experimental.
• Coverage (#80). The article "Coverage of the Published Methodology" maps each work on the design to what the package does with it. Every reference is in APA Style (7th ed.), verified against Crossref (#42).

Changes that can alter results or code

• Renamed arguments (#83). The former names keep working, with a deprecation warning, through v1.x, when the arguments that follow a former name are also named:
  • y_post and y_pre replace y and pretest_score;
  • fit replaces object and x for fitted models;
  • stouffer replaces combine_with_stouffer.
• power_solomon() argument order. The order is now n, delta, sens, rho, sigma, alpha. Formerly rho came before sens, and alpha came after sims and stouffer. A call that passes these by position is read in the new order, with a warning, so check positional calls.
• One posttest plot. plot_solomon() and plot_solomon_gg() are deprecated in favor of plot_solomon_means().
• Clustered fits. fit_solomon_glm(robust = "CR2") stops with a classed error when clusters are confounded with conditions. It warns when a contrast's Satterthwaite degrees of freedom fall below 4.
• Saturated SEMs. fit_solomon_sem() and fit_solomon_sem_latent() no longer report global fit indices for saturated models (df = 0).
• Labels. The 1988 procedure is attributed to Walton Braver and Braver (1988) in printed output, including the test column of power_solomon(). No numeric result changes.

Checks

• R CMD check: "Status: OK", with 0 errors, 0 warnings, and 0 notes, on Windows, macOS, and Ubuntu: R release, oldrel-1, and devel, and devel also with --as-cran. It also gave 0 errors, 0 warnings, and 0 notes locally with --as-cran on Windows (R 4.6.1).
• mac-builder (macOS, Apple M1, R 4.6.1 Patched): Status OK on the code of this release.
• Check run: https://github.com/JUhalt/solomonR/actions/runs/37136223140
• Pre-release review: an independent review checked version consistency, the README's claims against the code, the roadmap against GitHub, the release notes, and the citation metadata. Its confirmed findings were fixed before the release (PR #100).
• pkgdown site: https://juhalt.github.io/solomonR/

Installation

install.packages(
  "solomonR",
  repos = c(
    JUhalt = "https://juhalt.r-universe.dev",
    CRAN = "https://cloud.r-project.org"
  )
)

or

pak::pak("JUhalt/solomonR@v0.8.0")

Not in this release

• CRAN pre-submission checks (#84). Their remaining results are pending, for v0.9.0.
• The sources Steyn (2009) names (#96).
• The v1.0.0 release, the companion manuscript, and CRAN submission (#85). These follow the maintainer's review.

References

Solomon, R. L. (1949). An extension of control group design. Psychological Bulletin, 46(2), 137–150. https://doi.org/10.1037/h0062958

Steyn, R. (2009). Re-designing the Solomon four-group: Can we improve on this exemplary model? Design Principles and Practices: An International Journal—Annual Review, 3(1), 383–394. https://doi.org/10.18848/1833-1874/CGP/v03i01/37588

Walton Braver, M. C., & Braver, S. L. (1988). Statistical treatment of the Solomon four-group design: A meta-analytic approach. Psychological Bulletin, 104(1), 150–154. https://doi.org/10.1037/0033-2909.104.1.150

solomonR 0.4.0

solomonR 0.4.0

solomonR 0.4.0 adds validated power and sample-size planning, figures designed for the Solomon four-group design, and a single index of the package's simulation evidence.

Highlights

Power and sample-size planning

• power_solomon() is rebuilt and validated (#18).
  • For each Solomon test it reports the rejection rate with its Monte Carlo standard error.
  • Every row names the estimand and the true effect behind it.
  • New alpha and seed arguments.
  • A pre-specified simulation study (126 scenarios, 315,000 replications) found:
    • exact agreement with the analytic benchmark for the 2x2 ANOVA interaction;
    • nominal size for Test I under the complete null;
    • no fit failures.
  • GLM-based rates are conservative with 20 or fewer participants per cell, following the HC3 standard errors used by default.
• New plan_solomon() (#24). It finds the smallest design, at a chosen allocation across the four cells, whose power reaches a target for each estimand.
  • It works analytically, returning the exact minimum, or by simulating the package's own GLM test.
  • The help page shows that the sensitization contrast has four times the sampling variance of the average treatment effect, so detecting sensitization as large as the average effect needs about four times as many participants.

Figures

Function	Issue	What it shows
plot_solomon_design()	#25	The design in Campbell and Stanley's notation, labeled with group sizes and posttest means, with empty or sparse groups flagged
plot_sensitization()	#26	Model-adjusted group means whose interaction equals the fitted sensitization contrast exactly, with observed means for comparison
plot_solomon_effects()	#27	A forest plot of the four Solomon contrasts, stating the confidence level and reference distribution
plot_solomon_change()	#28	Pretest-to-posttest change for the pretested groups, with the unpretested groups labeled as posttest-only by design
plot_classic_flow()	#29	The historical Tests A-I decision path, highlighting a fitted analysis's route, with its Type I error caution
plot_power_solomon()	#30	Power curves by estimand, marking the designs plan_solomon() returns

Evidence

• New article: "Validation Evidence" (#11). It gathers the package's simulation studies, with each study's protocol, scenarios, replications, failed fits, and results against its pre-registered tolerances.
• Shared tables. studies.csv and benchmarks.csv are rebuilt from each study's committed results and can be downloaded from the repository.

Changes that can alter results

• power_solomon() returns a new data frame.
  • The columns estimand, test, true_effect, power, mcse, sims, failures, and alpha replace the former metric and power.
  • The experimental warning is removed.
• Wider cell-mean intervals in two plots. plot_solomon() and plot_solomon_gg() now use t intervals with n - 1 degrees of freedom for cell means instead of the normal quantile, so the intervals are slightly wider with small cells.

Checks

• R CMD check: 0 errors, 0 warnings, 0 notes on Windows, macOS, and Ubuntu (R release, oldrel-1, and devel; devel also with --as-cran).
• Check run: https://github.com/JUhalt/solomonR/actions/runs/36130773705
• pkgdown site: https://juhalt.github.io/solomonR/

Installation

install.packages(
  "solomonR",
  repos = c(
    JUhalt = "https://juhalt.r-universe.dev",
    CRAN = "https://cloud.r-project.org"
  )
)

or

pak::pak("JUhalt/solomonR@v0.4.0")

Not in this release

• plot_sensitization() for fit_solomon_ml() fits. The ML small-sample degrees-of-freedom computation must be exposed first.
• Binary and count outcomes, clustered and longitudinal designs, and cluster-level randomization inference (#19). These are planned for v0.5.0.

solomonR 0.3.0

solomonR 0.3.0

solomonR 0.3.0 is aimed at graduate students and applied researchers who analyze, interpret, and report Solomon four-group studies.

• Inference: corrects small-sample inference and reports confidence intervals throughout.
• New tools: equivalence testing for pretest sensitization, design and missingness checks, and side-by-side method comparison.
• Documentation: records where each procedure comes from and what evidence supports it.

Highlights

Getting started

• New article, Getting Started: Analyzing a Solomon Four-Group Study (#17)
• New solomon_example teaching data with known true effects; solomon_demo kept as a second example (#21)

Inference

• Confidence intervals for GLM, classic Tests A-H, maximum-likelihood, and SEM effect summaries, with reference distributions chosen from the literature (#8)
• fit_solomon_glm() defaults to HC3 standard errors with t reference distributions (#8)
• CR2 cluster-robust tests use Satterthwaite degrees of freedom (#14)
• fit_solomon_ml(inference = "satterthwaite"): a small-sample option validated by simulation, with a warning when the smallest cell has fewer than 40 participants (#22)
• perm_solomon() refuses cluster-randomized fits (#15); the latent SEM mean structure is identified (#16)

New analyses and checks

• equivalence_solomon(): two one-sided tests (TOST) for sensitization and other Solomon contrasts (#4)
• compare_solomon_methods(): estimates, assumptions, reference distributions, and intervals aligned across methods (#7)
• validate_solomon() and check_solomon_missing(): design coding, cell structure, and structural versus incidental missingness (#5, #6)

Research grounding and validation

• New article on the origin, estimand, assumptions, and evidence for each Solomon method (#9)
• New article, Validating fit_solomon_ml(): a pre-specified simulation study (84 scenarios, 2,000 replications each) with its script and results (#10, #22)
• Help pages cite the methodological sources they implement; citations were checked against Crossref

Changes that can alter results

• fit_solomon_glm() uses HC3 unless robust is supplied, and Gaussian tests use t with residual degrees of freedom.
• Latent SEM means are relative to a reference group. Solomon contrasts are unchanged; model degrees of freedom and fit indices change.
• The Groups 3-4 Hedges' g interval uses the noncentral t method.
• Invalid design coding, permutation of clustered fits, and invalid conf_level values now raise errors.
• Result tables gain df, conf.low, and conf.high columns.

License

Starting with 0.3.0, solomonR is licensed under the GNU General Public License, version 3 only (SPDX: GPL-3.0-only). Version 0.2.0 and earlier remain under their original MIT license.

Checks

• R CMD check: 0 errors, 0 warnings, 0 notes on Windows, macOS, and Ubuntu (R release, oldrel-1, and devel; devel also with --as-cran)
• Check run: https://github.com/JUhalt/solomonR/actions/runs/35035821935
• pkgdown site: https://juhalt.github.io/solomonR/

Installation

install.packages(
  "solomonR",
  repos = c(
    JUhalt = "https://juhalt.r-universe.dev",
    CRAN = "https://cloud.r-project.org"
  )
)

or

pak::pak("JUhalt/solomonR@v0.3.0")

Not in this release

• power_solomon() remains experimental; a validated rebuild is planned for v0.4.0 (#18).
• Cluster-level randomization inference is planned for v0.5.0 (#19).
• Bell-McCaffrey (HC2) intervals are under evaluation.

solomonR 0.2.0

solomonR 0.2.0

The first substantial release of solomonR, an R package for analyzing and
teaching the Solomon four-group experimental design.

This release expands solomonR from its initial implementation into a
multi-framework Solomon analysis toolkit spanning historical procedures,
modern regression, randomization inference, maximum likelihood, and
structural equation modeling.

Highlights

Historical Solomon analysis

• Full historical Tests A-I workflow
• Solomon decision-path reporting
• ANCOVA, gain-score, repeated-measures, and posttest-only procedures
• Explicit cautions regarding historical conditional-testing strategies

Modern observed-variable analysis

• Unified Solomon GLM
• Equal-weighted average treatment effect
• Pretest sensitization interaction
• Treatment effects within pretested and unpretested participants
• HC3 heteroskedasticity-robust covariance estimation
• CR2 cluster-robust covariance estimation
• Wald-based partial R² for Gaussian models

Randomization inference

• Treatment-label permutation within pretest-assignment strata
• Studentized permutation statistics
• Finite Monte Carlo p-value correction
• Permutation-distribution plotting

Maximum likelihood

• Full-information Solomon-specific maximum-likelihood analysis
• Structural pretest absence handled as part of the experimental design

Structural equation modeling

• Observed-variable Solomon SEM
• ANCOVA-style SEM
• Latent-variable Solomon SEM
• Measurement-invariance controls
• Scalar-invariance safeguards for latent mean comparisons
• Explicit convergence checks

Documentation and reliability

• Expanded classic and modern analysis vignettes
• Rebuilt README and package documentation
• Extensive automated tests
• Cross-platform GitHub Actions checks
• R CMD check: 0 errors, 0 warnings, 0 notes

Installation

The release can be installed directly from GitHub with:

install.packages("pak")
pak::pak("JUhalt/solomonR@v0.2.0")