Merged
Conversation
Allow for permutation tests directly on the SES scale.
Needed to improve workflow when working back and forth together on code improvements.
Added more stable method for small samples with the two-sample case.
added default behavior for SE calculations for non-parametric SES calculations.
Update SE method
Finally got BCa some people have been asking for! Not the default and computationally heavy but it is there if people want to use it.
Needed to studentized test. Changed now and fixed. Checked locally with other tests.
Need some fixes to compare wilcox.test to hodges_lehmann
Need to add score method to the SES calculation
want to make it run faster on CRAN and not overload the system
smd_calc functions now allow direct specification of the denominator to use for SMD calculations.
Mainly, two-sample tests now include mean difference in output
Error where paired didn't implement plusone properly for randomization test. Also added reference document for future calibration for TOST
Essentially avoids edge case errors. Also made slight edits to SMD_calcs vignette and added some key references.
Adding new labels for rank probability estimates. Also, updating labels so output is easier to understand. Need to update labeling for simple_htest, perm_t_test, boot_t_test, etc to follow similar conventions.
Note, similar implementation still needed for other htest functions
Streamlined and easier (?) to understand now.
still working out kinks. need to update NEWS to reflect incoming changes.
reflect correlation test changes.
…o 112-update-ses
Need to fix error for boot_ses_calc
Introduced error in description. This should fix it to avoid issue in example.
noticed var.equal got ignored.
Bootstrap t-test has error due to trimming
Need to fix tests for output checks on string. Also, should probably update labels for cor_test/corsum_test/boot_cor_test.
…o 112-update-ses
Very broken at the moment. Need to fix later.
Contributor
There was a problem hiding this comment.
Pull request overview
This PR finalizes a broad update to TOSTER’s robust testing and effect-size infrastructure, including new transformations/utilities, expanded bootstrap CI support (incl. BCa), updated documentation/vignettes, and strengthened test coverage.
Changes:
- Added new effect-size helpers (
trans_rank_prob(),rank_diff()) and expanded labeling/sample-size reporting acrosshtestoutputs. - Added/extended bootstrap CI methods (including BCa) and aligned bootstrap p-values with the selected CI method.
- Updated documentation (Rd + vignettes) and added extensive regression tests for robust procedures (Brunner–Munzel, bootstrap tests, etc.).
Reviewed changes
Copilot reviewed 101 out of 111 changed files in this pull request and generated 10 comments.
Show a summary per file
| File | Description |
|---|---|
| vignettes/hypothesis_testing.R | Adds new hypothesis-testing vignette content (as an .R spin-style file). |
| vignettes/correlations.Rmd | Enables citation linking and refines narrative guidance. |
| vignettes/correlations.R | Removes previously included “advanced usage” examples from the spun .R. |
| vignettes/SMD_calcs.Rmd | Expands SMD vignette (robust trimming section, hypothesis-testing examples). |
| vignettes/SMD_calcs.R | Updates spun vignette code to match new examples. |
| vignettes/IntroductionToTOSTER.Rmd | Enables citation linking. |
| vignettes/IntroductionToTOSTER.R | Normalizes chunk header formatting in spun .R. |
| vignettes/IntroTOSTt.Rmd | Enables citation linking. |
| tests/testthat/test-wilcox.R | Adds bootstrap SES tests and reorganizes one-/two-/paired coverage. |
| tests/testthat/test-twoprop.R | Splits a long randomized test into a separate skip_on_cran() block. |
| tests/testthat/test-trans_rank_prob.R | Adds unit tests for trans_rank_prob() transformations and round-trips. |
| tests/testthat/test-tTOST.R | Adds explicit boot-vs-analytic SMD parity tests; adjusts older expectations. |
| tests/testthat/test-smd-se.R | Updates tests to request output = "data.frame" explicitly. |
| tests/testthat/test-rank_diff.R | Adds full coverage for new rank_diff() transformation. |
| tests/testthat/test-log.R | Marks heavy tests skip_on_cran() and adds CI/p-value agreement checks. |
| tests/testthat/test-htest.R | Updates expectations given new estimate structure/labels; adds regression test for $estimate. |
| tests/testthat/test-brunner_munzel_scale.R | Adds tests for new scale reporting in brunner_munzel(). |
| tests/testthat/test-brunner_munzel_perm_pval.R | Adds regression tests for paired permutation p_method routing. |
| tests/testthat/test-brunner_munzel_degenerate.R | Adds regression tests for multiple Brunner–Munzel edge cases. |
| tests/testthat/test-boot_t_test_variance_fix.R | Adds regression tests for a bootstrap variance/centering fix. |
| tests/testthat/test-boot_t_test_trimmed.R | Adds comprehensive tests for trimmed-mean support in boot_t_test(). |
| tests/testthat/test-bootTOST.R | Adds CI/p-value agreement tests for boot_t_TOST(). |
| man/z_cor_test.Rd | Documents se_method and updated stderr/estimate semantics. |
| man/wilcox_TOST.Rd | Documents new se_method options for Wilcoxon-based TOST. |
| man/trans_rank_prob.Rd | Adds new exported documentation for trans_rank_prob(). |
| man/stud_ci.Rd | Documents internal studentized CI helper for correlations. |
| man/smd_calc.Rd | Documents new denom/output/hypothesis-testing/trimming capabilities. |
| man/ses_calc.Rd | Documents new score SE method + correction options and boundary handling. |
| man/rank_diff.Rd | Adds new exported documentation for rank_diff(). |
| man/perm_t_test.Rd | Documents NPC-based equivalence/minimal-effect permutation approach. |
| man/log_TOST.Rd | Updates seealso links (adds boot_ses_test). |
| man/hodges_lehmann.Rd | Clarifies permutation limitations for equivalence/MET and improves narrative. |
| man/dot-fisher_z_se.Rd | Documents internal Fisher-z SE helper. |
| man/corsum_test.Rd | Updates stderr/estimate semantics and method details. |
| man/brunner_munzel.Rd | Documents new scale argument and permutation p-value options. |
| man/boot_t_test.Rd | Documents tr and boot_ci = "bca" and CI-consistent p-values. |
| man/boot_t_TOST.Rd | Documents boot_ci = "bca" and CI-consistent p-values. |
| man/boot_smd_calc.Rd | Documents denom/output/hypothesis-testing/trimming + BCa CI option. |
| man/boot_ses_calc.Rd | Documents BCa CI support + “none” alternative and CI-consistent p-values. |
| man/boot_pvalue.Rd | Documents internal boot_pvalue() dispatcher. |
| man/boot_log_TOST.Rd | Documents BCa CI option and CI-consistent p-values. |
| man/boot_cor_test.Rd | Documents new CI methods and how p-values match the CI method. |
| man/bca_params.Rd | Documents internal BCa parameter helper. |
| man/bca_ci.Rd | Documents internal BCa CI helper. |
| junk/debug_direction2.R | Adds local debugging script (non-production). |
| junk/debug_direction.R | Adds local debugging script (non-production). |
| junk/check_installed.R | Adds local debugging script (non-production). |
| _pkgdown.yml | Reorganizes reference sections; adds new helpers and boot_ses_test. |
| R/wilcox_TOST.R | Adds se_method argument and routes through ses_calc() accordingly. |
| R/trans_rank_prob.R | Adds new exported transformer + internal label helpers. |
| R/simple_htest.R | Adds sample-size reporting; fixes missing $estimate in some paths; improves labeling. |
| R/rank_diff.R | Adds new exported Kornbrot rank-difference transformer. |
| R/perm_t_test.R | Expands labeling, adds sample-size to output, and relabels formula outputs. |
| R/others.R | Adds internal BCa CI helpers (bca_ci, bca_params). |
| R/methods.TOSTt.R | Prints note for BCa CI usage. |
| R/htest_labels.R | Adds internal label builders for htest-style estimates and SMD notation. |
| R/htest_helpers.R | Adds internal relabel helper for formula interfaces. |
| R/globals.R | Adds globals for new symbols used in the package. |
| R/corsum_test.R | Updates SE formulas (incl. Bonett–Wright), stderr structure, and method string. |
| R/corr_calcs.R | Extends CI computation to accept SE overrides; adds studentized CI and bootstrap p-value dispatch. |
| R/cor_test.R | Adds se_method argument and jackknife SE support; updates stderr structure. |
| R/boot_t_TOST.R | Adds BCa CI option + CI-consistent p-values; adds jackknife plumbing. |
| R/boot_log_TOST.R | Adds BCa CI option + CI-consistent p-values; adds jackknife plumbing. |
| NEWS.md | Documents new helpers, BCa CI support, labeling changes, and correlation SE improvements. |
| NAMESPACE | Exports new functions and registers S3 methods; imports stats::dlogis. |
| DESCRIPTION | Updates RoxygenNote. |
| CLAUDE.md | Adds contributor workflow and code-style guidance. |
| .github/workflows/test-coverage.yaml | Sets NOT_CRAN env for CI runs. |
| .github/workflows/R-CMD-check.yaml | Sets NOT_CRAN env for CI runs. |
| .Rbuildignore | Ignores .claude artifacts. |
You can also share your feedback on Copilot code review. Take the survey.
Contributor
There was a problem hiding this comment.
Pull request overview
Updates TOSTER’s standardized effect size (SES/SMD) and robust testing ecosystem by expanding bootstrap/CI methods (incl. BCa), adding rank/probability-scale helpers, and aligning documentation/vignettes/tests with the new APIs.
Changes:
- Added new effect-size utilities (
rank_diff(),trans_rank_prob()) and expanded robust reporting/labeling across multiplehtestproducers. - Added BCa and studentized CI support to multiple bootstrap functions and introduced shared bootstrap p-value dispatch (
boot_pvalue()). - Updated man pages, vignettes, pkgdown reference structure, and significantly expanded/regressed test coverage for new/changed behaviors.
Reviewed changes
Copilot reviewed 104 out of 119 changed files in this pull request and generated 8 comments.
Show a summary per file
| File | Description |
|---|---|
| vignettes/SMD_calcs.Rmd | Adds citation linking and new robust SMD trimming section + updated SMD guidance/examples |
| vignettes/SMD_calcs.R | Syncs vignette extracted R code with new examples (bootstrap + trimming) |
| vignettes/IntroductionToTOSTER.Rmd | Enables link-citations for better reference rendering |
| vignettes/IntroductionToTOSTER.R | Updates chunk header formatting in extracted vignette code |
| vignettes/IntroTOSTt.Rmd | Enables link-citations |
| tests/testthat/test-wilcox.R | Updates hush sink target and reorganizes/extends SES/Wilcoxon bootstrap tests |
| tests/testthat/test-twoprop.R | Updates hush sink target and splits loop test into its own block |
| tests/testthat/test-trans_rank_prob.R | Adds unit tests for new probability-scale transformation helper |
| tests/testthat/test-tTOST.R | Adjusts SMD expectations to new output defaults and adds bootstrap consistency tests |
| tests/testthat/test-smd-se.R | Updates smd_calc() calls to request legacy data.frame output explicitly |
| tests/testthat/test-rank_diff.R | Adds tests for new Kornbrot rank-difference transformation |
| tests/testthat/test-powerraw.R | Updates hush sink target |
| tests/testthat/test-power_correlation.R | Updates hush sink target |
| tests/testthat/test-power_consistent.R | Updates hush sink target |
| tests/testthat/test-old_errors.R | Updates hush sink target |
| tests/testthat/test-log.R | Adds CRAN skips and CI/p-value agreement tests for boot_log_TOST() |
| tests/testthat/test-known_results.R | Updates hush sink target |
| tests/testthat/test-htest.R | Adjusts expectations for changed estimate labeling/structure and adds regression test for missing estimates |
| tests/testthat/test-ftests.R | Updates hush sink target |
| tests/testthat/test-data_summary_equivalent.R | Updates hush sink target |
| tests/testthat/test-compare_smds.R | Updates hush sink target |
| tests/testthat/test-brunner_munzel_scale.R | Adds tests for new brunner_munzel(scale=...) behavior |
| tests/testthat/test-brunner_munzel_perm_pval.R | Adds regression tests for paired permutation p-value computation and p_method |
| tests/testthat/test-brunner_munzel_degenerate.R | Adds regression tests for degenerate/edge case handling in Brunner–Munzel paths |
| tests/testthat/test-boot_t_test_variance_fix.R | Adds regression tests around bootstrap variance/p-value coherence fixes |
| tests/testthat/test-boot_t_test_trimmed.R | Adds comprehensive tests for trimmed-means support in boot_t_test() |
| tests/testthat/test-bootTOST.R | Adds CRAN skips and CI/p-value agreement tests for boot_t_TOST() |
| man/z_cor_test.Rd | Documents new se_method and revised stderr/estimate naming for correlation tests |
| man/wilcox_TOST.Rd | Documents new se_method argument for Wilcoxon TOST wrapper |
| man/trans_rank_prob.Rd | Adds documentation for new trans_rank_prob() helper |
| man/stud_ci.Rd | Documents new internal correlation studentized CI helper |
| man/smd_calc.Rd | Documents expanded SMD API (denom/output/testing/trimming) |
| man/ses_calc.Rd | Documents expanded SES API (score SE method, correction flag, boundary handling) |
| man/rank_diff.Rd | Adds documentation for new rank_diff() helper |
| man/perm_t_test.Rd | Documents permutation approach for equivalence/minimal effect testing |
| man/log_TOST.Rd | Adds cross-links to new robust SES bootstrap test helper |
| man/hodges_lehmann.Rd | Clarifies permutation limitations for equivalence/MET and updates method details |
| man/dot-fisher_z_se.Rd | Documents internal Fisher-z SE helper |
| man/corsum_test.Rd | Updates SE method description and stderr/estimate naming |
| man/brunner_munzel.Rd | Documents new scale argument and permutation p-value notes |
| man/boot_t_test.Rd | Documents trimming + BCa + CI-consistent p-values |
| man/boot_t_TOST.Rd | Documents BCa + CI-consistent p-values |
| man/boot_smd_calc.Rd | Documents expanded bootstrap SMD API (denom/output/testing/trimming/BCa) |
| man/boot_ses_calc.Rd | Documents expanded bootstrap SES API (BCa + optional testing) |
| man/boot_pvalue.Rd | Documents new internal p-value dispatch helper |
| man/boot_log_TOST.Rd | Documents BCa + CI-consistent p-values |
| man/boot_cor_test.Rd | Documents BCa/stud CI options and CI-inverted p-values |
| man/bca_params.Rd | Documents internal BCa parameter extraction helper |
| man/bca_ci.Rd | Documents internal BCa CI helper |
| junk/debug_direction2.R | Adds local debugging script (non-package logic) |
| junk/debug_direction.R | Adds local debugging script (non-package logic) |
| junk/check_installed.R | Adds local debugging script (non-package logic) |
| _pkgdown.yml | Reorganizes reference sections and adds new helpers to docs |
| R/wilcox_TOST.R | Adds se_method plumbed through to SES effect size computation |
| R/trans_rank_prob.R | Adds new exported transformer + shared label helpers |
| R/simple_htest.R | Adds sample size info, restores missing estimates for certain alts, and improves estimate labeling |
| R/rank_diff.R | Adds new exported Kornbrot rank-difference transformation |
| R/perm_t_test.R | Improves estimate labeling, adds sample size, and expands docs/comments (incl. formula relabeling) |
| R/others.R | Adds BCa CI + BCa parameter helpers |
| R/methods.TOSTt.R | Prints note for BCa CI method |
| R/htest_labels.R | Adds shared internal label helpers for estimates and SMD notation |
| R/htest_helpers.R | Adds internal relabel helper for formula interfaces |
| R/globals.R | Adds globals to satisfy checks for new symbols used in NSE contexts |
| R/corsum_test.R | Updates SE logic (Bonett–Wright for Spearman) and stderr structure |
| R/corr_calcs.R | Adds stud_ci(), boot_pvalue(), and Fisher-z SE helper; enhances CI SE override capability |
| R/cor_test.R | Adds se_method (analytic/jackknife) and updates SE + CI construction |
| R/boot_t_TOST.R | Adds BCa support; replaces p-values with CI-consistent p-values via boot_pvalue() |
| R/boot_log_TOST.R | Adds BCa support; replaces p-values with CI-consistent p-values via boot_pvalue() |
| NEWS.md | Documents new functions and major API/behavior changes (BCa, labels, correlation SE changes) |
| NAMESPACE | Exports new helpers and registers new S3 methods |
| DESCRIPTION | Updates RoxygenNote |
| CLAUDE.md | Adds contributor workflow/style guidance (repo-local edits, junk/, headings) |
| .github/workflows/test-coverage.yaml | Sets NOT_CRAN=true in CI env for conditional test behavior |
| .github/workflows/R-CMD-check.yaml | Sets NOT_CRAN=true in CI env for conditional test behavior |
| .Rbuildignore | Adds ignore for ^.claude$ |
You can also share your feedback on Copilot code review. Take the survey.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Updated SES functions, documentation, and added other features for robust TOST. This should be the final big changes before writing the updated preprint and submitting to AMPSS.