IndrajeetPatil / pairwiseComparisons

`pairwiseComparisons`: Multiple Pairwise Comparison Tests

Package	Status	Usage	GitHub	References

Introduction

pairwiseComparisons provides a tidy data friendly way to carry out pairwise comparison tests.

It currently supports post hoc multiple pairwise comparisons tests for both between-subjects and within-subjects one-way analysis of variance designs. For both of these designs, parametric, non-parametric, robust, and Bayes Factor statistical tests are available.

Installation

To get the latest, stable CRAN release:

install.packages("pairwiseComparisons")

You can get the development version of the package from GitHub. To see what new changes (and bug fixes) have been made to the package since the last release on CRAN, you can check the detailed log of changes here: https://indrajeetpatil.github.io/pairwiseComparisons/news/index.html

If you are in hurry and want to reduce the time of installation, prefer-

# needed package to download from GitHub repo
install.packages("remotes")

# downloading the package from GitHub
remotes::install_github(
  repo = "IndrajeetPatil/pairwiseComparisons", # package path on GitHub
  dependencies = FALSE, # assumes you have already installed needed packages
  quick = TRUE # skips docs, demos, and vignettes
)

If time is not a constraint-

remotes::install_github(
  repo = "IndrajeetPatil/pairwiseComparisons", # package path on GitHub
  dependencies = TRUE, # installs packages which pairwiseComparisons depends on
  upgrade_dependencies = TRUE # updates any out of date dependencies
)

Summary of types of statistical analyses

Following table contains a brief summary of the currently supported pairwise comparison tests-

Between-subjects design

Type	Equal variance?	Test	p-value adjustment?
Parametric	No	Games-Howell test	Yes
Parametric	Yes	Student’s t-test	Yes
Non-parametric	No	Dunn test	Yes
Robust	No	Yuen’s trimmed means test	Yes
Bayes Factor	`NA`	Student’s t-test	`NA`

Within-subjects design

Type	Test	p-value adjustment?
Parametric	Student’s t-test	Yes
Non-parametric	Durbin-Conover test	Yes
Robust	Yuen’s trimmed means test	Yes
Bayes Factor	Student’s t-test	`NA`

Examples

Here we will see specific examples of how to use this function for different types of

designs (between or within subjects)
statistics (parametric, non-parametric, robust, Bayes Factor)
p-value adjustment methods

Between-subjects design

# for reproducibility
set.seed(123)
library(pairwiseComparisons)

# parametric
# if `var.equal = TRUE`, then Student's *t*-test will be run
pairwise_comparisons(
  data = ggplot2::msleep,
  x = vore,
  y = brainwt,
  type = "parametric",
  var.equal = TRUE,
  paired = FALSE,
  p.adjust.method = "bonferroni"
)
#> # A tibble: 6 x 7
#>   group1  group2  p.value significance test.details     p.value.adjustment label                                        
#>   <chr>   <chr>     <dbl> <chr>        <chr>            <chr>              <chr>                                        
#> 1 carni   herbi     1     ns           Student's t-test Bonferroni         list(~italic(p)[Bonferroni-corrected]==1.000)
#> 2 carni   insecti   1     ns           Student's t-test Bonferroni         list(~italic(p)[Bonferroni-corrected]==1.000)
#> 3 carni   omni      1     ns           Student's t-test Bonferroni         list(~italic(p)[Bonferroni-corrected]==1.000)
#> 4 herbi   insecti   1     ns           Student's t-test Bonferroni         list(~italic(p)[Bonferroni-corrected]==1.000)
#> 5 herbi   omni      0.979 ns           Student's t-test Bonferroni         list(~italic(p)[Bonferroni-corrected]==0.979)
#> 6 insecti omni      1     ns           Student's t-test Bonferroni         list(~italic(p)[Bonferroni-corrected]==1.000)

# if `var.equal = FALSE`, then Games-Howell test will be run
pairwise_comparisons(
  data = ggplot2::msleep,
  x = vore,
  y = brainwt,
  type = "parametric",
  var.equal = FALSE,
  paired = FALSE,
  p.adjust.method = "bonferroni"
)
#> # A tibble: 6 x 8
#>   group1  group2  statistic p.value significance test.details      p.value.adjustment label                                        
#>   <chr>   <chr>       <dbl>   <dbl> <chr>        <chr>             <chr>              <chr>                                        
#> 1 carni   herbi        2.17       1 ns           Games-Howell test Bonferroni         list(~italic(p)[Bonferroni-corrected]==1.000)
#> 2 carni   insecti     -2.17       1 ns           Games-Howell test Bonferroni         list(~italic(p)[Bonferroni-corrected]==1.000)
#> 3 carni   omni         1.10       1 ns           Games-Howell test Bonferroni         list(~italic(p)[Bonferroni-corrected]==1.000)
#> 4 herbi   insecti     -2.41       1 ns           Games-Howell test Bonferroni         list(~italic(p)[Bonferroni-corrected]==1.000)
#> 5 herbi   omni        -1.87       1 ns           Games-Howell test Bonferroni         list(~italic(p)[Bonferroni-corrected]==1.000)
#> 6 insecti omni         2.19       1 ns           Games-Howell test Bonferroni         list(~italic(p)[Bonferroni-corrected]==1.000)

# non-parametric
pairwise_comparisons(
  data = ggplot2::msleep,
  x = vore,
  y = brainwt,
  type = "nonparametric",
  paired = FALSE,
  p.adjust.method = "none"
)
#> # A tibble: 6 x 8
#>   group1  group2  statistic p.value significance test.details p.value.adjustment label                               
#>   <chr>   <chr>       <dbl>   <dbl> <chr>        <chr>        <chr>              <chr>                               
#> 1 carni   herbi       0.582  0.561  ns           Dunn test    None               list(~italic(p)[uncorrected]==0.561)
#> 2 carni   insecti     1.88   0.0595 ns           Dunn test    None               list(~italic(p)[uncorrected]==0.060)
#> 3 carni   omni        1.14   0.254  ns           Dunn test    None               list(~italic(p)[uncorrected]==0.254)
#> 4 herbi   insecti     1.63   0.102  ns           Dunn test    None               list(~italic(p)[uncorrected]==0.102)
#> 5 herbi   omni        0.717  0.474  ns           Dunn test    None               list(~italic(p)[uncorrected]==0.474)
#> 6 insecti omni        1.14   0.254  ns           Dunn test    None               list(~italic(p)[uncorrected]==0.254)

# robust
pairwise_comparisons(
  data = ggplot2::msleep,
  x = vore,
  y = brainwt,
  type = "robust",
  paired = FALSE,
  p.adjust.method = "fdr"
)
#> # A tibble: 6 x 10
#>   group1  group2  estimate conf.low conf.high p.value significance test.details              p.value.adjustment
#>   <chr>   <chr>      <dbl>    <dbl>     <dbl>   <dbl> <chr>        <chr>                     <chr>             
#> 1 carni   herbi   -0.0530   -0.274     0.168    0.969 ns           Yuen's trimmed means test FDR               
#> 2 carni   insecti  0.0577   -0.0609    0.176    0.969 ns           Yuen's trimmed means test FDR               
#> 3 carni   omni     0.00210  -0.151     0.155    0.969 ns           Yuen's trimmed means test FDR               
#> 4 herbi   insecti  0.111    -0.0983    0.320    0.969 ns           Yuen's trimmed means test FDR               
#> 5 herbi   omni     0.0551   -0.173     0.283    0.969 ns           Yuen's trimmed means test FDR               
#> 6 insecti omni    -0.0556   -0.184     0.0728   0.969 ns           Yuen's trimmed means test FDR               
#>   label                                 
#>   <chr>                                 
#> 1 list(~italic(p)[FDR-corrected]==0.969)
#> 2 list(~italic(p)[FDR-corrected]==0.969)
#> 3 list(~italic(p)[FDR-corrected]==0.969)
#> 4 list(~italic(p)[FDR-corrected]==0.969)
#> 5 list(~italic(p)[FDR-corrected]==0.969)
#> 6 list(~italic(p)[FDR-corrected]==0.969)

# Bayes Factor
pairwise_comparisons(
  data = ggplot2::msleep,
  x = vore,
  y = brainwt,
  type = "bayes",
  paired = FALSE
)
#> # A tibble: 6 x 17
#>   group1  group2  term       estimate conf.low conf.high    pd rope.percentage prior.distribution prior.location prior.scale effects
#>   <chr>   <chr>   <chr>         <dbl>    <dbl>     <dbl> <dbl>           <dbl> <chr>                       <dbl>       <dbl> <chr>  
#> 1 carni   herbi   Difference   0.383    -0.349    1.15   0.801           0.135 cauchy                          0       0.707 fixed  
#> 2 carni   insecti Difference  -0.0348   -0.104    0.0271 0.812           0.975 cauchy                          0       0.707 fixed  
#> 3 carni   omni    Difference   0.0441   -0.117    0.192  0.692           0.734 cauchy                          0       0.707 fixed  
#> 4 herbi   insecti Difference  -0.371    -1.41     0.526  0.746           0.131 cauchy                          0       0.707 fixed  
#> 5 herbi   omni    Difference  -0.371    -0.884    0.234  0.856           0.144 cauchy                          0       0.707 fixed  
#> 6 insecti omni    Difference   0.0758   -0.102    0.286  0.742           0.586 cauchy                          0       0.707 fixed  
#>   component    bf10 log_e_bf10 label                        test.details    
#>   <chr>       <dbl>      <dbl> <chr>                        <chr>           
#> 1 conditional 0.540     -0.617 list(~log[e](BF[10])==-0.62) Student's t-test
#> 2 conditional 0.718     -0.332 list(~log[e](BF[10])==-0.33) Student's t-test
#> 3 conditional 0.427     -0.851 list(~log[e](BF[10])==-0.85) Student's t-test
#> 4 conditional 0.540     -0.616 list(~log[e](BF[10])==-0.62) Student's t-test
#> 5 conditional 0.571     -0.560 list(~log[e](BF[10])==-0.56) Student's t-test
#> 6 conditional 0.545     -0.606 list(~log[e](BF[10])==-0.61) Student's t-test

Within-subjects design

# for reproducibility
set.seed(123)

# parametric
pairwise_comparisons(
  data = bugs_long,
  x = condition,
  y = desire,
  type = "parametric",
  paired = TRUE,
  p.adjust.method = "BH"
)
#> # A tibble: 6 x 7
#>   group1 group2  p.value significance test.details     p.value.adjustment label                                    
#>   <chr>  <chr>     <dbl> <chr>        <chr>            <chr>              <chr>                                    
#> 1 HDHF   HDLF   1.06e- 3 **           Student's t-test FDR                list(~italic(p)[FDR-corrected]==0.001)   
#> 2 HDHF   LDHF   7.02e- 2 ns           Student's t-test FDR                list(~italic(p)[FDR-corrected]==0.070)   
#> 3 HDHF   LDLF   3.95e-12 ***          Student's t-test FDR                list(~italic(p)[FDR-corrected]==3.95e-12)
#> 4 HDLF   LDHF   6.74e- 2 ns           Student's t-test FDR                list(~italic(p)[FDR-corrected]==0.067)   
#> 5 HDLF   LDLF   1.99e- 3 **           Student's t-test FDR                list(~italic(p)[FDR-corrected]==0.002)   
#> 6 LDHF   LDLF   6.66e- 9 ***          Student's t-test FDR                list(~italic(p)[FDR-corrected]==6.66e-09)

# non-parametric
pairwise_comparisons(
  data = bugs_long,
  x = condition,
  y = desire,
  type = "nonparametric",
  paired = TRUE,
  p.adjust.method = "BY"
)
#> # A tibble: 6 x 8
#>   group1 group2 statistic  p.value significance test.details        p.value.adjustment label                                   
#>   <chr>  <chr>      <dbl>    <dbl> <chr>        <chr>               <chr>              <chr>                                   
#> 1 HDHF   HDLF        4.78 1.44e- 5 ***          Durbin-Conover test BY                 list(~italic(p)[BY-corrected]==1.44e-05)
#> 2 HDHF   LDHF        2.44 4.47e- 2 *            Durbin-Conover test BY                 list(~italic(p)[BY-corrected]==0.045)   
#> 3 HDHF   LDLF        8.01 5.45e-13 ***          Durbin-Conover test BY                 list(~italic(p)[BY-corrected]==5.45e-13)
#> 4 HDLF   LDHF        2.34 4.96e- 2 *            Durbin-Conover test BY                 list(~italic(p)[BY-corrected]==0.050)   
#> 5 HDLF   LDLF        3.23 5.05e- 3 **           Durbin-Conover test BY                 list(~italic(p)[BY-corrected]==0.005)   
#> 6 LDHF   LDLF        5.57 4.64e- 7 ***          Durbin-Conover test BY                 list(~italic(p)[BY-corrected]==4.64e-07)

# robust
pairwise_comparisons(
  data = bugs_long,
  x = condition,
  y = desire,
  type = "robust",
  paired = TRUE,
  p.adjust.method = "hommel"
)
#> # A tibble: 6 x 10
#>   group1 group2 estimate conf.low conf.high  p.value significance test.details              p.value.adjustment
#>   <chr>  <chr>     <dbl>    <dbl>     <dbl>    <dbl> <chr>        <chr>                     <chr>             
#> 1 HDHF   HDLF      1.16    0.318      2.00  1.49e- 3 **           Yuen's trimmed means test Hommel            
#> 2 HDHF   LDHF      0.5    -0.188      1.19  6.20e- 2 ns           Yuen's trimmed means test Hommel            
#> 3 HDHF   LDLF      2.10    1.37       2.82  1.79e-10 ***          Yuen's trimmed means test Hommel            
#> 4 HDLF   LDHF     -0.701  -1.71       0.303 6.20e- 2 ns           Yuen's trimmed means test Hommel            
#> 5 HDLF   LDLF      0.938   0.0694     1.81  1.36e- 2 *            Yuen's trimmed means test Hommel            
#> 6 LDHF   LDLF      1.54    0.810      2.27  1.16e- 6 ***          Yuen's trimmed means test Hommel            
#>   label                                       
#>   <chr>                                       
#> 1 list(~italic(p)[Hommel-corrected]==0.001)   
#> 2 list(~italic(p)[Hommel-corrected]==0.062)   
#> 3 list(~italic(p)[Hommel-corrected]==1.79e-10)
#> 4 list(~italic(p)[Hommel-corrected]==0.062)   
#> 5 list(~italic(p)[Hommel-corrected]==0.014)   
#> 6 list(~italic(p)[Hommel-corrected]==1.16e-06)

# Bayes Factor
pairwise_comparisons(
  data = WRS2::WineTasting,
  x = Wine,
  y = Taste,
  type = "bayes",
  paired = TRUE,
  bf.prior = 0.77
)
#> # A tibble: 3 x 17
#>   group1 group2 term       estimate conf.low conf.high    pd rope.percentage prior.distribution prior.location prior.scale effects
#>   <chr>  <chr>  <chr>         <dbl>    <dbl>     <dbl> <dbl>           <dbl> <chr>                       <dbl>       <dbl> <chr>  
#> 1 Wine A Wine B Difference -0.00769  -0.0479    0.0308 0.628           1     cauchy                          0        0.77 fixed  
#> 2 Wine A Wine C Difference -0.0776   -0.128    -0.0261 0.991           0.809 cauchy                          0        0.77 fixed  
#> 3 Wine B Wine C Difference -0.0694   -0.104    -0.0394 1.00            1     cauchy                          0        0.77 fixed  
#>   component     bf10 log_e_bf10 label                        test.details    
#>   <chr>        <dbl>      <dbl> <chr>                        <chr>           
#> 1 conditional  0.219      -1.52 list(~log[e](BF[10])==-1.52) Student's t-test
#> 2 conditional  3.60        1.28 list(~log[e](BF[10])==1.28)  Student's t-test
#> 3 conditional 50.5         3.92 list(~log[e](BF[10])==3.92)  Student's t-test

Using `pairwiseComparisons` with `ggsignif`

Example-1: between-subjects

# needed libraries
library(ggplot2)
library(pairwiseComparisons)
library(ggsignif)

# converting to factor
mtcars$cyl <- as.factor(mtcars$cyl)

# creating a basic plot
p <- ggplot(mtcars, aes(cyl, wt)) + geom_boxplot()

# using `pairwiseComparisons` package to create a dataframe with results
(df <-
  pairwise_comparisons(mtcars, cyl, wt, messages = FALSE) %>%
  dplyr::mutate(.data = ., groups = purrr::pmap(.l = list(group1, group2), .f = c)) %>%
  dplyr::arrange(.data = ., group1))
#> # A tibble: 3 x 9
#>   group1 group2 statistic   p.value significance test.details      p.value.adjustment label                                     
#>   <chr>  <chr>      <dbl>     <dbl> <chr>        <chr>             <chr>              <chr>                                     
#> 1 4      6           5.39 0.00831   **           Games-Howell test Holm               list(~italic(p)[Holm-corrected]==0.008)   
#> 2 4      8           9.11 0.0000124 ***          Games-Howell test Holm               list(~italic(p)[Holm-corrected]==1.24e-05)
#> 3 6      8           5.12 0.00831   **           Games-Howell test Holm               list(~italic(p)[Holm-corrected]==0.008)   
#>   groups   
#>   <list>   
#> 1 <chr [2]>
#> 2 <chr [2]>
#> 3 <chr [2]>

# using `geom_signif` to display results
p +
  ggsignif::geom_signif(
    comparisons = df$groups,
    map_signif_level = TRUE,
    tip_length = 0.01,
    y_position = c(5.5, 5.75, 6),
    annotations = df$label,
    test = NULL,
    na.rm = TRUE,
    parse = TRUE
  )

Example-2: within-subjects

# needed libraries
library(ggplot2)
library(pairwiseComparisons)
library(ggsignif)

# creating a basic plot
p <- ggplot(WRS2::WineTasting, aes(Wine, Taste)) + geom_boxplot()

# using `pairwiseComparisons` package to create a dataframe with results
(df <-
  pairwise_comparisons(WRS2::WineTasting, Wine, Taste, type = "bayes", paired = TRUE) %>%
  dplyr::mutate(.data = ., groups = purrr::pmap(.l = list(group1, group2), .f = c)) %>%
  dplyr::arrange(.data = ., group1))
#> # A tibble: 3 x 18
#>   group1 group2 term       estimate conf.low conf.high    pd rope.percentage prior.distribution prior.location prior.scale effects
#>   <chr>  <chr>  <chr>         <dbl>    <dbl>     <dbl> <dbl>           <dbl> <chr>                       <dbl>       <dbl> <chr>  
#> 1 Wine A Wine B Difference -0.00664  -0.0462    0.0312 0.632           1     cauchy                          0       0.707 fixed  
#> 2 Wine A Wine C Difference -0.0765   -0.128    -0.0250 0.99            0.811 cauchy                          0       0.707 fixed  
#> 3 Wine B Wine C Difference -0.0696   -0.103    -0.0409 0.999           0.988 cauchy                          0       0.707 fixed  
#>   component     bf10 log_e_bf10 label                        test.details     groups   
#>   <chr>        <dbl>      <dbl> <chr>                        <chr>            <list>   
#> 1 conditional  0.235      -1.45 list(~log[e](BF[10])==-1.45) Student's t-test <chr [2]>
#> 2 conditional  3.71        1.31 list(~log[e](BF[10])==1.31)  Student's t-test <chr [2]>
#> 3 conditional 50.5         3.92 list(~log[e](BF[10])==3.92)  Student's t-test <chr [2]>

# using `geom_signif` to display results
p +
  ggsignif::geom_signif(
    comparisons = df$groups,
    map_signif_level = TRUE,
    tip_length = 0.01,
    y_position = c(6.5, 6.65, 6.8),
    annotations = df$label,
    test = NULL,
    na.rm = TRUE,
    parse = TRUE
  )

Acknowledgments

The hexsticker was generously designed by Sarah Otterstetter (Max Planck Institute for Human Development, Berlin).

Contributing

I’m happy to receive bug reports, suggestions, questions, and (most of all) contributions to fix problems and add features. I personally prefer using the GitHub issues system over trying to reach out to me in other ways (personal e-mail, Twitter, etc.). Pull Requests for contributions are encouraged.

Here are some simple ways in which you can contribute (in the increasing order of commitment):

Read and correct any inconsistencies in the documentation
Raise issues about bugs or wanted features
Review code
Add new functionality

Please note that this project is released with a Contributor Code of Conduct. By participating in this project you agree to abide by its terms.

IndrajeetPatil / pairwiseComparisons

README.md

`pairwiseComparisons`: Multiple Pairwise Comparison Tests

Introduction

Installation

Summary of types of statistical analyses

Between-subjects design

Within-subjects design

Examples

Between-subjects design

Within-subjects design

Using `pairwiseComparisons` with `ggsignif`

Example-1: between-subjects

Example-2: within-subjects

Acknowledgments

Contributing

About

Releases 14

Packages

Languages

IndrajeetPatil / pairwiseComparisons

Join GitHub today

Launching GitHub Desktop

Launching GitHub Desktop

Launching Xcode

Launching Visual Studio

Latest commit

Git stats

Files

README.md

pairwiseComparisons: Multiple Pairwise Comparison Tests

Introduction

Installation

Summary of types of statistical analyses

Between-subjects design

Within-subjects design

Examples

Between-subjects design

Within-subjects design

Using pairwiseComparisons with ggsignif

Example-1: between-subjects

Example-2: within-subjects

Acknowledgments

Contributing

About

Resources

License

Releases 14

Packages 0

Languages

Essential cookies

Always active

Analytics cookies

`pairwiseComparisons`: Multiple Pairwise Comparison Tests

Using `pairwiseComparisons` with `ggsignif`

Packages