Eindhoven University of Technology (TU/e) course "Improving Your Statistical Questions" by Daniël Lakens on Coursera (completed December 14, 2022).
Daniël Lakens - Associate Professor in the Human-Technology interaction group at Eindhoven University of Technology (TU/e)
This course examines many statistical concepts through simulations or calculations in the free software R.
The work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License. https://creativecommons.org/licenses/by-nc-sa/4.0/
R version 4.1.2 (2021-11-01) -- "Bird Hippie"
(1.) TOSTER - Two One-Sided Tests (TOST) Equivalence Testing You can install TOSTER in R using: install.packages('TOSTER').
Alternatively, you can download a spreadsheet to perform these calculations: https://osf.io/qzjaj/
(2.) ANOVA_power: Superpower (version 0.1.2) - Simulation function used to estimate power
Alternatively, you can use a shiny app to perform these calculations: https://shiny.ieis.tue.nl/anova_power/
(3.) METAFOR Package: A Meta-Analysis Package for R.
(4.) MBESS Package: Methods for the Behavioral, Educational, and Social Sciences: An R package
statcheck 1.3.0 https://rpubs.com/michelenuijten/statcheckmanual
- Automatically extracts statistics from reasearch articles and recomputes their p-values, as long as statistics are reported following guidelines from the American Psychological Association (APA). Upload a PDF, word document, or HTML file.
- Manual: https://michelenuijten.shinyapps.io/statcheck-web/
Likelihood Ratio for Mixed Results https://shiny.ieis.tue.nl/mixed_results_likelihood/
- Shiny app accompanying: Lakens, D., & Etz, A. J. (2017). Too true to be bad: When sets of studies with significant and non-significant findings are probably true. Social Psychological and Personality Science.
Positive Predictive Value (PPV) of a p-value https://shiny.ieis.tue.nl/PPV/
- When does a significant p-value indicate a true effect?
- Conduct statistical power analysis and calculate probabilities as well as some more test cases
https://download.cnet.com/G-Power/3000-2054_4-10647044.html
This course aims to help you to ask better statistical questions when performing empirical research. We will discuss how to design informative studies, both when your predictions are correct, as when your predictions are wrong. We will question norms, and reflect on how we can improve research practices to ask more interesting questions. In practical hands on assignments you will learn techniques and tools that can be immediately implemented in your own research, such as thinking about the smallest effect size you are interested in, justifying your sample size, evaluate findings in the literature while keeping publication bias into account, performing a meta-analysis, and making your analyses computationally reproducible.
If you have the time, it is recommended that you complete my course 'Improving Your Statistical Inferences' before enrolling in this course, although this course is completely self-contained.
Week 1: Introduction
1.1. Introduction
1.2. Do you really want to test a hypothesis?
1.3. Risky predictions
- Assignment 1.1.: testing Range Predictions
Week 2: Falsifying Predictions
2.1. Falsifying Predictions in Theory
2.2. Setting the Smallest Effect Size of Interest (SESOI)
- Assignment 2.1.: The Smallest Telescope Approach to Setting a SESOI
- Assignment 2.1.: Setting a SESOI based on Resources
2.3. Falsifying Predictions in Practice
- Assignment 2.3.: Equivalence testing
Week 3: Designing Informative Studies
3.1. Justifying Error Rates
- Assignemnt 3.1.: Confidence Intervals for Standard Deviations
3.2. Power Analysis
- Assignemnt 3.2.: Power Analysis for ANOVA Designs
3.3. Simulation
Week 4: Meta-Analysis and Bias Detection
4.1. Mixed Results
- Assignemnt 4.1.: Likelihood of Significant Findings
4.2. Intro to Meta-Analysis
- Assignment 4.2.: Introduction to Meta-Analysis
4.3. Bias Detection
- Assignment 4.3.: Detecting Publication Bias
- Assignment 4.4.: Checking your Stats
Week 5: Computational Reproductibility, Philosophy of Science, and Science Integrity
5.1. Computational Reproducibility
- Assignment 5.1.: Computational Reproducibility
5.2. Philosophy of Science in Practice
- Assignment 5.2.: Does your Philosophy of Science matters in Practice?
5.3. Scientific Integrity in Practice
- Assignment 5.3.: Applied research Ethics
Week 6: Final Exam
Eindhoven University of Technology (TU/e) is a young university, founded in 1956 by industry, local government and academia. Today, their spirit of collaboration is still at the heart of the university community. We foster an open culture where everyone feels free to exchange ideas and take initiatives.
We offer academic education that is driven by fundamental and applied research. Our educational philosophy is based on personal attention and room for individual ambitions and talents. Our research meets the highest international standards of quality. We push the limits of science, which puts us at the forefront of rapidly emerging areas of research.