Eindhoven University of Technology (TU/e) course "Improving your statistical inferences" by Daniël Lakens on Coursera
(completed December 26, 2022).
Daniël Lakens - Associate Professor in the Human-Technology interaction group at Eindhoven University of Technology (TU/e)
In assignments 7, I define a theoretical hypothesis, translate it into a statistical hypothesis, design a study to test my hypothesis, collect the data, report the results, create a reproducible analysis file, and publicly share the data, the analysis file, and the research report publicly online. Learning these Open Science practices will allow to share all aspects of my research with others. To keep the research project in this assignment feasible, the data collection will consist of movie ratings on the IMDB. I can come up with any hypothesis Iu want, as long as it can be tested based on the IMDB ratings of movies! This assignment will be peer-reviewed by fellow students.
Date created: 2022-12-02
https://osf.io/shbu4/?view_only=
This course examines many statistical concepts through simulations or calculations in the free software R.
R version 4.1.2 (2021-11-01) -- "Bird Hippie"
TOSTER - Two One-Sided Tests (TOST) Equivalence Testing
You can install TOSTER in R using: install.packages('TOSTER'). Alternatively, you can download a spreadsheet to perform these calculations: https://osf.io/qzjaj/
G*Power - Conduct statistical power analysis and calculate probabilities as well as some more test cases
( https://download.cnet.com/G-Power/3000-2054_4-10647044.html )
The work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License. https://creativecommons.org/licenses/by-nc-sa/4.0/
This course aims to help you to draw better statistical inferences from empirical research. First, we will discuss how to correctly interpret p-values, effect sizes, confidence intervals, Bayes Factors, and likelihood ratios, and how these statistics answer different questions you might be interested in. Then, you will learn how to design experiments where the false positive rate is controlled, and how to decide upon the sample size for your study, for example in order to achieve high statistical power. Subsequently, you will learn how to interpret evidence in the scientific literature given widespread publication bias, for example by learning about p-curve analysis. Finally, we will talk about how to do philosophy of science, theory construction, and cumulative science, including how to perform replication studies, why and how to pre-register your experiment, and how to share your results following Open Science principles.
In practical, hands on assignments, you will learn how to simulate t-tests to learn which p-values you can expect, calculate likelihood ratio's and get an introduction the binomial Bayesian statistics, and learn about the positive predictive value which expresses the probability published research findings are true. We will experience the problems with optional stopping and learn how to prevent these problems by using sequential analyses. You will calculate effect sizes, see how confidence intervals work through simulations, and practice doing a-priori power analyses. Finally, you will learn how to examine whether the null hypothesis is true using equivalence testing and Bayesian statistics, and how to pre-register a study, and share your data on the Open Science Framework.
Week 1: Introduction and Frequentist Statistics
1.1. Introduction
1.2. What is a p-value
1.3. Type 1 and Type 2 errors
- Assignment 1.1.: Which p-value can you expect?
- Assignment 1.2.: Understanding common misconceptions about p-values
Week 2: Likelihoods and Bayesian Statistics
2.1. Likelihoods
- Assignment 2.1.: Likelihoods
2.2. Binominal Bayesian Inference
- Assignment 2.2.: Bayesian Statistics
2.3. Bayesian Thinking
Week 3: Multiple Comparisons, Statistics Power, Pre-Registration
3.1. Type 1 error control
3.2. Type 2 error control
- Assignemnt 3.1.: Positive Predictive Value
- Assignment 3.2.: Optional Stopping
3.3. Pre Registration
Week 4: Statistics Learning Perspectives
4.1. Effect Size
4.2. Cohen's d
4.3. Correlation
- Assignment 4: Calculating Effect Size
Week 5: Confidence intervals, Sample Size justification, P-curve analysis
5.1. Confidence Intervals
5.2. Sample Size Justification
- Assignment 5: Random Variations and Power Analysis
5.3. P-curve analysis
Week 6: Philosophy of Science and Theory
6.1. Philosophy of Science
6.2. The Null is always false
- Assignment 6: Equivalence Testing
6.3. Theory Construction
Week 7: Open Science
7.1. Replications
7.2. Publication Bias
7.3. Open Science
- Assignment 7: Open Science Project
Week 8: Final Exam
Eindhoven University of Technology
Eindhoven University of Technology (TU/e) is a young university, founded in 1956 by industry, local government and academia. Today, their spirit of collaboration is still at the heart of the university community. We foster an open culture where everyone feels free to exchange ideas and take initiatives.
We offer academic education that is driven by fundamental and applied research. Our educational philosophy is based on personal attention and room for individual ambitions and talents. Our research meets the highest international standards of quality. We push the limits of science, which puts us at the forefront of rapidly emerging areas of research.