This notebook provides a comprehensive guide to understanding Simpson's Paradox through real-world examples and interactive visualizations. Simpson's Paradox is a phenomenon in probability and statistics, where a trend appears in several groups of data but disappears or reverses when these groups are combined.
- Identify Simpson's Paradox in real-world scenarios.
- Understand the role of confounding variables.
- Visualize how seemingly contradictory results can occur.
- Experiment with creating your own paradox scenarios.
- Learn how to avoid being misled by aggregate data.
- Introduction: A brief explanation of Simpson's Paradox.
- Medical Treatment Scenario: Explore a classic example involving kidney stone treatments in two hospitals, demonstrating how overall success rates can be misleading when a confounding variable (stone size) is not considered.
- University Admissions Scenario: An interactive example showing how overall university admission rates by gender can be reversed when analyzed by individual departments. You can input your own numbers to see the paradox in action.
- Custom Paradox Experiment: An interactive section where you can define your own scenario and input data to attempt to create Simpson's Paradox.
- Summary and Key Takeaways: A detailed summary of the concept, real-world implications, how to detect and avoid the paradox, and practice exercises.
Run the code cells sequentially to follow the examples and interact with the data. The visualizations will help you grasp the concept intuitively. The interactive sections allow you to experiment with different numbers and observe how the paradox manifests.
This notebook requires the following libraries:
numpypandasmatplotlibseabornscipy
These are commonly available in Google Colab environments.
Feel free to modify and experiment with the code. If you have suggestions for improvement, you can adapt this notebook for your own use.