Author: Carl McBride Ellis (LinkedIn)

The following represents a selection of my kaggle notebooks

1. eda (exploratory data analysis)

• Anscombe's quartet and the importance of EDA (+ dataset)
• Absolute beginners Titanic 'EDA' using dabl
• Exploratory data analysis using pandas pivot table
• Use case example: Jane Street: EDA of day 0 and feature importance
• Use case example: Riiid: EDA and feature importance
• Use case example: Ventilator Pressure: EDA and simple submission

2. data cleaning / preparation

• Filtering outliers using the Isolation Forest
• Data anonymization using Faker (Titanic example)
• AWS PyDeequ unit tests to measure data quality
• Naïve Dataset Distillation

3. baselines

An important practice is to create a baseline with which to compare future work against:

• Titanic: Machine Learning from Disaster: Titanic baseline: an all zeros csv file
• House Prices: Create a House Prices baseline score (0.42577)
• OpenVaccine: COVID-19: Baseline: mean values + Gaussian noise ≈ 0.495
• Mechanisms of Action (MoA) Prediction: MoA Baseline = 0.02398 (∀ 0's = 0.13073)
• INGV - Volcanic Eruption Prediction: The mean volcano
• Jane Street Market Prediction: A random walk down Jane Street...
• Store Sales - Time Series Forecasting: Store Sales: Using the average of the last 16 days
• UltraMNIST baseline: All class 20

4. feature selection / engineering

• Feature importance using the LASSO
• Feature selection using the Boruta-SHAP package
• Recursive Feature Elimination (RFE) example
• House Prices: Permutation Importance example
• SHAP Permutation explainer + random "probe"
• What is Adversarial Validation?
• Jane Street: t-SNE using RAPIDS cuML
• Synthanic feature engineering: Beware!

5. classification / regression

This is a collection of my python example scripts for either classification, using the Titanic: Machine Learning from Disaster competition data, or regression, for which I use the House Prices: Advanced Regression Techniques competition data:

algorithm	classification	regression
Logistic regression	link	---
Generalized Additive Models (GAM)	link	---
Iterative Dichotomiser 3 (ID3)	link	---
Decision tree	link	---
Regularized Greedy Forest (RGF)	link	link
XGBoost	---	link
TabNet	link	link
Neural networks (using keras)	link	link
Gaussian process	link	link
Hyperparameter grid search	link	link

• Classification using TensorFlow Decision Forests
• Titanic in pure H2O.ai
• Predict house prices using H2O.ai (regression)
• Automatic tuning of XGBoost with XGBTune
• MNIST with no neural network
• PyTorch Tabular: Gated Additive Tree Ensemble

6. conformal prediction

• Regression prediction intervals with MAPIE
• Prediction intervals: Quantile Regression Forests
• Locally-weighted conformal regression
• Classifier probability calibration using Venn-ABERS predictors (winner of the Gabriel Preda "Best recent Kaggle Notebook" prize)

7. time series and forecasting

• Time series: A simple moving average (MA) model
• Time series decomposition: Naive example
• LSTM time series prediction: sine wave example
• LSTM time series + stock price prediction = FAIL
• Interrupted time series analysis: Causal Impact
• Temporal Convolutional Network using Keras-TCN
• Plotting OHLC and V ticker data using mplfinance
• Correlograms of 14 cryptocurrencies (1 day)
• Granger causality testing for 1 day along with Granger causality Part II: The Movie
• Store Sales: Day of the Week model
• TPS Jan 2022: A simple average model (no ML)
• TPS Jan 2022: Prophet + holidays + GDP regressor
• Multivariable time series forecasting: Linear tree

Prediction intervals

• Probabilistic forecasting using GluonTS: Bitcoin
• [PFI Starter] Skforecast example - starter notebook provided for the "Probabilistic forecasting I: Temperature" competition

8. ensemble methods

• Ensemble methods: majority voting example
• ML-Ensemble example using House Prices data
• Stacking ensemble using the House Prices data

9. explainability

• Explainability, collinearity and the variance inflation factor (VIF)
• KISS: Small and simple Titanic models
• House Prices: my score using only 'OverallQual' and also a simple two variable model
• Titanic explainability: Why me? asks Miss Doyle
• TabNet and interpretability: Jane Street example
• GPU accelerated SHAP values: Jane Street example

10. causality

• Causal Forests Double ML example using EconML
• Interrupted time series analysis: Causal Impact

11. statistics

• Animated histogram of the central limit theorem
• Hypothesis testing: The two sample t-test, p-value and power
• Explainability, collinearity and the variance inflation factor (VIF)

12. didactic notebooks

• Beautiful math in your notebook: a guide to using $\LaTeX$ math markup in kaggle notebooks.
• Titanic: In all the confusion... which looks at the confusion matrix, ROC curves, $F_1$ scores etc.
• Classification: How imbalanced is "imbalanced"? - (mentioned in "Notebooks of the week: Hidden Gems")
• Overfitting and underfitting the Titanic
• False positives, false negatives and the discrimination threshold
• Introduction to the Regularized Greedy Forest (using rgf_python)
• Extrapolation: Do not stray out of the forest!
• Titanic: some sex, a bit of class, and a tree...
• The Lehmer RNG algorithm for seed=42
• Pearson correlation coefficient, mutual information (MI) and Predictive Power Score (PPS) - a simple comparison

13. generative AI

• StableDiffusion: text-to-image with KerasCV
• [Gemma] I am replacing myself with an LLM

14. miscellaneous

• Titanic leaderboard: a score > 0.8 is great!
• House Prices: How to work offline (+ dataset)
• Pandas one-liners
• The latest trends in data science
• The Titanic using SQL
• Some pretty t-SNE plots
• Encuesta kaggle 2021: ¿España es diferente?
• How much do people on kaggle earn by country (2021)
• All in a pickle: Saving the Titanic - Saving our machine learning model to a file using pickle
• Machine learning review papers on arXiv [polars]

Geospatial analysis

• Choropleth map of kaggle Grandmaster locations
• Smartphone 2022: A look at the ground truth maps
• Animating the path of a smartphone GPS signal

Finance related

• S&P 500 daily returns: Normal and Cauchy fits
• Bitcoin candlestick chart (2021)
• Irrational Exuberance? S&P vs Bitcoin

fun with the meta kaggle dataset

The Meta Kaggle dataset consists of data regarding the kaggle site

• Kaggle in numbers - updated almost daily
• Simple EDA of kaggle Grandmasters - updated almost daily
• Number of active Kaggle users
• Notebooks: Number of views, and days, per vote
• kaggle discussions: busiest time of the day? - (mentioned in "Notebooks of the week: Hidden Gems")
• The kaggle working week
• WordCloud of gold medal winning notebook titles
• Shakeup interactive scatterplot maker
• Shakeup scatterplots: Boxes, strings and things...
• When will my notebook get its medal?

All the best!