Python implementation of a case study in Module 2 of the MITProfessionalX course "Data Science: Data to insights".
The case study is: "Module 2 Case Study - Regression and prediction". This case study is about doing linear regression in R on wages data. I did it in python using two different libraries.
Points of interest
Also, it is interesting how features were created from the existing ones in the "flexible" model by calculating interactions between existing features. For this task patsy was really handy.
Cross validating does not make fully sense in this case but it is interesting to see anyway.
Our goals are:
Predict wages using various characteristics of workers.
Assess the predictive performance using adjusted MSE and R^2 , and out-of-sample MSE and R^2.
Data is from the March Supplement of the U.S. Current Population Survey, year 2012.
Focus on the single (never married) workers with education levels equal to high-school, some college, or college graduates.
The sample is of size n ≈ 4,000.
The outcome Y is hourly wage, and X are various characteristics of workers.