Module 2: Regression and prediction

Introduction

Regression, prediction, and other tools are very useful to be applied in machine learning for casual inference.

Regression analysis is about discovering correlations between the _outcome _Y and the regression or features X

When we are interested in a certain effect caused by a particular component of X we can partition the elements of X as follows

X = (D,W')'

Where D is the target regressor and W is called control components

When we are using regression, we want to answer two questions:

1. Prediction: How can we use X to predict Y well?
2. Inference: How does Y change if we change a component of X and we leave the rest of the components fixed

Linear regression for prediction

When we are facing a regression problm at a population level, we can assume theoritically that we count on infinite ammounts of data, therefore we can compute a theoritical expected value.

We also can find the Best linear predictor: that is defined as the best rule among all linear rules of the form B'X to predict Y. Best means that the squared error of the prediction is being minimized.

#Prediction quality

Data spliting

A good way to measure the quality of our predictions is to split our dat in two random groups.

1. Group number one wil be the training sample. This is the data we will use to estimate or train our prediction rules.

2. The second group will be the valiation sample. We will use the other part of the data to evaluate the predictive performance of our model.