Locally weighted Regression (loess)

Locally weighted regression, or loess, is a way of estimating a regression surface through a multivariate smoothing procedure, fitting a function of the independent variables locally and in a moving fashion analogous to how a moving average is computed for a time series.

Paper Recreation:

In the notebook section_5.ipynb I recreated every single plot from Section 5 of Cleveland's Paper [1]. Using the same dataset, I got the same results as the ones they displayed, so the algorithm recreation was working.

Personal Experimentation:

In script synth_data_creation.py I created the following 4 surfaces in $R^3$:

Then I added some noise:

And estimated them with Loess using different parameters, for example this is the result with $f=0.5$ and $d=2$:

In the notebook experimentation.ipynb, using the data created by the previous script, I created the plots above, and for a better look at the results, I plotted the level curves of the estimations, and how they compare with the real data, and fitted data:

Plane:

Saddle:

Cubic:

Absolute:

References:

[1] William S. Cleveland and Susan J. Devlin (1968). Locally Weighted Regression: An Approach to Regression Analysis by Local Fitting. Journal of the American Statistical Association, Vol. 83, No. 403 (Sep., 1988), pp. 596-610