Example notebook: kernel ridge regression #320
Conversation
| # $$ | ||
| # \mathbf{w} = (\mathrm{X}^\top \mathrm{X} + \lambda \mathbb{1})^{-1} \mathrm{X}^\top \mathbf{y} | ||
| # $$ | ||
| # using the [matrix inversion lemma](https://tlienart.github.io/pub/csml/mtheory/matinvlem.html#basic_lemmas) |
There was a problem hiding this comment.
Maybe rather link to the Wikipedia article about the Woodbury matrix identity?
There was a problem hiding this comment.
I still think it would be more natural to link to Wikipedia?
There was a problem hiding this comment.
I found the blog post more instructive/helpful than the wikipedia article (and the latter is straightforward to find for anyone who wants it)..
…tions.jl into st/examples--kernel-ridge-regression
| else | ||
| title = string(nameof(typeof(kernel))) | ||
| end | ||
| scatter(x_train, y_train; label=nothing) |
There was a problem hiding this comment.
I didn't know that nothing works, I always use label="" to hide the label for a specific series. However, it seems you want to remove labels for all series here in which caseI guess you can just use legend=false
| scatter(x_train, y_train; label=nothing) | |
| scatter(x_train, y_train; legend=false) |
is simpler.
| p = plot!( | ||
| x_test, | ||
| y_pred; | ||
| label=nothing, |
There was a problem hiding this comment.
This is not needed if legend=false:
| label=nothing, |
Co-authored-by: David Widmann <devmotion@users.noreply.github.com>
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
|
This looks really good! Great job! |
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
…iaGaussianProcesses/KernelFunctions.jl into st/examples--kernel-ridge-regression
|
|
||
| ## | ||
| # ## Kernel ridge regression | ||
| # Instead of constructing the feature matrix explicitly, we can use *kernels* to replace inner products of feature vectors with a kernel evaluation: $\langle \phi(x), \phi(x') \rangle = k(x, x')$ or $\mathrm{X} \mathrm{X}^\top = \mathrm{K}$, where $\mathrm{K}_{ij} = k(x_i, x_j)$. |
There was a problem hiding this comment.
Here talking about XX^\top=K is a bit misleading, it does not sound like we are using any feature mapping, maybe \phi(X)\phi(X)^\top
There was a problem hiding this comment.
well, X is the "extended" x with feature columns, each column of X is one \phi applied to the input x
There was a problem hiding this comment.
but it's a bit ambiguous, you're right. maybe call "featurized X" \tilde{X} instead?
There was a problem hiding this comment.
NB: XX' = K is exact for the LinearKernel k(x,x') = xx' :)
No description provided.