ReproducingKernelHilbertSpace

The Role of Inner Products and Reproducing Kernel Hilbert Space(RKHS) in Stochastic Processes

Inner Products and RKHS

A Reproducing Kernel Hilbert Space (RKHS) is a particular kind of Hilbert space where functions serve as elements. The inner product in this space can be induced by a kernel function $k(s, t)$. Specifically, for any two functions $f, g \in \mathcal{H}$ within an RKHS $\mathcal{H}$, the inner product is given by:

$$\langle f, g \rangle = \int \int f(s) \overline{g(t)} k(s, t) ds dt$$

This construct ensures that the inner product is a strict inner product in $\mathcal{H}$.

Riesz Representation Theorem

Theorem

For every continuous linear functional $F$ on a Hilbert space $\mathcal{H}$, there exists a unique element $y$ in $\mathcal{H}$ such that:

$$F(x) = \langle x, y \rangle \forall x \in \mathcal{H}$$

Implications for RKHS

In the case of RKHS, the Riesz Representation Theorem ensures that the evaluation functional $F(f) = f(s)$ can be represented as an inner product:

$$F(f) = f(s) = \langle f, k(s, \cdot) \rangle$$

This verifies that the kernel $k(s, \cdot)$ effectively represents function evaluations and functionals in the RKHS, thereby making $k(s, \cdot)$ a reproducing kernel.

Stochastic Processes and Covariance Function

When $f(s) = X(s)$ and $g(s) = X(t)$ for a stochastic process $X \in \mathcal{H}$, the covariance function $C(s, t)$ is naturally represented as:

$$C(s, t) = \langle X(s), X(t) \rangle$$

In this special case, $C(s, t)$ becomes a true inner product in $\mathcal{H}$. Here, $k(s, t)$ serves as the autocovariance kernel function of $X$.