[FDA] 6. Functional CLT

In statistics, Central Limit Theorem means the sample average connverges to normal distribution for cenrain random variables. We will discuss CLT in the functional spaces.

 

Expectation

Firstly, we need to define expectation in the functional space, $\mathcal{H}$.

Let $X$ be a random element of $\mathcal{H}$. We say that $\mu \in \mathcal{H}$ is the mean of $X$ if it satisfies
$$E \langle X, x \rangle = \langle \mu, x \rangle \ \text{for all} \ x\in \mathcal{H}$$

For example, Let $X \in \mathbb{R}^d$ be a random vector. Then we define $\mu \in \mathbb{R}^d$ as

$$E \langle X, x\rangle = E \sum_{i=1}^d X_i x_i = \sum_{i=1}^d E[X_i]x_i = \sum_{i=1}^d \mu_i x_i = \langle \mu, x \rangle.$$

Thus, $\mu$ is a d-dimensional vector whose element is $E[X_i]$.

 

And expectation has two following properties:

•  $\lVert E X \rVert \le E \lVert X \rVert$ (contractive property)
• if $L$ is a bounded linear operator, then $EL(X) = L(EX)$.

 

Covariances

In general Hilbert spaces, we say that $C$ is the covariance operator of $X$ if

$$C(x) = E[ \lVert X-EX,x \rVert (X-EX)]\ \text{for all} \ x\in\mathcal{H}.$$

It exists only when finite $E\lVert X \rVert ^2$.

 

Any covariance operator, $C$, also has some property.

• $C$ is symmetric
• $C$ is nonnegative-definite
• $C$ is Hilbert-Schmidt
• $C$ is also called nuclear or a trace class operator in that it satisfies $\sum \lambda_u < \infty$ ($\lambda_i$ are its eigenvalues).

 

Gaussian

We need to know how to define Gaussian in functional space. We say that a random function $X \in \mathcal{H}$ is Gaussian if $\langle X, y \rangle$ is normally distributed (with finite variance) for any $y\in\mathcal{H}$.

If $X$ is Gaussian, it has following properties:

• $X$ is strongly integrable and so $EX$ exists.
• $E\lVert X \rVert ^2 < \infty$ and so it has a covariance operator $C$.
• $E\lVert X \rVert ^p < \infty$ for any positive p.

This means that

$$\langle X, x\rangle \sim  \mathcal{N} ( \langle \mu, x \rangle, \langle C(x), x \rangle).$$

 

Characteristic functions

Last one that is needed to understand and define functional CLT properly is Characteristic functions.

The characteristic function of a random function $X$ is defined as

$$\psi(x) = E \exp({i\langle X,x\rangle})$$

We know that $X$ and $Y$ have the same distribution if and only if they have the same characteristic functions. And if $X$ is Gaussian with mean $\mu$ and covariance $C$, then

$$\psi(x) = \exp(i \langle \mu,x\rangle - \frac{1}{2} \langle C(x), x \rangle ).$$

 

Central Limit Theorem

Finally, we can reach functional CLT. Until now, we build backgrounds for proper understanding of functional CLT (especially in Hilbert spaces).

Suppose $\{Y_n:n=1,2,\dots \}$ is an iid sequence of random functions in $\mathcal{H}$ which are square integrable, $E\lVert Y_n \rVert ^2 < \infty.$ Let $\mu$ denote their mean and $C$ their covariance. Then we have
$$ \frac{1}{\sqrt{N}} \sum_{i=1}^N (Y_n -\mu) \overset{\mathcal{D}}{\to} \mathcal{N} (0, C)$$

 

Intuitively, the different thing compared to the CLT we know earlier is that 'random variables' is replaced with 'random functions'. To prove this, there are several concepts such as Karhunen-Loéve expansion, Pracevals theorem, and Markov inequality etc. I will skip the proof.

 

After making our random functions to follow the Gaussian distribution asymptotically, our next interests will be hypothesis test, confindence interval etc. Thus, I will introduce some of these concepts.