# Shae's Ramblings

Stuff I find slightly meaningful and close to achievable

## Convex sum of diracs on a bounded subset of the real line

In this post, we aim to prove that any continuous density $x\mapsto p(x)$ over a bounded subset $\mathcal{X} \subset [a,b]$ can be approximated by a convex combination of deltas $\delta_{x_i}$: there is always a $N$ for which we have with $\varepsilon$-accuracy that $p(x) \approx p_n(x) = \sum_{i=1}^n\gamma_i \delta_{x_i}(x)$ for all $n > N$.

Because any bounded subset in $\mathbb{R}$ can be finitely covered by closed intervals $[a_i, b_i]$, we can prove our result for each interval and use the fact that for any $c,d\in\mathcal{X}$ bounded, the integral is equal to

$$\int_c^dp(x)\mathrm{d}x = \sum_{i=1}^n\int_{a_i}^{b_i}p(x)\mathrm{d}x$$

Thus to simplify the problem, we can assume $\mathcal{X} = [a,b]$ some interval.

## Weak convergence

For continuous distributions $f,g$ on $\mathcal{X}$ define the inner product

$$\langle f,g\rangle \triangleq \int_\mathcal{X}f(x)g(x)\,\mathrm{d}x$$
We say a sequence $f_n$ converges weakly to a limit $f_\star$ if for every distribution $g$ on $\mathcal{X}$ we have that $\langle f_n, g\rangle \to \langle f_\star, g\rangle$ as $n\to\infty$.

## The proof

We can use Riemann sums to prove the statement. First, notice that because $\int p(x)\mathrm{d}x = 1$, we can for any $n$ find a partition $a = x_0 < x_1 < \dots < x_n$ with gaps $\Delta x_i = x_{i+1} – x_i$ so that

$$\sum_{i=0}^{n-1}p(x_i^\star)\Delta x_i = 1 + u_n$$
with $u_n \to 0$ as $n$ tends to infinity.
We can assume that $u_n \leq 0$ by taking $p(x_i^\star) = \inf\{p(x)\mid x_i \leq x \leq x_{i+1}\}$ which is attained on a closed interval by the continuity of $p$. We can therefore for any $n$ construct the following $\gamma_i$:

$$\gamma_i \triangleq p(x_i^\star)\Delta x_i – u_n/n$$
so that we always have $\gamma_i \geq 0$ and $\sum_{i=0}^{n-1}\gamma_i = 1 + u_n – u_n = 1$.

Finally, consider the convex sum $p_n = \sum_{i=0}^{n-1} \gamma_i \delta_{x_i^\star}$, we have

$$\langle p_n , f\rangle = \sum_{i=0}^{n-1} \gamma_i f(x_i^\star) = \sum_{i=0}^{n-1}\left( p(x_i^\star)\Delta x_i – u_n/n \right) f(x_i^\star)$$
which expands into

$$\sum_{i=0}^{n-1} \gamma_i g(x_i^\star) = \sum_{i=0}^{n-1}p(x_i^\star)f(x_i^\star)\Delta x_i – \dfrac{u_n}{n}\sum_{i=0}^{n-1}f(x_i^\star)$$
which converges to $\langle p, f \rangle$ because $\dfrac{u_n}{n}\sum_{i=0}^{n-1} f(x_i^\star) \to 0$ from the bounded nature of $f$.