(statistical-noise-modeling)=
Statistical Noise Modeling
===

Before we can understand what *statistical noise modeling* is we must
first understand basic terminologies. *Statistical noise* describes
unexplained and random fluctuation of data. The origin of this
expression dates back to signal processing, where unwanted electrical or
electromagnetic energy distorted the data-values. As image processing is
based on electrically processing data, similar distortion is to be
expected or at least something similar. All in all *statistical noise
modeling* describes the approach to understand and analyze said random
fluctuation of data. Before we get to *statistical noise modeling* we
require some basic knowledge about statistical mathematics.

In the discrete setting, let $u:\Omega_h \to \R$ denote a clean image
and $f:\Omega_h \to \R$ shall denote a noisy version of $u$. We assume
$u,f$ to be observed instances of multivariate random variables $U,F$.\

````{prf:definition} Probability Space
A *probability space* is defined as a triple $\left(\Omega,E,p\right)$,\
where $\Omega$ describes the sample space, which is the set of all
possible outcomes in a random process, $E$ is called the sample space,
and $p$ denotes a probability measure with $p(\Omega) = 1$ and
$p(A) \geq 0$ for all $A\in E$.

````

````{prf:definition} Conditional Probability
:label: def:conditional_probability
Let $\left(\Omega,E,p\right)$ be a probability-space. Let $A,B\in E$ be
two events and $p(B)>0$. Then we define the conditional probability of
$A$ under *(the occurrence of)* $B$ as:\

```{math}
p(A \mid B)=\frac{p(A\cap B)}{p(B)}.
```

````

One central question is the determination of
$p\left(F=f \mid U=u\right)$, the probability of $f$ under the occurence
of $u$. Therefore, we make the following assumptions with $N$ being the
number of Pixels in our image.

-   Image pixels are getting distorted by noise **independently**, i.e.,
    ```{math}
    \operatorname{cov}(F)=\text{diag}(\sigma_1^2,...,\sigma_N^2)
    ```
    or $X_i ~ \perp\!\!\!\!\perp ~X_j,~\forall i \neq j$. With that, we
    can calculate:\

    ```{math}
    p(F=f\mid U=u)=p\left(\bigcap_{i=0}^{N-1}(F_{i}=f_i)\mid \bigcap_{i=0}^{N-1}(U_{i}=u_i)\right)=\prod_{i=0}^{N-1}{\frac{p\left(F_{i}=f_i \cap U_{i}=u_i\right)}{p\left(U_{i}=u_i\right)}}.
    ```

-   The probability, that the random variable $U_i$ matches the pixel
    value $u_i$ is following a uniform distribution
    $p(U_i = u_i) = \frac{1}{N}$.

With our assumptions and with
[\[def:conditional_probability\]](#def:conditional_probability){reference-type="ref+Label"
reference="def:conditional_probability"} we see, that the conditional
probability $p(F=f\mid U=u)$ depends on the one-dimensional probability
$p\left(F_{i}=f_i \cap U_{i}=u_i\right)$. We will investigate this
probability for different noise models.

````{prf:example} Different noise models
:label: ex:noise_models

1.  **Additive Gaussian noise**:\
    Here we assume $f=u+\varepsilon$, where
    $\varepsilon\sim \mathcal{N}(0,\Sigma=\sigma^2\mathcal{I})$ , the
    probability-density-function of $\varepsilon$ is:
    ```{math}
    p_\varepsilon(x)=\frac{1}{\sqrt{2\pi\sigma^2}}e^{-\frac{\|x\|_2^2}{2\sigma^2}}.
    ```
    With $f=u+\varepsilon \Longleftrightarrow \varepsilon=f-u$ we get
    $F\sim\mathcal{N}(u,\Sigma)$. Hence,
    ```{math}
    p(f\cap u)=p_\varepsilon(f-u)=\frac{1}{\sqrt{2\pi\sigma^2}}e^{-\frac{\|f-u\|_2^2}{2\sigma^2}}
    ```
    and also
    ```{math}
    p(f\mid u)=\frac{p_\varepsilon(f-u)}{p(u)}.
    ```
    The expected value and the variance are given as
    $\mathbb{E}(F)=u,~ \mathbb{V}(F_i)=\sigma^2$ .

2.  **Poisson noise**:\
    Here the noise has only discrete values $k\in \mathbb{N}$ as it is
    (usually) a result of counting rare events with low power in a small
    exposure time. The probability mass function is denoted by
    ```{math}
    p_\lambda(X=k)=\frac{\lambda^k e^{-\lambda}}{k!}
    ```
    for a parameter $\lambda>0$ and we write
    $X\sim \mathcal{P}(\lambda)$. Let $u_i\in\R^+,f_i\in \N$ with
    $F\sim\mathcal{P}(u)$ and we have
    ```{math}
    p(f\cap u)= p_u(f) = \frac{u^f \cdot e^{-u}}{f!}\,.
    ```
    We calculate the expected value and the variance
    ```{math}
    \begin{aligned}
                        \mathbb{E}(F) &= \sum_{k\in\N}{k \cdot p(F=k)} = \sum_{k\in\N} k \cdot \frac{u^k}{k!}e^{-u} =u \sum_{k=1}^{+\infty}\frac{u^{(k-1)}}{(k-1)!}  e^{-u} = u\cdot e^{u}e^{-u}=u\\
                        \mathbb{E}(F^2) &= \sum_{k\in\N}k^2 \cdot p(F=k) = \sum_{k=1}^{+\infty}k\cdot{\frac{u^{(k)}}{(k-1)!}}  e^{-u} = u\cdot \sum_{k=0}^{+\infty}(k+1)\cdot{\frac{u^{(k)}}{(k)!}}  e^{-u}\\
                        &= u\cdot \left(\sum_{k=0}^{+\infty}{k\cdot{\frac{u^{(k)}}{(k)!}}}+\sum_{k=0}^{+\infty}{1\cdot{\frac{u^{(k)}}{(k)!}}} \right) e^{-u}=u\cdot \left(u\cdot e^{u}+e^{u}\right)e^{-u}=u^2+u\\
                        \mathbb{V}(F)&=\mathbb{E}(F^2)-\mathbb{E}(F)^2=u^2+u-u^2=u.
                    
    \end{aligned}
    ```

````