Statistical Noise Modeling

3. Statistical Noise Modeling#

Before we can understand what statistical noise modeling is we must first understand basic terminologies. Statistical noise describes unexplained and random fluctuation of data. The origin of this expression dates back to signal processing, where unwanted electrical or electromagnetic energy distorted the data-values. As image processing is based on electrically processing data, similar distortion is to be expected or at least something similar. All in all statistical noise modeling describes the approach to understand and analyze said random fluctuation of data. Before we get to statistical noise modeling we require some basic knowledge about statistical mathematics.

In the discrete setting, let \(u:\Omega_h \to \R\) denote a clean image and \(f:\Omega_h \to \R\) shall denote a noisy version of \(u\). We assume \(u,f\) to be observed instances of multivariate random variables \(U,F\).\

Definition 3.1 (Probability Space)

A probability space is defined as a triple \(\left(\Omega,E,p\right)\),
where \(\Omega\) describes the sample space, which is the set of all possible outcomes in a random process, \(E\) is called the sample space, and \(p\) denotes a probability measure with \(p(\Omega) = 1\) and \(p(A) \geq 0\) for all \(A\in E\).

Definition 3.2 (Conditional Probability)

Let \(\left(\Omega,E,p\right)\) be a probability-space. Let \(A,B\in E\) be two events and \(p(B)>0\). Then we define the conditional probability of \(A\) under (the occurrence of) \(B\) as:\

\[p(A \mid B)=\frac{p(A\cap B)}{p(B)}.\]

One central question is the determination of \(p\left(F=f \mid U=u\right)\), the probability of \(f\) under the occurence of \(u\). Therefore, we make the following assumptions with \(N\) being the number of Pixels in our image.

  • Image pixels are getting distorted by noise independently, i.e.,

    \[\operatorname{cov}(F)=\text{diag}(\sigma_1^2,...,\sigma_N^2)\]

    or \(X_i ~ \perp\!\!\!\!\perp ~X_j,~\forall i \neq j\). With that, we can calculate:\

    \[p(F=f\mid U=u)=p\left(\bigcap_{i=0}^{N-1}(F_{i}=f_i)\mid \bigcap_{i=0}^{N-1}(U_{i}=u_i)\right)=\prod_{i=0}^{N-1}{\frac{p\left(F_{i}=f_i \cap U_{i}=u_i\right)}{p\left(U_{i}=u_i\right)}}.\]

  • The probability, that the random variable \(U_i\) matches the pixel value \(u_i\) is following a uniform distribution \(p(U_i = u_i) = \frac{1}{N}\).

With our assumptions and with [def:conditional_probability]{reference-type=”ref+Label” reference=”def:conditional_probability”} we see, that the conditional probability \(p(F=f\mid U=u)\) depends on the one-dimensional probability \(p\left(F_{i}=f_i \cap U_{i}=u_i\right)\). We will investigate this probability for different noise models.

Example 3.1 (Different noise models)

  1. Additive Gaussian noise:
    Here we assume \(f=u+\varepsilon\), where \(\varepsilon\sim \mathcal{N}(0,\Sigma=\sigma^2\mathcal{I})\) , the probability-density-function of \(\varepsilon\) is:

    \[p_\varepsilon(x)=\frac{1}{\sqrt{2\pi\sigma^2}}e^{-\frac{\|x\|_2^2}{2\sigma^2}}.\]

    With \(f=u+\varepsilon \Longleftrightarrow \varepsilon=f-u\) we get \(F\sim\mathcal{N}(u,\Sigma)\). Hence,

    \[p(f\cap u)=p_\varepsilon(f-u)=\frac{1}{\sqrt{2\pi\sigma^2}}e^{-\frac{\|f-u\|_2^2}{2\sigma^2}}\]

    and also

    \[p(f\mid u)=\frac{p_\varepsilon(f-u)}{p(u)}.\]

    The expected value and the variance are given as \(\mathbb{E}(F)=u,~ \mathbb{V}(F_i)=\sigma^2\) .

  2. Poisson noise:
    Here the noise has only discrete values \(k\in \mathbb{N}\) as it is (usually) a result of counting rare events with low power in a small exposure time. The probability mass function is denoted by

    \[p_\lambda(X=k)=\frac{\lambda^k e^{-\lambda}}{k!}\]

    for a parameter \(\lambda>0\) and we write \(X\sim \mathcal{P}(\lambda)\). Let \(u_i\in\R^+,f_i\in \N\) with \(F\sim\mathcal{P}(u)\) and we have

    \[p(f\cap u)= p_u(f) = \frac{u^f \cdot e^{-u}}{f!}\,.\]

    We calculate the expected value and the variance

    \[\begin{aligned} \mathbb{E}(F) &= \sum_{k\in\N}{k \cdot p(F=k)} = \sum_{k\in\N} k \cdot \frac{u^k}{k!}e^{-u} =u \sum_{k=1}^{+\infty}\frac{u^{(k-1)}}{(k-1)!} e^{-u} = u\cdot e^{u}e^{-u}=u\\ \mathbb{E}(F^2) &= \sum_{k\in\N}k^2 \cdot p(F=k) = \sum_{k=1}^{+\infty}k\cdot{\frac{u^{(k)}}{(k-1)!}} e^{-u} = u\cdot \sum_{k=0}^{+\infty}(k+1)\cdot{\frac{u^{(k)}}{(k)!}} e^{-u}\\ &= u\cdot \left(\sum_{k=0}^{+\infty}{k\cdot{\frac{u^{(k)}}{(k)!}}}+\sum_{k=0}^{+\infty}{1\cdot{\frac{u^{(k)}}{(k)!}}} \right) e^{-u}=u\cdot \left(u\cdot e^{u}+e^{u}\right)e^{-u}=u^2+u\\ \mathbb{V}(F)&=\mathbb{E}(F^2)-\mathbb{E}(F)^2=u^2+u-u^2=u. \end{aligned}\]