Maximum a-posteriori estimation

3.2. Maximum a-posteriori estimation#

Instead of maximizing \(p(f\mid u)\) we want to maximize, the posterior

\[p(U=u\mid F=f)\]

The probability of the unknown image being \(u\) under observation of the noisy image being \(f\).

Theorem 3.1 (Bayes’ Theorem)

Let \((\Omega,E,p)\) be a probability space and let \(A,B \in E\) be two events with \(p(B)>0\). Then,

\[p(A|B) = \frac{p(B|A)\cdot p(A)}{p(B)}.\]

Proof. One can show this theorem by extending the fracture with \(1\) and simply inserting the definition:

\[p(A|B) = \frac{p(A\cap B)}{p(B)} = \frac{p(A\cap B) \cdot p(A)}{p(B)\cdot p(A)}= \frac{p(B|A)\cdot p(A)}{p(B)}.\]

◻

With Bayes’ theorem we can calculate the maximum a-posteriori estimation (MAP-estimation) using again the negative log

\[\begin{aligned} \operatorname{arg\,min}_{u\in X}-\log{p(u|f)} &= \operatorname{arg\,min}_{u\in X}-\log{p(f|u)}-\log{p(u)} + \log{p(f)}\\ &= \operatorname{arg\,min}_{u\in X}-\log{p(f|u)}-\log{p(u)}\,. \end{aligned}\]

The MAP-estimation gives us an unknown image, which fits the obeservation \(f\) with maximal probability. To calculate it with Bayes’ theorem, we only need the probability of \(u\) under observation of \(f\) and the probability of the event of \(u\), which is called the prior. We introduce the Gibbs prior \(p(U=u) = e^{-\alpha R(u)}\) with a positive weighting parameter \(\alpha>0\) and a function \(R(u)\), which introduces prior information of the problem. We calculate now the MAP-estimator with Gibbs prior

\[\begin{aligned} \operatorname{arg\,min}_{u\in X}-\log{p(u|f)}= \operatorname{arg\,min}_{u\in X}{ -\log{p(f|u)} + \alpha R(u)}, \end{aligned}\]

Revisiting the calculated conditional probabilities for the different noise models Example 3.1, we have the following MAP-estimators for the Gaussian and Poisson noise models.

\[\begin{aligned} \operatorname{MAP}_G (f) &= \operatorname{arg\,min}_{u\in X}\frac{\|f-u\|_2^2}{2\sigma^2} + \alpha R(u) \\ \operatorname{MAP}_P (f) &= \operatorname{arg\,min}_{u\in X}\langle \mathbf{1}, u\rangle - \langle f, \log(u)\rangle + \alpha R(u) \end{aligned}\]

We can generalize these MAP-estimators with Gibbs priors to the following class of optimization problems

\[\operatorname{arg\,min}_{u\in X}{\underset{\color{green}\text{data fidelity term}}{D(u_i, f)} + \underset{\color{orange}\text{regularization term}}{\alpha R(u)}}\]

It remains for us to study, based on a given data fidelity term and regularizer,

  • the existence and uniqueness of minimizers \(u \in X\),

  • the regularity of the solution \(u\),

  • the calculation or approximation of \(u\).

  • Which regularizer \(R\) and data term \(D\) is the best model for our application?

We will answer these questions in the upcoming chapters by applying tools from variational calculus and convex analysis.