(existence-and-uniqueness-of-minimizers)= # Existence and uniqueness of minimizers Before we will discuss approaches to solve our minimization problem {eq}`eq:variational_model` we want to ensure that a solution exist and discuss guarantees for unique solutions. Since this section will be quite theoretical, we are presenting some necessary prerequisites. We say that a functional $E:X\to \R$ is **proper** if for a $c\in\R$ it holds $E(u) > c$ for all $u\in X$ and there exist a $v\in X$ such that $E(v)<\infty$. In other words we call a functional proper if it is bounded from below and not infinity everywhere. This means that we do not have to deal with trivial special cases. We call a functional $E:X\to \R$ **lower semi-continuous** (lsc) if it satisfies $\liminf_{v\to u}E(v) \geq E(u)$ for all $u\in X$. Meaning that we allow the functional to have jumps, but only to lower values. This property ensures that, when we approach a minimum with a minimizing sequence, we will still reach it. It is obvious that any continous functional is lower semi-continuous. We say that a functional $E:X\to \R$ is **convex** if for any $u, v\in X$ and $\lambda \in (0,1)$ it holds $E((1-\lambda)u+\lambda v)\leq (1-\lambda)E(u) + \lambda E(v)$. We say the functional is **strictly convex** if the inequality is strict. Geometrically speaking for a convex functional its graph is below the connecting line segment of two points. This property ensures that local minimizers are global ones. At last we say that a set $K\subset X$ is sequentially **compact** if for any sequence $(u^k)_{k\in\N}$ there exist a converging subsequence $(u^{k^l})_{l\in\N}$ with $\lim_{l\to\infty}u^{k^l} = \bar{u}$. This last property can be quite restrictive since it is not obvious in infinite dimensional spaces. In finite dimensions we have with the Heine-Borel theorem that every closed and bounded set is compact. In infinite dimension this is not true. Even though a given sequence is bounded, it can still 'escape' via the dimensions and thus not provide a convergent subsequence. We illustrate the behavior with the simple sequence ```{math} \begin{aligned} u^0 &= (1,\ 0,\ 0,\ 0,\ \dots)\\ u^1 &= (0,\ 1,\ 0,\ 0,\ \dots)\\ u^2 &= (0,\ 0,\ 1,\ 0,\ \dots)\\ u^3 &= (0,\ 0,\ 0,\ 1,\ \dots) \end{aligned} ``` We see that all $u^k$ have norm one and the pointwise limit is the zero sequence. We also observe that we cannot extract a converging subsequence to this limit. We can now apply the *direct method of the calculus of variations* to show the existence of a minimizer for a functional $E$. ````{prf:theorem} Existence of Minimizers :label: thm:existence Let $X$ be a topological vector space. Let $E:X\to \R$ be proper and lower semi-continuous with compact sublevelsets in $X$. Then there exist a minimizer of $E$ in $X$. ```` ````{prf:proof} We start by defining a minimizing sequence $(u^n)_{n\in\N}$, that is, a sequence $E(u^n)\to \inf_{u\in X}E(u)$. Such a sequence exist, because $E$ is proper. We extract a subsequence $(u^{n^k})_{k\in\N}$, which satisfies $E(u^{n^k})\leq L$, meaning that it lies within a sublevelset of $E$: $(u^{n^k})_{k\in\N} \subset S_L\coloneqq \{u\in X\mid E(u)\leq L \}$. Since the sublevelsets of $E$ are compact, we can extract another subsequence, which converges towards a limitpoint $\bar{u}\in X$. We use now the lower semi-continuity of $E$ to conclude that this limitpoint is a global minimizer of $E$ ```{math} \inf_{u\in X}E(u) = \lim_{n\to\infty}E(u^n) = \lim_{l\to\infty} E(u^{n^{k^l}}) \geq E(\bar{u}) \geq \inf_{u\in X}E(u)\,. ```  ◻ ```` Taking a closer look at [\[thm:existence\]](#thm:existence){reference-type="ref+Label" reference="thm:existence"} we see that the existence of a minimizer of $E$ does not so much rely on $E$ but rather on the choice of the space $X$ and its topology. Thus for a given functional $E$ we need to choose our space $X$, such that $E$ is proper, lower semicontinous and has compact sublevelsets in $X$. We note that a (semi-)norm on a space $X$ is proper and lower semi-continous on $X$. The most common data and regularization terms are (semi-)norms. Further, we can exploit compact embeddings to receive compact sublevelsets of norms. ````{prf:example} Gradient Regularization There exist a minimizer of ```{math} E(u) = \frac{1}{2}\|u-f\|_2^2 + \frac{\alpha}{2} \|\nabla u\|_2^2 ``` in $L^2(\R^n)$ using the Rellich-Kondrachov theorem. ```` ````{prf:proof} We have the Banach space $(X, \|\cdot\|_X) = (L^2(\R^n), \|\cdot\|_2)$ the Banach space norm $\|\cdot \|_2$ is trivially continous, thus lower semi-continuous, since it induces the (strong) topology. We define the regularizer $R(u) = \begin{cases} \frac{1}{2}\|\nabla u\|_2^2 \quad &\text{if } u\in W^{1,2}(\R^n)\,,\\ +\infty &\text{else.} \end{cases}$ and see that it is lower semi-continuous in $L^2(\R^n)$, since it is a semi-norm on this space. The regularizer and the $L^2$-norm are proper as well. We have that the finite sublevelsets of $R$ are in $W^{1,2}(\R^n)$ per definition. By the Rellich-Kondrachov compact embedding theorem we have that the sublevelsets of $R$ are compact in $L^2(\R^n)$. Thus the sublevelsets of $E$ are compact. We follow now with [\[thm:existence\]](#thm:existence){reference-type="ref+Label" reference="thm:existence"} that $E$ has a minimizer in $L^2(\R^n)$. We can even further state more regularization on the minimzer, since it must be in $W^{1,2}(\R^n)$ as well. ◻ ```` Another approach to receive compactness is by changing the topology to the weak topology. When changing to the weak topology we gain more compact sets, but loose regularity of the functional. ````{prf:example} Tikhonov Regularization There exist a minimizer of ```{math} E(u) = \frac{1}{2}\|u-f\|_2^2 + \frac{\alpha}{2} \| u\|_2^2 ``` in $L^2(\R^n)$. ```` ````{prf:proof} We equip $L^2(\R^n)$ with the weak topology. We now have a topological vector space on the set $X$. In this space we have the property that a bounded set by the $L^2$-norm is compact. We note that when equipping a Banach space with its weak topology, the resulting topological vector space is not necessarily any more a Banach space and the norm-function is not any more weakly continuous, but proper and weakly lower semi-continous. We now apply [\[thm:existence\]](#thm:existence){reference-type="ref+Label" reference="thm:existence"} and have that a minimizer in $L^2(\R^n)$ exists. ◻ ```` Uniqueness of the minimizer can only be guaranteed if $E$ is strictly convex. ````{prf:lemma} Uniqueness :label: prop:uniqueness Let $\bar{u}$ be a minimizer of $E$ and let $E$ be strictly convex. Then $\bar{u}$ is the only global and local minimizer of $E$. ```` We have that (semi-)norms are convex, but not necessarily strictly convex.