SQUARE45

A convex optimization problem is defined by two ingredients: - The objective function, which is a real-valued convex function of n variables,

f:{\mathcal {D}}\subseteq \mathbb {R} ^{n}\to \mathbb {R}

; - The feasible set, which is a convex subset

C\subseteq \mathbb {R} ^{n}

. The goal of the problem is to find some

\mathbf {x^{\ast }} \in C

attaining

\inf\{f(\mathbf {x} ):\mathbf {x} \in C\}

. In general, there are three options regarding the existence of a solution: - If such a point x* exists, it is referred to as an optimal point or solution; the set of all optimal points is called the optimal set; and the problem is called solvable. - If

f

is unbounded below over

C

, or the infimum is not attained, then the optimization problem is said to be unbounded. - Otherwise, if

C

is the empty set, then the problem is said to be infeasible. A convex optimization problem is in standard form if it is written as

{\begin{aligned}&{\underset {\mathbf {x} }{\operatorname {minimize} }}&&f(\mathbf {x} )\\&\operatorname {subject\ to} &&g_{i}(\mathbf {x} )\leq 0,\quad i=1,\dots ,m\\&&&h_{i}(\mathbf {x} )=0,\quad i=1,\dots ,p,\end{aligned}}

where: -

\mathbf {x} \in \mathbb {R} ^{n}

is the vector of optimization variables; - The objective function

f:{\mathcal {D}}\subseteq \mathbb {R} ^{n}\to \mathbb {R}

is a convex function; - The inequality constraint functions

g_{i}:\mathbb {R} ^{n}\to \mathbb {R}

i=1,\ldots ,m

, are convex functions; - The equality constraint functions

h_{i}:\mathbb {R} ^{n}\to \mathbb {R}

i=1,\ldots ,p

, are affine transformations, that is, of the form:

h_{i}(\mathbf {x} )=\mathbf {a_{i}} \cdot \mathbf {x} -b_{i}

, where

\mathbf {a_{i}}

is a vector and

b_{i}

is a scalar. The feasible set

C

of the optimization problem consists of all points

\mathbf {x} \in {\mathcal {D}}

satisfying the inequality and the equality constraints. This set is convex because

{\mathcal {D}}

is convex, the sublevel sets of convex functions are convex, affine sets are convex, and the intersection of convex sets is convex. Many optimization problems can be equivalently formulated in this standard form. For example, the problem of maximizing a concave function

f

can be re-formulated equivalently as the problem of minimizing the convex function

-f

. The problem of maximizing a concave function over a convex set is commonly called a convex optimization problem. In the standard form it is possible to assume, without loss of generality, that the objective function f is a linear function. This is because any program with a general objective can be transformed into a program with a linear objective by adding a single variable t and a single constraint, as follows:

{\begin{aligned}&{\underset {\mathbf {x} ,t}{\operatorname {minimize} }}&&t\\&\operatorname {subject\ to} &&f(\mathbf {x} )-t\leq 0\\&&&g_{i}(\mathbf {x} )\leq 0,\quad i=1,\dots ,m\\&&&h_{i}(\mathbf {x} )=0,\quad i=1,\dots ,p,\end{aligned}}

Every convex program can be presented in a conic form, which means minimizing a linear objective over the intersection of an affine plane and a convex cone:

{\begin{aligned}&{\underset {\mathbf {x} }{\operatorname {minimize} }}&&c^{T}x\\&\operatorname {subject\ to} &&x\in (b+L)\cap K\end{aligned}}

where K is a closed pointed convex cone, L is a linear subspace of R^{n}, and b is a vector in R^{n}. A linear program in standard form is the special case in which K is the nonnegative orthant of R^{n}. It is possible to convert a convex program in standard form, to a convex program with no equality constraints. Denote the equality constraints h_{i}(x)=0 as Ax=b, where A has n columns. If Ax=b is infeasible, then of course the original problem is infeasible. Otherwise, it has some solution x_{0} , and the set of all solutions can be presented as: Fz+x_{0}, where z is in R^{k}, k=n-rank(A), and F is an n-by-k matrix. Substituting x = Fz+x_{0} in the original problem gives:

{\begin{aligned}&{\underset {\mathbf {x} }{\operatorname {minimize} }}&&f(\mathbf {F\mathbf {z} +\mathbf {x} _{0}} )\\&\operatorname {subject\ to} &&g_{i}(\mathbf {F\mathbf {z} +\mathbf {x} _{0}} )\leq 0,\quad i=1,\dots ,m\\\end{aligned}}

where the variables are z. Note that there are rank(A) fewer variables. This means that, in principle, one can restrict attention to convex optimization problems without equality constraints. In practice, however, it is often preferred to retain the equality constraints, since they might make some algorithms more efficient, and also make the problem easier to understand and analyze. - ^Hiriart-Urruty, Jean-Baptiste; Lemaréchal, Claude (1996). Convex analysis and minimization algorithms: Fundamentals. Springer. p. 291. ISBN9783540568506. - ^Ben-Tal, Aharon; Nemirovskiĭ, Arkadiĭ Semenovich (2001). Lectures on modern convex optimization: analysis, algorithms, and engineering applications. pp. 335–336. ISBN9780898714913. - ^ ^{a}^{b}^{c}^{d}Boyd, Stephen; Vandenberghe, Lieven (2004). Convex Optimization(PDF). Cambridge University Press. ISBN978-0-521-83378-3. Retrieved 12 Apr 2021. - ^"Optimization Problem Types - Convex Optimization". 9 January 2011. - ^ ^{a}^{b}Arkadi Nemirovsky (2004). Interior point polynomial-time methods in convex programming.

Convex Optimization

Sequence of Expressions

Definition

Properties