Lecture 7-2: Convex Functions: Part 2

Lecture 7-2: Convex Functions: Part 2#

Download the original slides: CMSE382-Lec7_2.pdf

Warning

This is an AI-generated transcript of the lecture slides and may contain errors or inaccuracies. Please refer to the original course materials for authoritative content.

Review (Mostly)#

Definition of convex functions#

Definition: A function \(f: C \to \mathbb{R}\) defined on a convex set \(C \subset \mathbb{R}^n\) is

  • convex if and only if

\[ \begin{aligned} f(\lambda \mathbf{x} + (1 - \lambda)\mathbf{y}) \le \lambda f(\mathbf{x}) + (1 - \lambda)f(\mathbf{y}) \end{aligned} \]

for all \(\mathbf{x}, \mathbf{y} \in C\), \(\lambda \in [0, 1]\).

  • strictly convex if and only if

\[ \begin{aligned} f(\lambda \mathbf{x} + (1 - \lambda)\mathbf{y}) < \lambda f(\mathbf{x}) + (1 - \lambda)f(\mathbf{y}) \end{aligned} \]

for all \(\mathbf{x} \ne \mathbf{y} \in C\), \(\lambda \in (0, 1)\).

Plot comparing convex and strictly convex function shapes

Definition of concave functions#

Definition: A function \(f: C \to \mathbb{R}\) defined on a convex set \(C \subset \mathbb{R}^n\) is concave if and only if

\[ \begin{aligned} f(\lambda \mathbf{x} + (1 - \lambda)\mathbf{y}) \ge \lambda f(\mathbf{x}) + (1 - \lambda)f(\mathbf{y}) \end{aligned} \]

for all \(\mathbf{x}, \mathbf{y} \in C\), \(\lambda \in [0, 1]\).

Equivalently, \(f\) is concave if and only if \(-f\) is convex.

Mirrored plot illustrating concavity relative to convexity

Second order characterization#

Theorem: Let \(f\) be a twice continuously differentiable function over an open convex set \(C \subseteq \mathbb{R}^n\).

  • \(f\) is convex over \(C\) if and only if \(\nabla^2 f(\mathbf{x}) \succeq 0\) for any \(\mathbf{x} \in C\).

  • If \(\nabla^2 f(\mathbf{x}) \succ 0\) for any \(\mathbf{x} \in C\), then \(f\) is strictly convex over \(C\).

Note: For 1D functions, this is just the second derivtive test (i.e., \(f''(x) \geq 0\) for all \(x\) in the domain implies \(f\) is convex).

Examples used for second-order convexity and strict convexity characterization

Some functions we know are convex#

  • Affine functions: \(f(\mathbf{x}) = \mathbf{a}^T\mathbf{x} + b\).

  • Quadratic functions: \(f(\mathbf{x}) = \mathbf{x}^\top A\mathbf{x} + 2\mathbf{b}^T\mathbf{x} + c\) where \(A \succeq 0\) is positive semidefinite.

  • Norms: \(f(\mathbf{x}) = \|\mathbf{x}\|\) for any norm on \(\mathbb{R}^n\).

  • Exponential function: \(f(x) = \exp{x}\).

Operations Preserving Convexity I#

Theorem (Nonnegative scaling and sums): Let \(C \subseteq \mathbb{R}^n\) be convex.

  • If \(f\) is convex on \(C\) and \(\alpha \ge 0\), then \(\alpha f\) is convex on \(C\).

  • If \(f_1,\ldots,f_p\) are convex on \(C\), then the sum \(f_1 + \cdots + f_p\) is convex on \(C\).

Diagram showing convexity preserved under nonnegative scaling

Diagram showing convexity preserved under summing convex functions

Affine change of variables#

Theorem (Affine change of variables): Let \(f: C \to \mathbb{R}\) be convex on a convex set \(C \subseteq \mathbb{R}^n\). Let \(A \in \mathbb{R}^{n\times m}\) and \(\mathbf{b} \in \mathbb{R}^n\). Define

\[ g(\mathbf{y}) = f(A\mathbf{y} + \mathbf{b}), \]

on the convex set \(D = \{\mathbf{y} \in \mathbb{R}^m : A\mathbf{y} + \mathbf{b} \in C\}\). Then \(g\) is convex on \(D\).

Diagram illustrating convexity under affine change of variables

Composition with monotone convex functions#

Theorem (Convex outer, nondecreasing): Let \(f: C \to \mathbb{R}\) be convex on a convex set \(C \subseteq \mathbb{R}^n\). Let \(g: I \to \mathbb{R}\) be a one-dimensional nondecreasing convex function defined on an interval \(I\), and assume \(f(C) \subseteq I\). Then the composition \(x \mapsto g(f(x))\) is convex on \(C\).

Sublevel sets of convex functions#

Convexity of sublevel sets of convex functions#

Definition: Let \(f: S \to \mathbb{R}\) be a function defined over a set \(S \subseteq \mathbb{R}^n\). Then the sublevel set of \(f\) with level \(\alpha\) is given by

\[ \text{SubLev}(f,\alpha) = \text{Lev}(f, \alpha) = \{x \in S: f(x) \leq \alpha\}. \]

Theorem: Let \(f: C \to \mathbb{R}\) be a convex function defined over a convex set \(C \subseteq \mathbb{R}^n\). Then for any \(\alpha \in \mathbb{R}\) the sublevel set \(\text{SubLev}(f,\alpha)\) is convex.

Cartoon of convex sublevel sets for a convex function

Quasi-convex functions#

Definition: A function \(f: C \to \mathbb{R}\) defined over the convex set \(C \subseteq \mathbb{R}^n\) is called quasi-convex if for any \(\alpha \in \mathbb{R}\) the set \(\text{Lev}(f,\alpha)\) is convex.

  • A quasi-convex function may be non-convex.

Example plot of a quasi-convex function that is not convex

Quasi-concave functions#

Definition: Define the superlevel set of \(f\) with level \(\alpha\) as

\[ \text{SupLev}(f,\alpha) = \{x \in S: f(x) \geq \alpha\}. \]

A function \(f: C \to \mathbb{R}\) defined over the convex set \(C \subseteq \mathbb{R}^n\) is called quasi-concave if for any \(\alpha \in \mathbb{R}\) the set \(\text{SupLev}(f,\alpha)\) is convex.

  • A quasi-concave function may be non-concave.

  • A function is quasi-concave if and only if its negative is quasi-convex.

Mirrored plot illustrating quasi-concavity via superlevel sets

Quasi-Linear#

Definition: A function is quasi-linear if it is both quasi-convex and quasi-concave.

Graph of x cubed illustrating a quasi-linear function example

Example: \(f(x) = x^3\) is quasi-linear but not linear, convex, or concave.

Contents