# Lecture 7-2: Convex Functions: Part 2

Download the original slides: [CMSE382-Lec7_2.pdf](CMSE382-Lec7_2.pdf)

```{warning}
This is an AI-generated transcript of the lecture slides and may contain errors or inaccuracies. Please refer to the original course materials for authoritative content.
```

---

## Review (Mostly)

### Definition of convex functions

**Definition:** A function $f: C \to \mathbb{R}$ defined on a convex set $C \subset \mathbb{R}^n$ is

- **convex** if and only if

$$
\begin{aligned}
f(\lambda \mathbf{x} + (1 - \lambda)\mathbf{y}) \le \lambda f(\mathbf{x}) + (1 - \lambda)f(\mathbf{y})
\end{aligned}
$$

for all $\mathbf{x}, \mathbf{y} \in C$, $\lambda \in [0, 1]$.

- **strictly convex** if and only if

$$
\begin{aligned}
f(\lambda \mathbf{x} + (1 - \lambda)\mathbf{y}) < \lambda f(\mathbf{x}) + (1 - \lambda)f(\mathbf{y})
\end{aligned}
$$

for all $\mathbf{x} \ne \mathbf{y} \in C$, $\lambda \in (0, 1)$.

![Plot comparing convex and strictly convex function shapes](../../../figures/convex_strictly_convexfunctions.png)

---

### Definition of concave functions

**Definition:** A function $f: C \to \mathbb{R}$ defined on a convex set $C \subset \mathbb{R}^n$ is **concave** if and only if

$$
\begin{aligned}
f(\lambda \mathbf{x} + (1 - \lambda)\mathbf{y}) \ge \lambda f(\mathbf{x}) + (1 - \lambda)f(\mathbf{y})
\end{aligned}
$$

for all $\mathbf{x}, \mathbf{y} \in C$, $\lambda \in [0, 1]$.

Equivalently, $f$ is concave if and only if $-f$ is convex.

![Mirrored plot illustrating concavity relative to convexity](../../../figures/convex_strictly_convexfunctions.png)

---

### Second order characterization

**Theorem:** Let $f$ be a twice continuously differentiable function over an open convex set $C \subseteq \mathbb{R}^n$.

- $f$ is convex over $C$ **if and only if** $\nabla^2 f(\mathbf{x}) \succeq 0$ for any $\mathbf{x} \in C$.
- **If** $\nabla^2 f(\mathbf{x}) \succ 0$ for any $\mathbf{x} \in C$, then $f$ is strictly convex over $C$.

Note: For 1D functions, this is just the second derivtive test (i.e., $f''(x) \geq 0$ for all $x$ in the domain implies $f$ is convex).

![Examples used for second-order convexity and strict convexity characterization](../../../figures/strict_convexity_examples.png)

---

### Some functions we know are convex

- Affine functions: $f(\mathbf{x}) = \mathbf{a}^T\mathbf{x} + b$.
- Quadratic functions: $f(\mathbf{x}) = \mathbf{x}^\top A\mathbf{x} + 2\mathbf{b}^T\mathbf{x} + c$ where $A \succeq 0$ is positive semidefinite.
- Norms: $f(\mathbf{x}) = \|\mathbf{x}\|$ for any norm on $\mathbb{R}^n$.
- Exponential function: $f(x) = \exp{x}$.

---

### Operations Preserving Convexity I

**Theorem (Nonnegative scaling and sums):** Let $C \subseteq \mathbb{R}^n$ be convex.

- If $f$ is convex on $C$ and $\alpha \ge 0$, then $\alpha f$ is convex on $C$.
- If $f_1,\ldots,f_p$ are convex on $C$, then the sum $f_1 + \cdots + f_p$ is convex on $C$.

![Diagram showing convexity preserved under nonnegative scaling](../../../figures/covexity_under_nonnegative_scaling.png)

![Diagram showing convexity preserved under summing convex functions](../../../figures/covexity_under_addition.png)

---

### Affine change of variables

**Theorem (Affine change of variables):** Let $f: C \to \mathbb{R}$ be convex on a convex set $C \subseteq \mathbb{R}^n$. Let $A \in \mathbb{R}^{n\times m}$ and $\mathbf{b} \in \mathbb{R}^n$. Define

$$
g(\mathbf{y}) = f(A\mathbf{y} + \mathbf{b}),
$$

on the convex set $D = \{\mathbf{y} \in \mathbb{R}^m : A\mathbf{y} + \mathbf{b} \in C\}$. Then $g$ is convex on $D$.

![Diagram illustrating convexity under affine change of variables](../../../figures/convexity_under_affine_transformation.png)

---

### Composition with monotone convex functions

**Theorem (Convex outer, nondecreasing):** Let $f: C \to \mathbb{R}$ be convex on a convex set $C \subseteq \mathbb{R}^n$. Let $g: I \to \mathbb{R}$ be a one-dimensional nondecreasing convex function defined on an interval $I$, and assume $f(C) \subseteq I$. Then the composition $x \mapsto g(f(x))$ is convex on $C$.

---

## Sublevel sets of convex functions

### Convexity of sublevel sets of convex functions

**Definition:** Let $f: S \to \mathbb{R}$ be a function defined over a set $S \subseteq \mathbb{R}^n$. Then **the sublevel set** of $f$ with level $\alpha$ is given by

$$
\text{SubLev}(f,\alpha) = \text{Lev}(f, \alpha) = \{x \in S: f(x) \leq \alpha\}.
$$

**Theorem:** Let $f: C \to \mathbb{R}$ be a convex function defined over a convex set $C \subseteq \mathbb{R}^n$. Then for any $\alpha \in \mathbb{R}$ the sublevel set $\text{SubLev}(f,\alpha)$ is convex.

![Cartoon of convex sublevel sets for a convex function](../../../figures/sublevel_sets_cartoon.png)

---

### Quasi-convex functions

**Definition:** A function $f: C \to \mathbb{R}$ defined over the convex set $C \subseteq \mathbb{R}^n$ is called *quasi-convex* if for any $\alpha \in \mathbb{R}$ the set $\text{Lev}(f,\alpha)$ is convex.

- A quasi-convex function may be non-convex.

![Example plot of a quasi-convex function that is not convex](../../../figures/quasi_convex_non_convex.png)

---

### Quasi-concave functions

**Definition:** Define the *superlevel set* of $f$ with level $\alpha$ as

$$
\text{SupLev}(f,\alpha) = \{x \in S: f(x) \geq \alpha\}.
$$

A function $f: C \to \mathbb{R}$ defined over the convex set $C \subseteq \mathbb{R}^n$ is called *quasi-concave* if for any $\alpha \in \mathbb{R}$ the set $\text{SupLev}(f,\alpha)$ is convex.

- A quasi-concave function may be non-concave.
- A function is quasi-concave if and only if its negative is quasi-convex.

![Mirrored plot illustrating quasi-concavity via superlevel sets](../../../figures/quasi_convex_non_convex.png)

---

### Quasi-Linear

**Definition:** A function is *quasi-linear* if it is both quasi-convex and quasi-concave.

![Graph of x cubed illustrating a quasi-linear function example](../../../figures/x_cubed.png)

Example: $f(x) = x^3$ is quasi-linear but not linear, convex, or concave.