# Lecture 8-2: Convex Optimization: Part 2

Download the original slides: [CMSE382-Lec8_2.pdf](CMSE382-Lec8_2.pdf)

```{warning}
This is an AI-generated transcript of the lecture slides and may contain errors or inaccuracies. Please refer to the original course materials for authoritative content.
```

---

## This Lecture

**Topics:**

- The orthogonal projection operator
- Projection on the non-negative orthant
- Projection on $B[0,r]$

**Announcements:**

- Homework 4 due Friday

---

## Orthogonal Projection Operator

### Orthogonal projection

**Definition (Orthogonal projection operator)**

Given a nonempty closed convex set $C$, the orthogonal projection operator $P_C:\mathbb{R}^n \to C$ is defined by

$$
P_C(\mathbf{x}) = \text{argmin}{\|\mathbf{y} - \mathbf{x}\|^2: \mathbf{y} \in C}.
$$

![](../../../figures/orthogonal_projection_illustration.png)

- Returns the vector in $C$ that is closest to input vector $\mathbf{x}$.
- Is a convex optimization problem:

$$
\begin{aligned}
& \text{min}  &  & \|\mathbf{y}-\mathbf{x}\|^2 \\
& \text{s.t.} & & \mathbf{y} \in C.
\end{aligned}
$$

---

### Orthogonal projection: First projection theorem

**Theorem (first projection theorem)**

Let $C$ be a nonempty closed convex set.
Then the problem $P_C(\mathbf{x})=\text{argmin}{\|\mathbf{y} - \mathbf{x}\|^2: \mathbf{y} \in C}$ has a unique optimal solution.

- Computing $P_C(\mathbf{x})$ can be difficult. Examples where it is easy to compute:
- Projection on non-negative orthant.
- Projection onto balls.

---

### Non-negative part of a vector

**Definition (Non-negative part of a vector)**

- For $\alpha \in \mathbb{R}$, the non-negative part of $\alpha$ is $[\alpha]_+ = \begin{cases}\alpha, & \alpha \geq 0\\ 0, & \alpha<0. \end{cases}$
- For a vector $\mathbf{v} \in \mathbb{R}^n$, the non-negative part of $\mathbf{v}$ is $[\mathbf{v}]_+ = \begin{bmatrix} [v_1]_+ \\ [v_2]_+ \\ \vdots \\ [v_n]_+ \end{bmatrix}$

![](../../../figures/nonnegative_part.png)

![](../../../figures/nonnegative_part.png)

---

### Orthogonal projection: Projection on the non-negative orthant

Let $C=\mathbb{R}^n_{+}$. To find the orthogonal projection of $\mathbf{x} \in \mathbb{R}^n$ onto $\mathbb{R}^n_{+}$:

$\underline{P_C(\mathbf{x})}$

$$
\begin{aligned}
& \text{min}  &  & \|\mathbf{y}-\mathbf{x}\|^2 \\
& \text{s.t.} & & \mathbf{y} \in C.
\end{aligned}
$$

$\underline{\text{Equivalently,}}$

$$
\begin{aligned}
& \text{min}  &  & \sum\limits_{i=1}^n{(y_i-x_i)^2} \\
& \text{s.t.} & & y_1,y_2,\ldots, y_n \geq 0.
\end{aligned}
$$

$\underline{\text{Separable}}$

$$
\begin{aligned}
& \text{min}  &  & (y_i-x_i)^2 \\
& \text{s.t.} & & y_i \geq 0.
\end{aligned}
$$

**Definition (Separable convex optimization problems)**

A convex optimization problem is called *separable* if the objective function and the constraints can be decomposed into components that each depend on one control/decision variable:

- Objective function: $f(\mathbf{x})=\sum{f_i(\mathbf{x}_i)}$.
- Constraint(s): $g(\mathbf{x})=\sum{g_i(\mathbf{x}_i)}$, or $\{g_i(\mathbf{x}_i)\}_{i}$

---

### Orthogonal projection: Projection on the non-negative orthant

Let $C=\mathbb{R}^n_{+}$. The orthogonal projection of $\mathbf{x} \in \mathbb{R}^n$ onto $\mathbb{R}^n_{+} = \{\mathbf{y} \in \R^n \mid y_i \geq 0 \; \forall i\}$ is

$$
\begin{aligned}
& \text{min}  &  & (y_i-x_i)^2 \\
& \text{s.t.} & & y_i \geq 0.
\end{aligned}
$$

**Orthogonal projection onto $\mathbb{R}^n_{+}$**

The orthogonal projection operator onto $\mathbb{R}^n_{+}$ is

$$
P_{\mathbb{R}^n_{+}} = [\mathbf{x}]_+.
$$

![](../../../figures/projection_on_nonnegative_orthant.png)

---

### Orthogonal projection: Projection onto balls

Let $C=B[\mathbf{0},r]=\{\mathbf{y}:\|\mathbf{y}\leq r\}$. The projection of $\mathbf{x}$ onto $C$ is

$$
\min\limits_{\mathbf{y}}\{\|\mathbf{y}-\mathbf{x}\|^2:\|\mathbf{y}\|^2 \leq r^2\}
$$

![](../../../figures/projection_on_ball.png)