# Worksheet 8-2: Orthogonal Projection

Download: [CMSE382-WS8_2.pdf](CMSE382-WS8_2.pdf)

```{warning}
This is an AI-generated transcript of the worksheet and may contain errors or inaccuracies. Please refer to the original course materials for authoritative content.
```

---

## Worksheet 8-2: Q1

For the following sets and each point drawn ($v$, $w$, $x$, $y$, and $z$), mark the point that minimizes the projection operator ($P_C(v)$, $P_C(w)$, $P_C(x)$, $P_C(y)$, and $P_C(z)$).

![Q1 figure](../../../figures/ProjectionExamples.png)

---

## Worksheet 8-2: Q2

![Q2 figure](../../../figures/WS_nonnegative_part_problem.png)

For each of the shown vectors $\mathbf{x}$, answer the following

1. Find an expression for the non-negative real part $[\mathbf{x}]_+$ in terms of $\mathbf{x} = (x_1,x_2)$.
2. Sketch $[\mathbf{x}]_+$ for each vector.

---

## Worksheet 8-2: Q3

1. Consider the set $C = \{(y_1,y_2,y_3) \in  \mathbb{R}^3 \mid y_1\geq 0, y_2\geq 0, y_3 = 0 \}$. Write the orthogonal projection operator $P_C(\mathbf{x})$ for any $\mathbf{x} = (x_1,x_2,x_3) \in  \mathbb{R}^3$. What point in $C$ minimizes $P_c(\mathbf{x})$? Write it in terms of $[\cdot]_+$ if possible.
2. For each of the following points, write the expression for the projected point in terms of just $x_1, x_2, x_3$. Sketch the point.

![Q3 figure](../../../figures/WS_orthogonal_projection_problem.png)

---

## Worksheet 8-2: Q4

A box is a subset of $\mathbb{R}^n$ of the form

$$
\begin{align*}
B &= [\ell_1,u_1]\times [\ell_2, u_2]\times \ldots \times [\ell_n,u_n]\\
& =\{\mathbf{x} \in \mathbb{R}^n: \ell_i \leq x_i \leq u_i\},
\end{align*}
$$

where $\ell_i \leq u_i$ for all $i=1,2,\ldots,n$.

![Q4 figure](../../../figures/box_examples.png)

We will assume that some of the $u_i$s can be $\infty$ and some of the $\ell_i$s can be $-\infty$; but in these cases we will assume that $-\infty$ or $\infty$ are not contained in the intervals. The figure above shows some two examples in boxes in $\mathbb{R}^2$ and $\mathbb{R}^3$. The orthognal projection on the box is the minimizer of the convex optimization problem

$$
\begin{aligned}
& \text{min}  &  & \|\mathbf{y}-\mathbf{x}\|^2 \\
& \text{s.t.} & & \mathbf{y} \in B
\end{aligned}
$$

for a given box $B$.

1. Write the minimization problem above in terms of only $x_i$'s and $y_i$'s.
2. Is the resulting functional equation separable? Justify your answer.
3. Write down and solve the optimization problem for each $y_i$. Use this to determine $\mathbf{y} = P_C(\mathbf{x})$.

---

## Worksheet 8-2: Q5

Consider the normal ball in $ \mathbb{R}^2$, $C = B[0,r] = \{\mathbf{y} =(y_1,y_2) \mid \|\mathbf{y}\|_2 \leq r\}$. We will find the projection $P_C(\mathbf{x})$ for some point $\mathbf{x} \in  \mathbb{R}^2$, which is the $\mathbf{y}$ that minimizes

$$
\begin{aligned}
& \text{min}  &  & \|\mathbf{y}-\mathbf{x}\|^2 \\
& \text{s.t.} & & \|\mathbf{y}\|^2 \leq r^2.
\end{aligned}
$$

1. Assume $\mathbf{x} \in B[0,r]$. What is $P_C(\mathbf{x})$ and why?
2. What is $\nabla f(\mathbf{z})$ for $f(\mathbf{z}) = \|\mathbf{z}\|^2$?
3. Now we can deal with the case where $\mathbf{x} \not\in B[0,r]$, equivalently $\|\mathbf{x}\|^2 \geq r^2$. We know (from the first order optimality condition for local optima, Thm 2.6 in the book) that if $\mathbf{x}^* \in \text{int}(C)$ is a local optimum and all partial derivatives exist, then $\nabla f(\mathbf{x}^*) = 0$. If $\mathbf{x} \not\in B[0,r]$ and somehow $\mathbf{x}^* = P_C(\mathbf{x}) \in \text{int}(B[0,r])$, use your calculated gradient above to conclude that the result is impossible so $P_C(\mathbf{x})$ must be on the boundary of $B[0,r]$.
4. By the previous, we know that if $\|\mathbf{x}\|\geq r$, the solution must be on the boundary, so we can replace the problem with

$$
\begin{aligned}
& \text{min}  &  & \|\mathbf{y}-\mathbf{x}\|^2 \\
& \text{s.t.} & & \|\mathbf{y}\|^2 = r^2.
\end{aligned}
$$

Then I can expand $\|\mathbf{y}-\mathbf{x}\|^2 = \|\mathbf{y}\|^2 -2 \mathbf{y}^\top \mathbf{x} + \|\mathbf{x}\|^2$ and replace this problem with

$$
\begin{aligned}
& \text{min}  &  & \|\mathbf{y}\|^2 -2 \mathbf{y}^\top \mathbf{x} + \|\mathbf{x}\|^2 \\
& \text{s.t.} & & \|\mathbf{y}\|^2= r^2.
\end{aligned}
$$

Why, then, can I replace this problem with the following problem?

$$
\begin{aligned}
& \text{min}  &  & -2 \mathbf{y}^\top \mathbf{x} \\
& \text{s.t.} & & \|\mathbf{y}\|^2 =  r^2.
\end{aligned}
$$

1. The Cauchy-Schwartz inequality ($|\mathbf{u}^\top \mathbf{v}| \leq \|\mathbf{u}\| \cdot \|\mathbf{v}\|$) gives us a bound

$$
\mathbf{y}^\top \mathbf{x} \leq |\mathbf{y}^\top \mathbf{x}| \leq \|\mathbf{y}\| \cdot \|\mathbf{x}\|
$$

Use the above to justify each inequality below.

$$
-2 \mathbf{y}^\top \mathbf{x} \geq -2\|\mathbf{y}\|  \|\mathbf{x}\| = -2r\|\mathbf{x}\|.
$$

1. Check that equality in Eqn. (bound) (meaning $-2 \mathbf{y}^\top \mathbf{x} = -2r\|\mathbf{x}\|$) occurs when $\mathbf{y}^* = r\frac{\mathbf{x}}{\|\mathbf{x}\|}$.
2. Check that $\mathbf{y}^* = r\frac{\mathbf{x}}{\|\mathbf{x}\|}$ is in $B[0,r]$.
3. Putting the above together, fill in the orthogonal projection for the ball:

$$
P_{B[0,r]} =
\begin{cases}
\boxed{\phantom{\prod}  \phantom{\prod}} & \|\mathbf{x}\|\leq r \\
\\
\boxed{\phantom{\prod}  \phantom{\prod}}  & \|\mathbf{x}\| > r
\end{cases}
$$