# Worksheet 8-1: Convex Optimization (with Solutions)

Download: [CMSE382-WS8_1.pdf](CMSE382-WS8_1.pdf), [CMSE382-WS8_1-Soln.pdf](CMSE382-WS8_1-Soln.pdf)

```{warning}
This is an AI-generated transcript of the worksheet and may contain errors or inaccuracies. Please refer to the original course materials for authoritative content.
```

---

## Worksheet 8-1: Q1

For each problem:

- Sketch the feasible set, is it convex?
- Is the objective function convex? You may use the atoms below.

| Atom | Property |
|---|---|
| $t^m$ | convex for $m=2,4,6$ |
| $\|\mathbf{x}\|$ | convex |
| $e^t$ | convex, nondecreasing |

- Based on this, state whether the problem is a convex optimization problem or not.

(a)

$$
\begin{aligned}
& \text{min} && x^2-y\\
& \text{s.t.} && x^2+y^2=3
\end{aligned}
$$

```{dropdown} Solution
The feasible set is the circle of radius $\sqrt{3}$, which is not a convex set.
Therefore, this is not a convex optimization problem.

![Q1(a) sketch](../../../figures/WS8-1-Prob1a.png)

Note that the objective function is convex, but since the constraint is not convex this is not a convex optimization problem.
```

(b)

$$
\begin{aligned}
& \text{min} && x+y^4\\
& \text{s.t.} && x+5y\le 10
\end{aligned}
$$

```{dropdown} Solution
The feasible set is the half-space defined by $x + 5y \le 10$, which is a convex set.
The objective function is the sum of a linear function and a convex function, which is convex. Therefore, this is a convex optimization problem.

![Q1(b) sketch](../../../figures/WS8-1-Prob1b.png)
```

(c)

$$
\begin{aligned}
& \text{min} && -e^{x^2}+y^2\\
& \text{s.t.} && \|\mathbf{x}\|\le 5
\end{aligned}
$$

```{dropdown} Solution
The feasible set is the ball of radius $5$, which is a convex set.
The objective function is the sum of a concave function and a convex function, which is not likely to be convex, however that's not enough to conclude that it's not a convex function.

Instead, we can check the Hessian of the objective function, which is given by
$$
\begin{bmatrix}
-2e^{x^2}(1+2x^2) & 0 \\
0 & 2
\end{bmatrix}.
$$

The eigenvalues of this Hessian are $-2e^{x^2}(1+2x^2)$ and $2$.
Since the first one is negative and the second one is positive, the Hessian is not positive definite. Therefore, the objective function is not convex.

Therefore, this is not a convex optimization problem.

![Q1(c) sketch](../../../figures/WS8-1-Prob1c.png)
```

(d)

$$
\begin{aligned}
& \text{min} && (1+x^2+x^{10})^2\\
& \text{s.t.} && 5x+1\le 4
\end{aligned}
$$

```{dropdown} Solution
The feasible set is the half-space defined by $5x + 1 \leq 4$, which is a convex set.

For the objective function, we need to build it up slowly.
- $x^2$ is convex.
- $x^{10}$ is convex.
- $1+x^2+x^{10}$ is the sum of a constant and two convex functions, which is convex.
- You can't just plug in a convex function (here $1+x^2+x^{10}$) into another convex function (here $(t)^2$) and expect it to be convex. However, we have a theorem that says that if we plug in a convex function into a nondecreasing convex function, then the resulting function is convex.
- $t^2$ is a nondecreasing function as long as we restrict our inputs to $t \ge 0$.
- For any input $x \in \mathbb{R}$, we have $1+x^2+x^{10} \ge 1 > 0$, so the input to the square function $t^2$ is always non-negative.
- Thus the composition $(1+x^2+x^{10})^2$ is convex.

The objective function is convex, and the constraint is convex, so this is a convex optimization problem.

![Q1(d) sketch](../../../figures/WS8-1-Prob1d.png)
```

(e)

$$
\begin{aligned}
& \text{min} && -x^4+3y\\
& \text{s.t.} && x+3y\le 3
\end{aligned}
$$

```{dropdown} Solution
The feasible set is the half-space defined by $x+3y \le 3$, which is a convex set.

The objective function is the sum of a concave function and a linear function.
The Hessian of the objective function is given by
$$
\begin{bmatrix}-12x^2 & 0 \\
0 & 0
\end{bmatrix}.
$$

The eigenvalues of this Hessian are $-12x^2$ and $0$.
Since the first one is negative and the second one is zero, the Hessian is not positive definite. Therefore, the objective function is not convex.
Since the objective function is not convex, this is not a convex optimization problem.

![Q1(e) sketch](../../../figures/WS8-1-Prob1e.png)
```

---

## Worksheet 8-1: Q2

For a position in a company, we need to schedule job interviews for 3 candidates, Alice, Bob, and Charlie, who are coming in that order.

The available interview time windows are:

| Candidate | Times | Interval as hours since 9:00am |
|---|---|---|
| Alice | 9:00 - 10:30 | $[0,1.5]$ |
| Bob | 10:00 - 11:30 | $[1,2.5]$ |
| Charlie | 9:30 - 12:30 | $[0.5,3.5]$ |

The goal is to schedule starting times $t_A,t_B,t_C$ to maximize the minimal starting-time difference between consecutive interviews.

(a) The objective can be written as

$$
f(t_A,t_B,t_C)=\min\{t_B-t_A,\,t_C-t_B\}.
$$

Write the full optimization problem.

```{dropdown} Solution
We want to maximize $f(t_A, t_B, t_C)$ subject to the constraints on the availability of each candidate, which we can write as the interavls in the table above. So the full problem is

$$
\begin{aligned}
\max_{t_A,t_B,t_C}\quad & \min\{t_B-t_A,\,t_C-t_B\}\\
\text{s.t.}\quad & 0\le t_A\le 1.5,\\
& 1\le t_B\le 2.5,\\
& 0.5\le t_C\le 3.5.
\end{aligned}
$$

Note that we want to have the candidates come in order, so we might want to add the constraints $t_A \le t_B \le t_C$. However, we can actually ignore this because an optimal solution will never have $t_B < t_A$ or $t_C < t_B$ since that would only increase the objective function.
```

(b) We can pull a mathematical trick by introducing a new variable $s$ to rewrite the optimization problem as

$$
\begin{aligned}
\max_{t_A,t_B,t_C,s}\quad & s\\
\text{s.t.}\quad & \min\{t_B-t_A,\,t_C-t_B\}\ge s,\\
& \text{(constraints from part (a))}
\end{aligned}
$$

Then break up the min function and fill in the blanks:

$$
\begin{aligned}
\max_{t_A,t_B,t_C,s}\quad & s\\
\text{s.t.}\quad & t_B-t_A\;\boxed{\phantom{\prod}}\;s,\\
& t_C-t_B\;\boxed{\phantom{\prod}}\;s,\\
& t_A\ge \boxed{\phantom{\prod}}\\
& t_A\le \boxed{\phantom{\prod}}\\
& t_B\ge \boxed{\phantom{\prod}}\\
& t_B\le \boxed{\phantom{\prod}}\\
& t_C\ge \boxed{\phantom{\prod}}\\
& t_C\le \boxed{\phantom{\prod}}
\end{aligned}
$$

```{dropdown} Solution
The filled version is

$$
\begin{aligned}
\max_{t_A,t_B,t_C,s}\quad & s\\
\text{s.t.}\quad & t_B-t_A\ge s,\\
& t_C-t_B\ge s,\\
& t_A\ge 0,\\
& t_A\le 1.5,\\
& t_B\ge 1,\\
& t_B\le 2.5,\\
& t_C\ge 0.5,\\
& t_C\le 3.5.
\end{aligned}
$$

Both inequality symbols in the first two lines are $\ge$.
```

(c) Use the above to write the problem in standard form

$$
\begin{aligned}
\max_{\mathbf{x}}\quad & \mathbf{c}^\top\mathbf{x}\\
\text{s.t.}\quad & A\mathbf{x}\le \mathbf{b}\\
& \mathbf{x}\ge 0
\end{aligned}
$$

by defining $\mathbf{x}$, $\mathbf{c}$, $A$, and $\mathbf{b}$.

```{dropdown} Solution
- The first thing to notice is that we want $Ax\leq B$, but we have some $\ge$ constraints. We can multiply those by $-1$ to get them in the right form. They also have variables on both side, so we can add one side to the other to get all varibles together. In that version, the whole problem becomes

$$
\begin{aligned}
\max_{t_A,t_B,t_C,s}\quad & s\\
\text{s.t.}\quad & -t_B+t_A+s\le 0,\\
& -t_C+t_B+s\le 0,\\
& -t_A\le 0,\\
& t_A\le 1.5,\\
& -t_B\le -1,\\
& t_B\le 2.5,\\
& -t_C\le -0.5,\\
& t_C\le 3.5.
\end{aligned}
$$

- So we can set the following:
- $\mathbf{x} = (t_A, t_B, t_C, s)^\top$,
- $\mathbf{c} = (0, 0, 0, 1)^\top$,
- $A = \begin{bmatrix}1 & -1 & 0 & 1 \\
0 & 1 & -1 & 1 \\
-1 & 0 & 0 & 0 \\
1 & 0 & 0 & 0 \\
0 & -1 & 0 & 0 \\
0 & 1 & 0 & 0 \\
0 & 0 & -1 & 0 \\
0 & 0 & 1 & 0 \end{bmatrix}$, and
- $\mathbf{b} = (0, 0, 0, 1.5, -1, 2.5, -0.5, 3.5)^\top$.

- Make sure you understand how each row in $A$ and each element in $\mathbf{b}$ corresponds to a constraint in the problem.
```

---

## Worksheet 8-1: Q3

Consider the optimization problem

$$
\begin{aligned}
\text{max}\quad & 6x+5y\\
\text{Subject to}\quad & 2x-3y\le 5\\
& x+4y\le 11\\
& 4x+y\le 15\\
& x,y\ge 0
\end{aligned}
$$

(a) Find $\mathbf{A}$, $\mathbf{b}$, and $\mathbf{c}$ such that the problem is in standard form.

```{dropdown} Solution
We have $\mathbf{c} = (6, 5)^\top$,

$$
A = \begin{bmatrix}2 & -3 \\
1 & 4 \\
4 & 1 \end{bmatrix},
$$

and $\mathbf{b} = (5, 11, 15)^\top$.
```

(b) Sketch the feasible set. Where are the extreme points?

```{dropdown} Solution
A sketch is below. The extreme points are the points where the lines intersect, which are (clockwise):
$(0, 0)$, $(0, \frac{11}{4})$, $(\frac{49}{15}, \frac{29}{15})$, $(\frac{25}{7}, \frac{5}{7})$, and $(\frac{5}{2}, 0)$.

![Q3 feasible region](../../../figures/WS8-1-Prob3-Annotated.jpg)
```

(c) Solve the problem by finding the optimal solution.

```{dropdown} Solution
Checking the function value on each of the extreme points, we have

| Extreme point | Objective function value |
|---|---|
| $(0,0)$ | $0$ |
| $(0,\frac{11}{4})$ | $13.75$ |
| $(\frac{49}{15},\frac{29}{15})$ | $29.27$ |
| $(\frac{25}{7},\frac{5}{7})$ | $25$ |
| $(\frac{5}{2},0)$ | $15$ |

The optimal solution is at $(\frac{29}{7}, \frac{1}{7})$ with the optimal value $\frac{179}{7}$.
```

---

## Worksheet 8-1: Q4

We are planning a football watch party for MSU by making pans of cookies and muffins.

- A pan of muffins sells for $\$6$.
- A pan of cookies sells for $\$10$.
- We want at least as many pans of cookies as muffins.
- We have 13 cups of sugar.
- Muffins require 0.5 cup sugar per pan.
- Cookies require 1 cup sugar per pan.
- We want to make at least $\$100$.

Let $X$ be muffin pans and $y$ be cookie pans.

(a) Write the variables and objective function.

```{dropdown} Solution
- **Variables:**
Let $x$ be the number of pans of muffins and $y$ be the number of pans of cookies.

- **Objective function:**
We want to minimize the number of pans we need to bake, which is given by $x + y$. So the objective is $\min \, x+y$.
```

(b) Write the constraints as inequalities/equalities.

```{dropdown} Solution
- We want to make at least $\$100$, which gives us the constraint $6x + 10 y \geq 100$.
- We want to make at least as many pans of cookies as pans of muffins, which gives us the constraint $y \geq x$.
- We have only 13 cups of sugar, and a pan of muffins requires 1/2 a cup of sugar and a pan of cookies requires 1 cup of sugar, which gives us the constraint $0.5 x + y \leq 13$.
- We can't make a negative number of pans, so we have the constraints $x \geq 0$ and $y \geq 0$.
```

(c) Recast inequalities as needed and write standard form.

```{dropdown} Solution
- **Recast inequalities:**
- $-6x - 10 y \leq -100$
- $x - y \leq 0$
- $0.5 x + y \leq 13$
- $x,y \geq 0$

- **Standard form:**

$$
\begin{aligned}
\min_{x,y}\quad & x+y\\
\text{s.t.}\quad &
\begin{bmatrix}
-6 & -10 \\
1 & -1 \\
0.5 & 1
\end{bmatrix}
\begin{bmatrix} x \\ y \end{bmatrix}
\leq
\begin{bmatrix} -100\\ 0 \\ 13 \end{bmatrix}\\
& \begin{bmatrix}x \\ y \end{bmatrix} \geq 0
\end{aligned}
$$
```

(d) Sketch the feasible region.

```{dropdown} Solution
![Q4 feasible region](../../../figures/linear_optimization_muffins_example.png)
```

(e) Find the minimum of the objective function.

```{dropdown} Solution
We need to check all the extreme points of the feasible region:

| Extreme point | Objective function value |
|---|---|
| $(0,10)$ | $10$ |
| $(0,13)$ | $13$ |
| $(6.25,6.25)$ | $12.5$ |
| $(8.667,8.667)$ | $17.334$ |

The minimum occurs at $(x,y)=(0,10)$ with the optimal value $10$. If we must have both muffins and coookies, then the next best option is $(x,y)=(6.25, 6.25)$ with the optimal value $12.5$.
```