# Worksheet 12-2: Duality (with Solutions)

Download: [CMSE382-WS12_2.pdf](CMSE382-WS12_2.pdf), [CMSE382-WS12_2-Soln.pdf](CMSE382-WS12_2-Soln.pdf)

```{warning}
This is an AI-generated transcript of the worksheet and may contain errors or inaccuracies. Please refer to the original course materials for authoritative content.
```

---

## Worksheet 12-2: Q1

Consider the problem

$$
\begin{aligned}
\min &\quad x^2+y^2\\
\text{s.t.} &\quad x+y+1 \leq 0
\end{aligned}
$$

1. Does strong duality hold? You need to check whether the problem is convex, and whether the constraints satisfy the generalized Slater's condition.

   ```{dropdown} Solution
   The problem is convex because the objective is a convex function (quadratic with positive semidefinite Hessian) and the constraint is a convex set (a half plane). The problem also satisfies the generalized Slater's condition because there exists a strictly feasible point, e.g., $(x,y)=(0,0)$, that satisfies $x+y+1<0$. Even more simply, the constraint is affine so generalized Slater's condition is automatically held without even needing to check for a point off the boundary. Therefore, strong duality holds for this problem.

   Since both of these are true, strong duality holds.
   ```

2. Now, determine $q^*$ using the following steps:

   a. What is the Lagrangian, $L(x,y,\lambda)$, and the gradient of the Lagrangian $\nabla_{x,y}L(x,y,\lambda)$?

   ```{dropdown} Solution

   $$
   \begin{aligned}
   L(x,y,\lambda) &= x^2+y^2 + \lambda(x+y+1)\\
   \nabla_{x,y}L(x,y,\lambda) &=
   \begin{bmatrix}
   2x+\lambda\\
   2y+\lambda
   \end{bmatrix}
   \end{aligned}
   $$

   ```

   b. When is the gradient of the Lagrangian zero? Solve for $x$ and $y$ in terms of $\lambda$.

   ```{dropdown} Solution
   The gradient is 0 when $2x+\lambda=0$ and $2y+\lambda=0$.
   Solving these equations gives $x=-\tfrac{1}{2}\lambda$ and $y=-\tfrac{1}{2}\lambda$.
   ```

   c. For a fixed $\lambda$, what is the minimum value of $L(x,y,\lambda)$ over $x,y$? Use this to write $q(\lambda) = \inf_{x,y} L(x,y,\lambda)$.

   ```{dropdown} Solution
   The minimum occurs when the gradient is 0, so from the last problem at $L(-\tfrac{1}{2}\lambda, -\tfrac{1}{2}\lambda, \lambda)$.
   Substituting these values back into the Lagrangian gives

   $$
   \begin{aligned}
   q(\lambda)
   &= L(-\tfrac{1}{2}\lambda, -\tfrac{1}{2}\lambda, \lambda)\\
   &= \tfrac{1}{4}\lambda^2 + \tfrac{1}{4}\lambda^2 + \lambda(-\tfrac{1}{2}\lambda - \tfrac{1}{2}\lambda + 1)\\
   &= \tfrac{1}{2}\lambda^2 + \lambda(1 - \lambda)\\
   &= \tfrac{1}{2}\lambda^2 + \lambda - \lambda^2\\
   &= -\tfrac{1}{2}\lambda^2 + \lambda
   \end{aligned}
   $$

   ```

   d. What is $q^*=\max_{\lambda} q(\lambda)$?

   ```{dropdown} Solution
   We need to maximize $q(\lambda)$, so we take a derivative to get
   $q'(\lambda) = -\lambda + 1$. Setting this equal to 0 gives $\lambda=1$.
   So $q$ is maximized when $\lambda=1$, and the maximum value is $q^*=q(1) = -\tfrac{1}{2} + 1 = \tfrac{1}{2}$.
   ```

3. What do you know about $f^*$, the optimal value of the primal problem?

   ```{dropdown} Solution
   From the first question, we know that strong duality holds, so $f^* = q^*$. Therefore, $f^* = \tfrac{1}{2}$.

   See [this desmos](https://www.desmos.com/3d/llg1yaicqh) for a visualization of the problem.
   ```

---

## Worksheet 12-2: Q2

Consider the problem

$$
\begin{aligned}
\min &\quad 2x+y\\
  \text{s.t.} &\quad -3x+3y\leq -1\\
&\quad -3x-2y \leq -4
\end{aligned}
$$

1. Does strong duality hold? You need to check whether the problem is convex, and whether the constraints satisfy the generalized Slater's condition.

   ```{dropdown} Solution
   The problem is convex because the objective is a convex function (linear) and the constraints are convex sets (half planes). The problem also satisfies the Slater's condition because there exists a strictly feasible point, e.g., $(x,y)=(0,2)$, that satisfies $-3x+3y<-1$ and $-3x-2y<-4$.
   ```

2. What are the matrices $A$, $\mathbf{b}$, and $\mathbf{c}$ such that the problem can be written as

   $$
   \begin{aligned} \min &\quad \mathbf{c}^\top \mathbf{x}\\
   \text{s.t.} &\quad A \mathbf{x}\leq\mathbf{b}
   \end{aligned}
   $$

   ```{dropdown} Solution
   With $\mathbf{x} = \begin{bmatrix} x\\y \end{bmatrix}$, we have 
   $A = \begin{bmatrix} -3 & 3\\ -3 & -2 \end{bmatrix}$, and $\mathbf{b} = \begin{bmatrix} -1\\ -4 \end{bmatrix}$.
   ```

3. What is the dual problem?

   ```{dropdown} Solution
   The dual problem is

   $$
   \begin{aligned}
   \max &\quad -\mathbf{b}^\top \boldsymbol{\lambda}\\
   \text{s.t.} &\quad A^\top \boldsymbol{\lambda} = -\mathbf{c}\\
   &\quad \boldsymbol{\lambda} \geq 0
   \end{aligned}
   $$

   which for our problem is 

   $$
   \begin{aligned}
   \max &\quad \lambda_1+4\lambda_2\\
    \text{s.t.} &\quad 
   -3\lambda_1 - 3\lambda_2 = -2\\
   & \quad 3\lambda_1 - 2\lambda_2 = -1\\
   &\quad \lambda_1, \lambda_2 \geq 0
   \end{aligned}
   $$
   ```

4. Solve the dual problem to find $q^*$.

   ```{dropdown} Solution
   From the constraints, we have $\lambda_1 = \tfrac{1}{15}$ and $\lambda_2 = \tfrac{3}{5}$. Substituting these values into the objective gives $q^* = \tfrac{1}{15}+4\tfrac{3}{5} = \tfrac{1}{15}+\tfrac{12}{5} = \tfrac{1+36}{15} = \tfrac{37}{15}$.
   ```

5. Use [this desmos](https://www.desmos.com/3d/ubtk3ztody) to visualize the primal problem. What is $f^*$, the optimal value of the primal problem? Is it the same as your $q^*$?

   ```{dropdown} Solution
   The optimal value of the primal problem is marked as a green point, where we see that the optimal value is $f^* = \tfrac{37}{15}\approx 2.47$, which is the same as $q^*$, as expected from strong duality.

   Note that *where* the optimal value is attained is not the same for the primal and dual problems, and not part of anything we figured out above. In the desmos, you are just checking to see if the $z$-coordinate of the point marked is the same as your calculated $q^*$.
   ```

---

## Worksheet 12-2: Q3

Consider the problem

$$
\begin{aligned}
\min &\quad -3x-2y-1z\\
   \text{s.t.} &\quad x+y+z\leq1\\
&\quad x-z \leq -2
\end{aligned}
$$

1. This is a linear programming problem. What are the matrices $A$ $\mathbf{b}$, and $\mathbf{c}$ such that the problem can be written as

   $$
   \begin{aligned}
   \min &\quad \mathbf{c}^\top \mathbf{x}\\
   \text{s.t.} &\quad A \mathbf{x}\leq\mathbf{b}
   \end{aligned}
   $$

   ```{dropdown} Solution
   With $\mathbf{x} = \begin{bmatrix} x\\y\\z \end{bmatrix}$, we have 
   $A = \begin{bmatrix} 1 & 1 & 1\\ 1 & 0 & -1 \end{bmatrix}$, and $\mathbf{b} = \begin{bmatrix} 1\\ -2 \end{bmatrix}$.
   ```

2. What is the dual problem?

   ```{dropdown} Solution
   The dual problem is

   $$
   \begin{aligned}
   \max &\quad -\mathbf{b}^\top \boldsymbol{\lambda}\\
   \text{s.t.} &\quad A^\top \boldsymbol{\lambda} = -\mathbf{c}\\
   &\quad \boldsymbol{\lambda} \geq 0
   \end{aligned}
   $$

   which for our problem is 

   $$
   \begin{aligned}
   \max &\quad -\lambda_1+2\lambda_2\\
   \text{s.t.} &\quad 
   \lambda_1 + \lambda_2 = 3\\
   & \quad \lambda_1 = 2\\
   & \quad \lambda_1 - \lambda_2 = 1\\
   &\quad \lambda_1, \lambda_2 \geq 0
   \end{aligned}
   $$
   ```

3. Solve the dual problem to find $q^*$.

   ```{dropdown} Solution
   From the constraints, we have $\lambda_1 = 2$ and $\lambda_2 = 1$. Substituting these values into the objective gives $q^* = -2 + 2 = 0$.
   ```

4. What is $f^*$, the optimal value of the primal problem? Justify your answer.

   ```{dropdown} Solution
   Since the primal problem is a linear program and the dual is feasible, strong duality holds, so $f^* = q^*$. Therefore, $f^* = 0$.
   ```