# Worksheet 10-2: KKT Conditions

Download: [CMSE382-WS10_2.pdf](CMSE382-WS10_2.pdf)

```{warning}
This is an AI-generated transcript of the worksheet and may contain errors or inaccuracies. Please refer to the original course materials for authoritative content.
```

---

## Worksheet 10-2: Q1

Consider the following problem:

$$

\begin{aligned}
& \min_{x,y} & & (x-0.1)^2+y^2 \\
& \text{s.t.} & & x + y \leq 1, \\
&             & & x-2y \leq 0.
\end{aligned}

$$

We will find the KKT conditions

(a) Write the Lagrangian function of this problem.

(b) Write down the constraints from the KKT condition (stationarity condition) for the problem.

(c) Write down the complementary slackness conditions for this problem.

(d) Write down the feasibility constraints (including the non-negativity constraints on the Lagrange multipliers) for this problem.

(e) Count the equations/inequalities found in the previous three parts. You should have 8 of them.

(f) Do you expect the local optima (if any) to be global optima? Why?

(g) In theory, you could now solve for the unknowns $x,y,\lambda_1,\lambda_2$ using the equations/inequalities you have. This turns into checking cases:

- Case 1: $\lambda_1 = \lambda_2 = 0$.
- Case 2: $\lambda_1 = 0$, $\lambda_2 > 0$.
- Case 3: $\lambda_1 > 0$, $\lambda_2 = 0$.
- Case 4: $\lambda_1 > 0$, $\lambda_2 > 0$.

In this example, Cases 3, and 4 do not lead to feasible points. Find the KKT point(s) for Cases 1 and 2 if they exist.

(h) What can you conclude about the optimal solution of this problem?

(i) Which of the constraints are active at the optimal solution?

(j) Take a look at [this desmos plot](https://www.desmos.com/3d/criwdot3cz). Visually confirm that the solution you found above is indeed the optimal solution. Then turn on the plot of $g(x,y)$, which is a copy of $f(x,y)$. Change the constraints for $g$ (the portions inside $\{\cdots\}$). Which constraint can be changed without changing the optimal solution? Which constraint can be changed to change the optimal solution? What does this have to do with the previous question?

---

## Worksheet 10-2: Q2

Consider the optimization problem $\min\{\mathbf{x}^T Q \mathbf{x} + 2 \mathbf{c}^T \mathbf{x}: A \mathbf{x}=\mathbf{b}\}$,
where $Q \in \mathbb{R}^{n\times n}$ is a positive definite matrix, $\mathbf{c} \in \mathbb{R}^n$, $\mathbf{b} \in \mathbb{R}^m$, and $A$ is an $m \times n$ matrix with linearly dependent rows. This is a convex optimization problem. Answer the following:

(a) Find the Lagrangian as a single matrix multiplication equation.

(b) Noting that we treat vectors as column vectors, write down the dimensions of all the matrices and vectors in the Lagrangian above.

(c) For a column vector $\mathbf{b}$, $\nabla_{\mathbf{x}}(\mathbf{b}^\top \mathbf{x}) = \mathbf{b}$. Use this to determine $\nabla_{\mathbf{x}} (\boldsymbol{\mu}^{\top}A \mathbf{x})$?

(d) Using the same as above, what is $\nabla_{\mathbf{x}} (2\mathbf{c}^{\top}\mathbf{x})$?

(e) Since $Q$ is symmetric, $\nabla_{\mathbf{x}} (\mathbf{x}^{\top}Q\mathbf{x}) = 2Q\mathbf{x}$. What is $\nabla_{\mathbf{x}} L(\mathbf{x},\boldsymbol{\mu})$?

(f) Write down the KKT conditions (stationarity and feasibility), and to simplify down the road, replace $\mu = 2\gamma$. What are the unknowns in these equations?

(g) Solve the stationarity condition for $\mathbf{x}$.

(h) Substitute the expression for $\mathbf{x}$ into the feasibility constraint, and solve for any other unknown.

(i) What is the stationary point in terms of $\mathbf{c}$, $Q$, $A$, and $\mathbf{b}$?

(j) Is the stationary point optimal? Explain.