# Lecture 11-2: The KKT Conditions

Download the original slides: [CMSE382-Lec11_2.pdf](CMSE382-Lec11_2.pdf)

```{warning}
This is an AI-generated transcript of the lecture slides and may contain errors or inaccuracies. Please refer to the original course materials for authoritative content.
```

---

## This Lecture

**Topics:**

- The convex case: KKT sufficiency
- Slater conditions
- The convex case: KKT necessity

**Announcements:**

- HW...

---

## Big picture

### Big picture view

![](../../../figures/useful_solution_charts/KKT_conditions_summary.png)

---

### Sufficiency and Necessity

**Sufficient**

- All KKT points are optima.
- Optima might not be KKT points.

**Necessary**

- All optima are KKT points.
- KKT points might not be optima.

---

## Convex case: KKT sufficiency

### Recall: KKT conditions for Inequality and equality constrained problems

**Theorem (Recall: Inequality and equality constrained problems)**

Let $\mathbf{x}^*$ be a local minimum of the problem

$$
\begin{aligned}
& \text{min}  & & f(\mathbf{x}) \\
& \text{such that} & & g_i(\mathbf{x}) \leq 0, i=1,2,\ldots m, \\
&  & & h_j(\mathbf{x}) = 0, j=1,2,\ldots p.
\end{aligned}
$$

where $f,g_1,\ldots,g_m,h_1,h_2,\ldots,h_p$ are continuously differentiable functions over $\mathbb{R}^n$.
Suppose that $\mathbf{x}^*$ is **regular**, then $\mathbf{x}^*$ is a **KKT point**.

- A necessary condition for local optimality of a regular point is that it is a KKT point.
- Regularity is not required in the linearly constrained case.

---

### Sufficiency of KKT conditions for convex problems

**Theorem**

Let $\mathbf{x}^*$ be a feasible solution of

$$
\begin{aligned}
& \text{min}  & & f(\mathbf{x}) \\
& \text{such that} & & g_i(\mathbf{x}) \leq 0, i=1,\ldots m, \\
&  & & h_j(\mathbf{x}) = 0, j=1,\ldots p.
\end{aligned}
$$

where $f,g_1,\ldots,g_m$ are continuously differentiable convex functions over $\mathbb{R}^n$, and $h_1,h_2,\ldots,h_p$ are affine functions.

Suppose that there exist $\lambda_1, \ldots, \lambda_m \geq 0$ and $\mu_1, \ldots, \mu_p \in \mathbb{R}$ such that

$$
\nabla f(\mathbf{x}^*) + \sum\limits_{i=1}^m{\lambda_i \nabla g_i(\mathbf{x}^*)} + \sum\limits_{j=1}^p{\mu_j \nabla h_j(\mathbf{x}^*)} = \mathbf{0},
$$

$$
\lambda_i g_i(\mathbf{x}^*) = 0, i=1,\ldots, m.
$$

Then $\mathbf{x}^*$ is an optimal solution.

In convex problems, the KKT conditions are always sufficient for optimality. No further conditions (such as regularity) are required.

---

## Convex case: KKT necessity

### Slater's condition

**Definition (Slater's condition)**

Given a set of convex inequalities

$$
 g_i(\mathbf{x}) \leq 0, \quad i=1,2,\ldots, m,
$$

where $g_1,g_2,\ldots, g_m$ are given convex functions, we say that Slater's condition is satisfied if there exists $\hat{\mathbf{x}} \in \mathbb{R}^n$ such that

$$
 g_i(\hat{\mathbf{x}}) < 0, \quad  i=1,2,\ldots, m.
$$

- Requires a point that strictly satisfies the constraints.
- Does not require knowledge on candidates for the optimal solution.
- Usually easier to check than regularity.

---

### Necessity of KKT conditions under Slater's condition

**Theorem**

Let $\mathbf{x}^*$ be an optimal solution of the problem

$$
\begin{aligned}
\min\ & f(\mathbf{x})  \\
\text{s.t.}\ & g_i(\mathbf{x}) \leq 0,\quad i=1,2,\dots, m,
\end{aligned}
$$

where $f, g_1, \dots, g_m$ are continuously differentiable functions over $\mathbb{R}^n$.
In addition, assume $g_1, g_2, \dots, g_m$ are convex functions and there exists $\hat{\mathbf{x}}$ which satisfies Slater's condition.
Then $\mathbf{x}^*$ is a KKT point.

Note: The point $\hat{\mathbf{x}}$ from Slater's condition does not need to be the same as $\mathbf{x}^*$ (the candidate for the optimal solution).

---

### Generalized Slater's condition

**Definition (Generalized Slater's condition)**

Consider the system

$$
\begin{aligned}
g_i(\mathbf{x}) &\leq 0, \quad i=1,2,\ldots, m, \\
h_j(\mathbf{x}) &\le 0, \quad j=1,2,\ldots, p, \\
s_k(\mathbf{x}) &= 0, \quad k=1,2,\ldots, q,
\end{aligned}
$$

where $g_1,g_2,\ldots, g_m$ are convex functions, and $h_1,h_2,\ldots, h_p, s_1,s_2,\ldots,s_k$ are affine functions.
We say that generalized Slater's condition is satisfied if there exists $\hat{\mathbf{x}} \in \mathbb{R}^n$ such that

$$
\begin{aligned}
g_i(\hat{\mathbf{x}}) &< 0, \quad i=1,2,\ldots, m, \\
h_j(\hat{\mathbf{x}}) &\leq 0, \quad j=1,2,\ldots, p, \\
s_k(\hat{\mathbf{x}}) &= 0, \quad k=1,2,\ldots, q.
\end{aligned}
$$

---

### Necessity of KKT conditions under the generalized Slater's condition

**Theorem**

Let $\mathbf{x}^*$ be an optimal solution of

$$
\begin{aligned}
\min\ & f(\mathbf{x}) \\
\text{s.t.}\ & g_i(\mathbf{x}) \le 0,\quad i = 1, \ldots, m \\
& h_j(\mathbf{x}) \le 0,\quad j = 1, \ldots, p \\
& s_k(\mathbf{x}) = 0,\quad k = 1, \ldots, q,
\end{aligned}
$$

where $f$ and all $g_i$ are continuously differentiable convex functions over $\mathbb{R}^n$, and all $h_j$, $s_k$ are affine.
Suppose that there exists $\hat{\mathbf{x}} \in \mathbb{R}^n$ satisfying the generalized Slater's condition.

Then $\mathbf{x}^*$ is a KKT point, i.e. there exist multipliers $\lambda_1, \ldots, \lambda_m,\eta_1,\ldots,\eta_p \ge 0$, $\mu_1,\ldots,\mu_q \in \mathbb{R}$ such that

$$
\nabla f (\mathbf{x}^*) + \sum_{i=1}^{m} \lambda_i\nabla g_i(\mathbf{x}^*) + \sum_{j=1}^{p} \eta_j\nabla h_j(\mathbf{x}^*) + \sum_{k=1}^{q} \mu_k\nabla s_k(\mathbf{x}^*) = 0,
$$

$$
\lambda_i g_i(\mathbf{x}^*) = 0, \quad i = 1,\ldots, m,
$$

$$
\eta_j h_j(\mathbf{x}^*) = 0, \quad j = 1,\ldots, p.
$$

---

## Big picture

### Big picture view

![](../../../figures/useful_solution_charts/KKT_conditions_summary.png)