# Lecture 12-3: Duality

Download the original slides: [CMSE382-Lec12_3.pdf](CMSE382-Lec12_3.pdf)

```{warning}
This is an AI-generated transcript of the lecture slides and may contain errors or inaccuracies. Please refer to the original course materials for authoritative content.
```

---

## This Lecture

**Topics:**

- Dual for strictly convex quadratic programming
- Dual for convex quadratic programming

**Announcements:**

- Homework 6 is due on Friday, April 17, 2026 at 11:59pm.

---

## Last Time

### Dual objective function

Consider the general model referred to as the primal model

$$
\begin{aligned}
& f^* = \text{min}  & & f(\mathbf{x}) \\
& \text{such that} & & g_i(\mathbf{x}) \leq 0, i=1,2,\ldots m, \\
&  & & h_j(\mathbf{x}) = 0, j=1,2,\ldots p, \\
&  & & \mathbf{x} \in X, \text{ where } X \subseteq \mathbb{R}^n,
\end{aligned}
$$

and $f, g_i,h_j$ are functions defined on $X$.

The Lagrangian of the problem is

$$
L(\mathbf{x},\boldsymbol{\lambda},\boldsymbol{\mu}) = f(\mathbf{x}) + \sum\limits_{i=1}^m{\lambda_i g_i(\mathbf{x})} + \sum\limits_{j=1}^p{\mu_j h_j(\mathbf{x})},
$$

The dual objective function $q: \mathbb{R}_+^m \times \mathbb{R}^p \to \mathbb{R} \cup \{-\infty\}$ is

$$
q(\boldsymbol{\lambda},\boldsymbol{\mu})=\min_{x\in X}{L(\mathbf{x},\boldsymbol{\lambda},\boldsymbol{\mu})},
$$

---

### Weak duality theorem

**Primal Problem**

$$
\begin{aligned}
& f^* = \text{min}  & & f(\mathbf{x}) \\
& \text{such that} & & g_i(\mathbf{x}) \leq 0, i=1,2,\ldots m, \\
&  & & h_j(\mathbf{x}) = 0, j=1,2,\ldots p, \\
&  & & \mathbf{x} \in X, \text{ where } X \subseteq \mathbb{R}^n,
\end{aligned}
$$

and $f, g_i,h_j$ are functions defined on $X$.

**Dual Problem**

$$
\begin{aligned}
& q^* = \text{max}  & & q(\boldsymbol{\lambda},\boldsymbol{\mu}) \\
& \text{such that} & &  (\boldsymbol{\lambda},\boldsymbol{\mu}) \in \text{dom}(q),
\end{aligned}
$$

where $\text{dom}(q)=\{(\boldsymbol{\lambda},\boldsymbol{\mu}) \in \mathbb{R}_{+}^m \times \mathbb{R}^p: q(\boldsymbol{\lambda},\boldsymbol{\mu}) > -\infty\}$, and $q(\boldsymbol{\lambda},\boldsymbol{\mu})=\min_{\mathbf{x}\in X}{L(\mathbf{x},\boldsymbol{\lambda},\boldsymbol{\mu})}$.

**Theorem (Weak duality theorem)**

Consider the primal problem and its dual. Then
$q^* \leq f^*$,
where $q^*, f^*$ are the optimal dual and primal values, respectively.

[Desmos example](https://www.desmos.com/3d/yppnizahux)

---

### Strong duality of convex problems with equality & inequality constraints

**Primal Problem (P)**

$$
\begin{aligned}
& f^* = \text{min}  & & f(\mathbf{x}) \\
& \text{such that} & & g_i(\mathbf{x})  \leq 0,  i=1,2,\ldots, m, \\
&                  & & h_j(\mathbf{x}) \le 0,  j=1,2,\ldots, p, \\
&                  & & s_k(\mathbf{x}) = 0,  k=1,2,\ldots, q, \\
&  & & \mathbf{x} \in X,
\end{aligned}
$$

**Dual Problem (D)**

$$
\begin{aligned}
& q^* = \text{max}  & & q(\boldsymbol{\lambda},\boldsymbol{\mu}) \\
& \text{such that} & &  (\boldsymbol{\lambda},\boldsymbol{\mu}) \in \text{dom}(q),
\end{aligned}
$$

where $\text{dom}(q)=\{(\boldsymbol{\lambda},\boldsymbol{\mu}) \in \mathbb{R}_{+}^m \times \mathbb{R}^p: q(\boldsymbol{\lambda},\boldsymbol{\mu}) > -\infty\}$, and $q(\boldsymbol{\lambda},\boldsymbol{\mu})=\min_{\mathbf{x}\in X}{L(\mathbf{x},\boldsymbol{\lambda},\boldsymbol{\mu})}$.

- For (P): $X$ is a convex set and $f,g_1,\ldots, g_m$ are convex functions over $X$. The functions $h_1,\ldots,h_p,s_1,\ldots,s_q$ are affine.

**Theorem (Strong duality under equality & inequality constraints)**

If the generalized Slater's condition is satisfied in (P) and $f^*$ has a finite optimal value, then the optimal value of (D) is attained, and the optimal values of the primal and dual problems are the same $f^*=q^*$.

---

### Dual for linear programming

**Primal**

$$
\begin{aligned}
& f^* = \text{min}  & & \mathbf{c}^{\top} \mathbf{x} \\
& \text{such that} & & A \mathbf{x} \leq \mathbf{b},
\end{aligned}
$$

**Dual**

$$
\begin{aligned}
& q^* = \text{max}  & & -\mathbf{b}^{\top} \boldsymbol{\lambda} \\
& \text{such that} & & A^{\top} \boldsymbol{\lambda}=-\mathbf{c},\\
& & & \boldsymbol{\lambda} \geq 0.
\end{aligned}
$$

**Strong duality holds**

If the primal problem is feasble (meaning the constraint set is not empty) and has a finite solution, then the optimal dual value is equal to the optimal primal value:

$$
q^*=f^*.
$$

---

## Quadratic Linear Programming

### *Strictly* Convex Quadratic Programming

**Primal**

$$
\begin{aligned}
& f^* = \text{min}  & & \mathbf{x}^{\top}Q \mathbf{x} + 2 \mathbf{c}^{\top} \mathbf{x} \\
& \text{such that} & & A \mathbf{x} \leq \mathbf{b},
\end{aligned}
$$

- $Q\in \mathbb{R}^{n \times n}$ is *positive definite*, $\mathbf{c} \in \mathbb{R}^n$, and $\mathbf{b} \in \mathbb{R}^m$.

**Dual**

$$
\begin{aligned}
& q^* = \max{q( \boldsymbol{\lambda})} & &  \\
& \text{such that} & & \boldsymbol{\lambda} \geq 0.
\end{aligned}
$$

- Strong duality holds
- $L(\mathbf{x},\boldsymbol{\lambda}) = \mathbf{x}^T Q\mathbf{x} + 2\mathbf{c}^T \mathbf{x} + 2\boldsymbol{\lambda}^T (A\mathbf{x} - \mathbf{b})$
- $\nabla_x L(\mathbf{x}^*,\boldsymbol{\lambda})=2 Q \mathbf{x}^* + 2 (A^{\top} \boldsymbol{\lambda} + \mathbf{c})=\mathbf{0}$
- $\mathbf{x}^*=-Q^{-1}(\mathbf{c} + A^{\top}\boldsymbol{\lambda})$

The objective function becomes

$$
\begin{aligned}
q(\boldsymbol{\lambda})
=-\boldsymbol{\lambda}^{\top} &A Q^{-1} A^T \boldsymbol{\lambda}\\
&-2(AQ^{-1} \mathbf{c}+\mathbf{b})^{\top} \boldsymbol{\lambda}\\
& - \mathbf{c}^{\top}Q^{-1}\mathbf{c}.
\end{aligned}
$$

---

### Dual for convex quadratic programming

**Convex Quadratic Program**

$$
\begin{aligned}
& \text{min}  & & \mathbf{x}^{\top} Q \mathbf{x} + 2 \mathbf{c}^{\top} \mathbf{x} \\
& \text{such that} & & A \mathbf{x} \leq \mathbf{b},
\end{aligned}
$$

where $Q \in \mathbb{R}^{n \times n}$ is *positive semi-definite*, $\mathbf{c} \in \mathbb{R}^n$, and $\mathbf{b} \in \mathbb{R}^m$.

Since $Q \succeq 0$

- $Q$ is not necessarily invertible
- The dual problem formulated for the strictly convex case is not possible in the convex case.
- A new formulation for the convex case is needed.
- The trick: write $Q = D^\top D$ for some matrix $D$, and make a new variable $\mathbf{z} = D\mathbf{x}$.

---

### Dual for convex quadratic programming

**Reformulated primal problem**

$$
\begin{aligned}
& \text{min}  & & \|\mathbf{z}\|^2 + 2 \mathbf{c}^{\top} \mathbf{x} \\
& \text{such that} & & A \mathbf{x} \leq \mathbf{b}, \\
&                  & & \mathbf{z} = D \mathbf{x}.
\end{aligned}
$$

**Dual Problem**

$$
\begin{aligned}
& \text{max}  & & -\|\boldsymbol{\mu}\|^2-2\mathbf{b}^{\top} \boldsymbol{\lambda} \\
& \text{such that} & & \mathbf{c} + A^{\top} \boldsymbol{\lambda} - D^{\top} \boldsymbol{\mu}=0, \\
&                  & & \boldsymbol{\lambda} \in \mathbb{R}^m_{+}, \boldsymbol{\mu} \in \mathbb{R}^n.
\end{aligned}
$$