Lecture 12-2: Duality

Lecture 12-2: Duality#

Download the original slides: CMSE382-Lec12_2.pdf

Warning

This is an AI-generated transcript of the lecture slides and may contain errors or inaccuracies. Please refer to the original course materials for authoritative content.

This Lecture#

Topics:

  • Strong duality in the convex case

  • Dual for linear programming

Announcements:

  • Homework 6 is due on Friday, April 17, 2026 at 11:59pm.

Last Time#

Dual objective function#

Consider the general model referred to as the primal model

\[\begin{split} \begin{aligned} & f^* = \text{min} & & f(\mathbf{x}) \\ & \text{such that} & & g_i(\mathbf{x}) \leq 0, i=1,2,\ldots m, \\ & & & h_j(\mathbf{x}) = 0, j=1,2,\ldots p, \\ & & & \mathbf{x} \in X, \text{ where } X \subseteq \mathbb{R}^n, \end{aligned} \end{split}\]

and \(f, g_i,h_j\) are functions defined on \(X\).

The Lagrangian of the problem is

\[ L(\mathbf{x},\boldsymbol{\lambda},\boldsymbol{\mu}) = f(\mathbf{x}) + \sum\limits_{i=1}^m{\lambda_i g_i(\mathbf{x})} + \sum\limits_{j=1}^p{\mu_j h_j(\mathbf{x})}, \]

The dual objective function \(q: \mathbb{R}_+^m \times \mathbb{R}^p \to \mathbb{R} \cup \{-\infty\}\) is

\[ q(\boldsymbol{\lambda},\boldsymbol{\mu})=\min_{x\in X}{L(\mathbf{x},\boldsymbol{\lambda},\boldsymbol{\mu})}, \]

Weak duality theorem#

Primal Problem

\[\begin{split} \begin{aligned} & f^* = \text{min} & & f(\mathbf{x}) \\ & \text{such that} & & g_i(\mathbf{x}) \leq 0, i=1,2,\ldots m, \\ & & & h_j(\mathbf{x}) = 0, j=1,2,\ldots p, \\ & & & \mathbf{x} \in X, \text{ where } X \subseteq \mathbb{R}^n, \end{aligned} \end{split}\]

and \(f, g_i,h_j\) are functions defined on \(X\).

Dual Problem

\[\begin{split} \begin{aligned} & q^* = \text{max} & & q(\boldsymbol{\lambda},\boldsymbol{\mu}) \\ & \text{such that} & & (\boldsymbol{\lambda},\boldsymbol{\mu}) \in \text{dom}(q), \end{aligned} \end{split}\]

where \(\text{dom}(q)=\{(\boldsymbol{\lambda},\boldsymbol{\mu}) \in \mathbb{R}_{+}^m \times \mathbb{R}^p: q(\boldsymbol{\lambda},\boldsymbol{\mu}) > -\infty\}\), and \(q(\boldsymbol{\lambda},\boldsymbol{\mu})=\min_{\mathbf{x}\in X}{L(\mathbf{x},\boldsymbol{\lambda},\boldsymbol{\mu})}\).

Theorem (Weak duality theorem)

Consider the primal problem and its dual. Then \(q^* \leq f^*\), where \(q^*, f^*\) are the optimal dual and primal values, respectively.

Desmos example

Strong Duality#

Strong duality of convex problems with equality & inequality constraints#

Primal Problem (P)

\[\begin{split} \begin{aligned} & f^* = \text{min} & & f(\mathbf{x}) \\ & \text{such that} & & g_i(\mathbf{x}) \leq 0, i=1,2,\ldots, m, \\ & & & h_j(\mathbf{x}) \le 0, j=1,2,\ldots, p, \\ & & & s_k(\mathbf{x}) = 0, k=1,2,\ldots, q, \\ & & & \mathbf{x} \in X, \end{aligned} \end{split}\]

Dual Problem (D)

\[\begin{split} \begin{aligned} & q^* = \text{max} & & q(\boldsymbol{\lambda},\boldsymbol{\mu}) \\ & \text{such that} & & (\boldsymbol{\lambda},\boldsymbol{\mu}) \in \text{dom}(q), \end{aligned} \end{split}\]

where \(\text{dom}(q)=\{(\boldsymbol{\lambda},\boldsymbol{\mu}) \in \mathbb{R}_{+}^m \times \mathbb{R}^p: q(\boldsymbol{\lambda},\boldsymbol{\mu}) > -\infty\}\), and \(q(\boldsymbol{\lambda},\boldsymbol{\mu})=\min_{\mathbf{x}\in X}{L(\mathbf{x},\boldsymbol{\lambda},\boldsymbol{\mu})}\).

  • For (P): \(X\) is a convex set and \(f,g_1,\ldots, g_m\) are convex functions over \(X\). The functions \(h_1,\ldots,h_p,s_1,\ldots,s_q\) are affine.

Theorem (Strong duality under equality & inequality constraints)

If the generalized Slater’s condition is satisfied in (P) and \(f^*\) has a finite optimal value, then the optimal value of (D) is attained, and the optimal values of the primal and dual problems are the same \(f^*=q^*\).

Duality gap#

Definition (Duality gap)

The duality gap is the difference between the optimal primal value \(f^*\) and the optimal dual value \(q^*\) given by

\[ \Delta = f^* - q^* \]

  • \(\Delta \geq 0\).

  • \(\Delta = 0\) if and only if strong duality holds.

Dual for linear programming#

Primal

\[\begin{split} \begin{aligned} & f^* = \text{min} & & \mathbf{c}^{\top} \mathbf{x} \\ & \text{such that} & & A \mathbf{x} \leq \mathbf{b}, \end{aligned} \end{split}\]

Dual

\[\begin{split} \begin{aligned} & q^* = \text{max} & & -\mathbf{b}^{\top} \boldsymbol{\lambda} \\ & \text{such that} & & A^{\top} \boldsymbol{\lambda}=-\mathbf{c},\\ & & & \boldsymbol{\lambda} \geq 0. \end{aligned} \end{split}\]

Strong duality holds

If the primal problem is feasble (meaning the constraint set is not empty) and has a finite solution, then the optimal dual value is equal to the optimal primal value:

\[ q^*=f^*. \]
Contents