Lecture 12-3: Duality

Lecture 12-3: Duality#

Download the original slides: CMSE382-Lec12_3.pdf

Warning

This is an AI-generated transcript of the lecture slides and may contain errors or inaccuracies. Please refer to the original course materials for authoritative content.

This Lecture#

Topics:

  • Dual for strictly convex quadratic programming

  • Dual for convex quadratic programming

Announcements:

  • Homework 6 is due on Friday, April 17, 2026 at 11:59pm.

Last Time#

Dual objective function#

Consider the general model referred to as the primal model

\[\begin{split} \begin{aligned} & f^* = \text{min} & & f(\mathbf{x}) \\ & \text{such that} & & g_i(\mathbf{x}) \leq 0, i=1,2,\ldots m, \\ & & & h_j(\mathbf{x}) = 0, j=1,2,\ldots p, \\ & & & \mathbf{x} \in X, \text{ where } X \subseteq \mathbb{R}^n, \end{aligned} \end{split}\]

and \(f, g_i,h_j\) are functions defined on \(X\).

The Lagrangian of the problem is

\[ L(\mathbf{x},\boldsymbol{\lambda},\boldsymbol{\mu}) = f(\mathbf{x}) + \sum\limits_{i=1}^m{\lambda_i g_i(\mathbf{x})} + \sum\limits_{j=1}^p{\mu_j h_j(\mathbf{x})}, \]

The dual objective function \(q: \mathbb{R}_+^m \times \mathbb{R}^p \to \mathbb{R} \cup \{-\infty\}\) is

\[ q(\boldsymbol{\lambda},\boldsymbol{\mu})=\min_{x\in X}{L(\mathbf{x},\boldsymbol{\lambda},\boldsymbol{\mu})}, \]

Weak duality theorem#

Primal Problem

\[\begin{split} \begin{aligned} & f^* = \text{min} & & f(\mathbf{x}) \\ & \text{such that} & & g_i(\mathbf{x}) \leq 0, i=1,2,\ldots m, \\ & & & h_j(\mathbf{x}) = 0, j=1,2,\ldots p, \\ & & & \mathbf{x} \in X, \text{ where } X \subseteq \mathbb{R}^n, \end{aligned} \end{split}\]

and \(f, g_i,h_j\) are functions defined on \(X\).

Dual Problem

\[\begin{split} \begin{aligned} & q^* = \text{max} & & q(\boldsymbol{\lambda},\boldsymbol{\mu}) \\ & \text{such that} & & (\boldsymbol{\lambda},\boldsymbol{\mu}) \in \text{dom}(q), \end{aligned} \end{split}\]

where \(\text{dom}(q)=\{(\boldsymbol{\lambda},\boldsymbol{\mu}) \in \mathbb{R}_{+}^m \times \mathbb{R}^p: q(\boldsymbol{\lambda},\boldsymbol{\mu}) > -\infty\}\), and \(q(\boldsymbol{\lambda},\boldsymbol{\mu})=\min_{\mathbf{x}\in X}{L(\mathbf{x},\boldsymbol{\lambda},\boldsymbol{\mu})}\).

Theorem (Weak duality theorem)

Consider the primal problem and its dual. Then \(q^* \leq f^*\), where \(q^*, f^*\) are the optimal dual and primal values, respectively.

Desmos example

Strong duality of convex problems with equality & inequality constraints#

Primal Problem (P)

\[\begin{split} \begin{aligned} & f^* = \text{min} & & f(\mathbf{x}) \\ & \text{such that} & & g_i(\mathbf{x}) \leq 0, i=1,2,\ldots, m, \\ & & & h_j(\mathbf{x}) \le 0, j=1,2,\ldots, p, \\ & & & s_k(\mathbf{x}) = 0, k=1,2,\ldots, q, \\ & & & \mathbf{x} \in X, \end{aligned} \end{split}\]

Dual Problem (D)

\[\begin{split} \begin{aligned} & q^* = \text{max} & & q(\boldsymbol{\lambda},\boldsymbol{\mu}) \\ & \text{such that} & & (\boldsymbol{\lambda},\boldsymbol{\mu}) \in \text{dom}(q), \end{aligned} \end{split}\]

where \(\text{dom}(q)=\{(\boldsymbol{\lambda},\boldsymbol{\mu}) \in \mathbb{R}_{+}^m \times \mathbb{R}^p: q(\boldsymbol{\lambda},\boldsymbol{\mu}) > -\infty\}\), and \(q(\boldsymbol{\lambda},\boldsymbol{\mu})=\min_{\mathbf{x}\in X}{L(\mathbf{x},\boldsymbol{\lambda},\boldsymbol{\mu})}\).

  • For (P): \(X\) is a convex set and \(f,g_1,\ldots, g_m\) are convex functions over \(X\). The functions \(h_1,\ldots,h_p,s_1,\ldots,s_q\) are affine.

Theorem (Strong duality under equality & inequality constraints)

If the generalized Slater’s condition is satisfied in (P) and \(f^*\) has a finite optimal value, then the optimal value of (D) is attained, and the optimal values of the primal and dual problems are the same \(f^*=q^*\).

Dual for linear programming#

Primal

\[\begin{split} \begin{aligned} & f^* = \text{min} & & \mathbf{c}^{\top} \mathbf{x} \\ & \text{such that} & & A \mathbf{x} \leq \mathbf{b}, \end{aligned} \end{split}\]

Dual

\[\begin{split} \begin{aligned} & q^* = \text{max} & & -\mathbf{b}^{\top} \boldsymbol{\lambda} \\ & \text{such that} & & A^{\top} \boldsymbol{\lambda}=-\mathbf{c},\\ & & & \boldsymbol{\lambda} \geq 0. \end{aligned} \end{split}\]

Strong duality holds

If the primal problem is feasble (meaning the constraint set is not empty) and has a finite solution, then the optimal dual value is equal to the optimal primal value:

\[ q^*=f^*. \]

Quadratic Linear Programming#

Strictly Convex Quadratic Programming#

Primal

\[\begin{split} \begin{aligned} & f^* = \text{min} & & \mathbf{x}^{\top}Q \mathbf{x} + 2 \mathbf{c}^{\top} \mathbf{x} \\ & \text{such that} & & A \mathbf{x} \leq \mathbf{b}, \end{aligned} \end{split}\]

  • \(Q\in \mathbb{R}^{n \times n}\) is positive definite, \(\mathbf{c} \in \mathbb{R}^n\), and \(\mathbf{b} \in \mathbb{R}^m\).

Dual

\[\begin{split} \begin{aligned} & q^* = \max{q( \boldsymbol{\lambda})} & & \\ & \text{such that} & & \boldsymbol{\lambda} \geq 0. \end{aligned} \end{split}\]

  • Strong duality holds

  • \(L(\mathbf{x},\boldsymbol{\lambda}) = \mathbf{x}^T Q\mathbf{x} + 2\mathbf{c}^T \mathbf{x} + 2\boldsymbol{\lambda}^T (A\mathbf{x} - \mathbf{b})\)

  • \(\nabla_x L(\mathbf{x}^*,\boldsymbol{\lambda})=2 Q \mathbf{x}^* + 2 (A^{\top} \boldsymbol{\lambda} + \mathbf{c})=\mathbf{0}\)

  • \(\mathbf{x}^*=-Q^{-1}(\mathbf{c} + A^{\top}\boldsymbol{\lambda})\)

The objective function becomes

\[\begin{split} \begin{aligned} q(\boldsymbol{\lambda}) =-\boldsymbol{\lambda}^{\top} &A Q^{-1} A^T \boldsymbol{\lambda}\\ &-2(AQ^{-1} \mathbf{c}+\mathbf{b})^{\top} \boldsymbol{\lambda}\\ & - \mathbf{c}^{\top}Q^{-1}\mathbf{c}. \end{aligned} \end{split}\]

Dual for convex quadratic programming#

Convex Quadratic Program

\[\begin{split} \begin{aligned} & \text{min} & & \mathbf{x}^{\top} Q \mathbf{x} + 2 \mathbf{c}^{\top} \mathbf{x} \\ & \text{such that} & & A \mathbf{x} \leq \mathbf{b}, \end{aligned} \end{split}\]

where \(Q \in \mathbb{R}^{n \times n}\) is positive semi-definite, \(\mathbf{c} \in \mathbb{R}^n\), and \(\mathbf{b} \in \mathbb{R}^m\).

Since \(Q \succeq 0\)

  • \(Q\) is not necessarily invertible

  • The dual problem formulated for the strictly convex case is not possible in the convex case.

  • A new formulation for the convex case is needed.

  • The trick: write \(Q = D^\top D\) for some matrix \(D\), and make a new variable \(\mathbf{z} = D\mathbf{x}\).

Dual for convex quadratic programming#

Reformulated primal problem

\[\begin{split} \begin{aligned} & \text{min} & & \|\mathbf{z}\|^2 + 2 \mathbf{c}^{\top} \mathbf{x} \\ & \text{such that} & & A \mathbf{x} \leq \mathbf{b}, \\ & & & \mathbf{z} = D \mathbf{x}. \end{aligned} \end{split}\]

Dual Problem

\[\begin{split} \begin{aligned} & \text{max} & & -\|\boldsymbol{\mu}\|^2-2\mathbf{b}^{\top} \boldsymbol{\lambda} \\ & \text{such that} & & \mathbf{c} + A^{\top} \boldsymbol{\lambda} - D^{\top} \boldsymbol{\mu}=0, \\ & & & \boldsymbol{\lambda} \in \mathbb{R}^m_{+}, \boldsymbol{\mu} \in \mathbb{R}^n. \end{aligned} \end{split}\]
Contents