Worksheet 12-2: Duality (with Solutions)#
Download: CMSE382-WS12_2.pdf, CMSE382-WS12_2-Soln.pdf
Warning
This is an AI-generated transcript of the worksheet and may contain errors or inaccuracies. Please refer to the original course materials for authoritative content.
Worksheet 12-2: Q1#
Consider the problem
Does strong duality hold? You need to check whether the problem is convex, and whether the constraints satisfy the generalized Slater’s condition.
Solution
The problem is convex because the objective is a convex function (quadratic with positive semidefinite Hessian) and the constraint is a convex set (a half plane). The problem also satisfies the generalized Slater’s condition because there exists a strictly feasible point, e.g., \((x,y)=(0,0)\), that satisfies \(x+y+1<0\). Even more simply, the constraint is affine so generalized Slater’s condition is automatically held without even needing to check for a point off the boundary. Therefore, strong duality holds for this problem.
Since both of these are true, strong duality holds.
Now, determine \(q^*\) using the following steps:
a. What is the Lagrangian, \(L(x,y,\lambda)\), and the gradient of the Lagrangian \(\nabla_{x,y}L(x,y,\lambda)\)?
Solution
\[\begin{split} \begin{aligned} L(x,y,\lambda) &= x^2+y^2 + \lambda(x+y+1)\\ \nabla_{x,y}L(x,y,\lambda) &= \begin{bmatrix} 2x+\lambda\\ 2y+\lambda \end{bmatrix} \end{aligned} \end{split}\]b. When is the gradient of the Lagrangian zero? Solve for \(x\) and \(y\) in terms of \(\lambda\).
Solution
The gradient is 0 when \(2x+\lambda=0\) and \(2y+\lambda=0\). Solving these equations gives \(x=-\tfrac{1}{2}\lambda\) and \(y=-\tfrac{1}{2}\lambda\).
c. For a fixed \(\lambda\), what is the minimum value of \(L(x,y,\lambda)\) over \(x,y\)? Use this to write \(q(\lambda) = \inf_{x,y} L(x,y,\lambda)\).
Solution
The minimum occurs when the gradient is 0, so from the last problem at \(L(-\tfrac{1}{2}\lambda, -\tfrac{1}{2}\lambda, \lambda)\). Substituting these values back into the Lagrangian gives
\[\begin{split} \begin{aligned} q(\lambda) &= L(-\tfrac{1}{2}\lambda, -\tfrac{1}{2}\lambda, \lambda)\\ &= \tfrac{1}{4}\lambda^2 + \tfrac{1}{4}\lambda^2 + \lambda(-\tfrac{1}{2}\lambda - \tfrac{1}{2}\lambda + 1)\\ &= \tfrac{1}{2}\lambda^2 + \lambda(1 - \lambda)\\ &= \tfrac{1}{2}\lambda^2 + \lambda - \lambda^2\\ &= -\tfrac{1}{2}\lambda^2 + \lambda \end{aligned} \end{split}\]d. What is \(q^*=\max_{\lambda} q(\lambda)\)?
Solution
We need to maximize \(q(\lambda)\), so we take a derivative to get \(q'(\lambda) = -\lambda + 1\). Setting this equal to 0 gives \(\lambda=1\). So \(q\) is maximized when \(\lambda=1\), and the maximum value is \(q^*=q(1) = -\tfrac{1}{2} + 1 = \tfrac{1}{2}\).
What do you know about \(f^*\), the optimal value of the primal problem?
Solution
From the first question, we know that strong duality holds, so \(f^* = q^*\). Therefore, \(f^* = \tfrac{1}{2}\).
See this desmos for a visualization of the problem.
Worksheet 12-2: Q2#
Consider the problem
Does strong duality hold? You need to check whether the problem is convex, and whether the constraints satisfy the generalized Slater’s condition.
Solution
The problem is convex because the objective is a convex function (linear) and the constraints are convex sets (half planes). The problem also satisfies the Slater’s condition because there exists a strictly feasible point, e.g., \((x,y)=(0,2)\), that satisfies \(-3x+3y<-1\) and \(-3x-2y<-4\).
What are the matrices \(A\), \(\mathbf{b}\), and \(\mathbf{c}\) such that the problem can be written as
\[\begin{split} \begin{aligned} \min &\quad \mathbf{c}^\top \mathbf{x}\\ \text{s.t.} &\quad A \mathbf{x}\leq\mathbf{b} \end{aligned} \end{split}\]Solution
With \(\mathbf{x} = \begin{bmatrix} x\\y \end{bmatrix}\), we have \(A = \begin{bmatrix} -3 & 3\\ -3 & -2 \end{bmatrix}\), and \(\mathbf{b} = \begin{bmatrix} -1\\ -4 \end{bmatrix}\).
What is the dual problem?
Solution
The dual problem is
\[\begin{split} \begin{aligned} \max &\quad -\mathbf{b}^\top \boldsymbol{\lambda}\\ \text{s.t.} &\quad A^\top \boldsymbol{\lambda} = -\mathbf{c}\\ &\quad \boldsymbol{\lambda} \geq 0 \end{aligned} \end{split}\]which for our problem is
\[\begin{split} \begin{aligned} \max &\quad \lambda_1+4\lambda_2\\ \text{s.t.} &\quad -3\lambda_1 - 3\lambda_2 = -2\\ & \quad 3\lambda_1 - 2\lambda_2 = -1\\ &\quad \lambda_1, \lambda_2 \geq 0 \end{aligned} \end{split}\]Solve the dual problem to find \(q^*\).
Solution
From the constraints, we have \(\lambda_1 = \tfrac{1}{15}\) and \(\lambda_2 = \tfrac{3}{5}\). Substituting these values into the objective gives \(q^* = \tfrac{1}{15}+4\tfrac{3}{5} = \tfrac{1}{15}+\tfrac{12}{5} = \tfrac{1+36}{15} = \tfrac{37}{15}\).
Use this desmos to visualize the primal problem. What is \(f^*\), the optimal value of the primal problem? Is it the same as your \(q^*\)?
Solution
The optimal value of the primal problem is marked as a green point, where we see that the optimal value is \(f^* = \tfrac{37}{15}\approx 2.47\), which is the same as \(q^*\), as expected from strong duality.
Note that where the optimal value is attained is not the same for the primal and dual problems, and not part of anything we figured out above. In the desmos, you are just checking to see if the \(z\)-coordinate of the point marked is the same as your calculated \(q^*\).
Worksheet 12-2: Q3#
Consider the problem
This is a linear programming problem. What are the matrices \(A\) \(\mathbf{b}\), and \(\mathbf{c}\) such that the problem can be written as
\[\begin{split} \begin{aligned} \min &\quad \mathbf{c}^\top \mathbf{x}\\ \text{s.t.} &\quad A \mathbf{x}\leq\mathbf{b} \end{aligned} \end{split}\]Solution
With \(\mathbf{x} = \begin{bmatrix} x\\y\\z \end{bmatrix}\), we have \(A = \begin{bmatrix} 1 & 1 & 1\\ 1 & 0 & -1 \end{bmatrix}\), and \(\mathbf{b} = \begin{bmatrix} 1\\ -2 \end{bmatrix}\).
What is the dual problem?
Solution
The dual problem is
\[\begin{split} \begin{aligned} \max &\quad -\mathbf{b}^\top \boldsymbol{\lambda}\\ \text{s.t.} &\quad A^\top \boldsymbol{\lambda} = -\mathbf{c}\\ &\quad \boldsymbol{\lambda} \geq 0 \end{aligned} \end{split}\]which for our problem is
\[\begin{split} \begin{aligned} \max &\quad -\lambda_1+2\lambda_2\\ \text{s.t.} &\quad \lambda_1 + \lambda_2 = 3\\ & \quad \lambda_1 = 2\\ & \quad \lambda_1 - \lambda_2 = 1\\ &\quad \lambda_1, \lambda_2 \geq 0 \end{aligned} \end{split}\]Solve the dual problem to find \(q^*\).
Solution
From the constraints, we have \(\lambda_1 = 2\) and \(\lambda_2 = 1\). Substituting these values into the objective gives \(q^* = -2 + 2 = 0\).
What is \(f^*\), the optimal value of the primal problem? Justify your answer.
Solution
Since the primal problem is a linear program and the dual is feasible, strong duality holds, so \(f^* = q^*\). Therefore, \(f^* = 0\).