Worksheet 12-1: Duality (with Solutions)#
Download: CMSE382-WS12_1.pdf, CMSE382-WS12_1-Soln.pdf
Warning
This is an AI-generated transcript of the worksheet and may contain errors or inaccuracies. Please refer to the original course materials for authoritative content.
Worksheet 12-1: Q1#
Consider the problem
Find the optimal solution by replacing \(x_1\) with \(x_2^2\) and making the objective a single variable function. What is the optimal value \(f^*\)? (Note \(f^*\) is the minimimal value of the objection function \(f(x_1,x_2) = x_1^2-x_2\), not the point \(x_1,x_2\) that achieves the minimum.)
Solution
After replacement, the objective function becomes \(f(x_2) = x_2^4-x_2\). Take a derivative, we have \(f'(x_2) = 4x_2^3-1\). Set \(f'(x_2)=0\), we have \(x_2 = \sqrt[3]{\frac{1}{4}}\). So, the optimal value is
\[f^* = -\frac{3}{4\sqrt[3]{4}} \approx -0.472.\]Now solve the problem using duality:
See this desmos plot for the visualization of the problem we discussed in class.
a. Write down the Lagrangian.
Solution
The Lagrangian is
\[L(x_1,x_2,\mu) = x_1^2-x_2 + \mu(x_1-x_2^2).\]b. What is \(\nabla_{x_1,x_2} L(x_1,x_2,\mu)\)? When is \(\nabla_{x_1,x_2} L(x_1,x_2,\mu) = 0\)?
Solution
The gradient is
\[\begin{split}\nabla_{x_1,x_2} L(x_1,x_2,\mu) = \begin{bmatrix} 2x_1 + \mu\\ -1 - 2\mu x_2 \end{bmatrix}.\end{split}\]The gradient is zero when \(x_1 = -\frac{\mu}{2}\) and \(x_2 = -\frac{1}{2\mu}\).
c. Write a formula for \(q(\mu) = \min_{x_1,x_2} L(x_1,x_2,\mu)\) in terms of \(\mu\).
Solution
From above, we have that the Lagrangian is minimized at \(x_1 = -\frac{\mu}{2}\) and \(x_2 = -\frac{1}{2\mu}\). This means the minimum value of the Lagrangian is
\[\begin{split}\begin{aligned} L\left(-\frac{\mu}{2}, -\frac{1}{2\mu}, \mu\right) &= \left(-\frac{\mu}{2}\right)^2 - \left(-\frac{1}{2\mu}\right) + \mu\left(-\frac{\mu}{2} - \left(-\frac{1}{2\mu}\right)^2\right)\\ &= -\frac{\mu^2}{4} + \frac{1}{4\mu}. \end{aligned}\end{split}\]d. What value of \(\mu\) maximizes \(q(\mu)\)?
Solution
Take a derivative, we have \(q'(\mu) = -\frac{\mu}{2} - \frac{1}{4\mu^2}\). Set \(q'(\mu)=0\), we have
\[\mu = -\sqrt[3]{\frac{1}{2}} = -2^{-1/3}.\]e. What is the optimal dual value \(q^* = \max_\mu q(\mu)\)?
Solution
The optimal dual value is
\[\begin{aligned} q^* = q(-2^{-1/3}) &= \frac{(-2^{-1/3})^2}{4} + \frac{1}{4(-2^{-1/3})} \approx -0.472 \end{aligned}\]We know from the weak duality theorem that \(q^* \leq f^*\). Is the bound tight for this problem? That is, do we have \(q^* = f^*\) in this problem?
Solution
Yes, we have \(q^* = f^*\), so the bound is tight.
Worksheet 12-1: Q2#
Consider the problem
Find the optimal solution by reducing the problem to a single variable unconstrained optimization problem without using duality.
Solution
The constraint \(x_2^2 \leq 0\) implies that \(x_2 = 0\). So, the problem reduces to \(\min_{x_1} x_1^2\), which has optimal solution \(x_1 = 0\) and optimal value \(f^* = 0\).
Write down the Lagrangian.
Solution
The Lagrangian is
\[L(x_1,x_2,\lambda) = x_1^2 - x_2 + \lambda x_2^2.\]Obtain the dual objective function \(q(\lambda)=\min_{x_1,x_2} L(x_1,x_2,\lambda)\). Note that you will need to consider the two cases: \(\lambda=0\) and \(\lambda>0\). This means your objective function will be a piece-wise function.
Solution
See this desmos plot for a visual of everything.
When \(\lambda=0\), the Lagrangian is just the objective function,
\[L(x_1,x_2,0) = x_1^2 -x_2.\]The gradient is
\[\begin{split}\nabla_{x_1,x_2} L(x_1,x_2,0) = \begin{bmatrix} 2x_1\\ -1 \end{bmatrix}\end{split}\]which is never zero. We also notice that the function goes to \(-\infty\) as \(x_2\) goes to \(\infty\), so there is no finite minimum. So, we have \(q(0) = -\infty\).
For \(\lambda > 0\), the gradient of the Lagrangian is
\[\begin{split}\nabla_{x_1,x_2} L(x_1,x_2,\lambda) = \begin{bmatrix} 2x_1\\ -1 + 2\lambda x_2 \end{bmatrix}\end{split}\]which is zero when \(x_1=0\) and \(x_2 = \frac{1}{2\lambda}\). The value of the Lagrangian at this point is
\[L\left(0, \frac{1}{2\lambda}, \lambda\right) = -\frac{1}{2\lambda} + \lambda \left(\frac{1}{2\lambda}\right)^2 = -\frac{1}{4\lambda}.\]So, we have
\[\begin{split}q(\lambda) = \begin{cases} -\infty & \text{if } \lambda = 0\\ -\frac{1}{4\lambda} & \text{if } \lambda > 0 \end{cases}.\end{split}\]What is \(q^* = \max_{\lambda \geq 0} q(\lambda)\)?
Solution
Since \(q(\lambda) = -\frac{1}{4\lambda}\) for \(\lambda > 0\), we have \(q(\lambda) \to 0\) as \(\lambda \to \infty\). So, we have \(q^* = \sup_{\lambda \geq 0} q(\lambda) = 0\).
Is the bound from the weak duality theorem tight for this problem? That is, do we have \(q^* = f^*\) in this problem?
Solution
Yes, we have \(q^* = 0 = f^*\), so the bound is tight.
Worksheet 12-1: Q3#
Using the desmos plot, what do you think the optimal solution \(f^*\) is?
Solution
The minimum occurs at \(x=\pm 1\) and \(y=0\), and the optimal value is \(f^* = 1\).
You can also get this by noting that the constraint is \(x^4 \ge 1\), so \(|x|\ge 1\).
Now solve the problem using duality:
a. Write down the Lagrangian.
Solution
The Lagrangian is
\[L(x,y,\lambda) = x^2+y^2 + \lambda(1-x^4).\]c. In a different Desmos plot from above, plot the Lagrangian as a function of \(x\) and \(y\) using a slider bar for different values of \(\lambda \geq 0\). What do you observe? What does this suggest about the dual function \(q(\lambda) = \min_{x,y} L(x,y,\lambda)\)? What is \(q^* = \max_\lambda q(\lambda)\)?
For an additional challenge, can you justify your observation without using Desmos?
Solution
When \(\lambda=0\), the Lagrangian is just the objective function, which is a bowl-shaped function with minimum at \(x=0\) and \(y=0\).
When \(\lambda > 0\), we can see that the function goes to \(-\infty\) as \(|x|\) goes to \(\infty\), so there is no finite minimum.
Since \(q(\lambda) = \min_{x,y} L(x,y,\lambda)\), we have \(q(0) = 0\) and \(q(\lambda) = -\infty\) for \(\lambda > 0\). This means that \(q^* = \max_\lambda q(\lambda) = 0\).
Based on what you found above, is the bound from the weak duality theorem tight for this problem? That is, do we have \(q^* = f^*\) in this problem?
Solution
No, we have \(q^* = 0 < 1 = f^*\), so the bound is not tight.