Worksheet 9-1: Optimization Over Convex Sets

Worksheet 9-1: Optimization Over Convex Sets#

Download: CMSE382-WS9_1.pdf

Warning

This is an AI-generated transcript of the worksheet and may contain errors or inaccuracies. Please refer to the original course materials for authoritative content.

Worksheet 9-1: Q1#

Consider the problem

\[ \min_{\mathbf{x}=(x_1,x_2)} f(\mathbf{x}) = -x_1x_2, \quad \text{s.t. } \mathbf{e}^{\top}\mathbf{x}=1, \text{ where } \mathbf{e}^T=\begin{bmatrix}1 & 1\end{bmatrix}, \]

The feasible set for this problem is

\[ U=\{\mathbf{x}\in\mathbb{R}^2:\mathbf{e}^{\top}\mathbf{x}=1\}=\{\mathbf{x}\in\mathbb{R}^2:x_1+x_2=1\}. \]

  1. Take a look at this desmos plot. What is the blue surface? What is the red line? What is the green line? Move the slider for \(a\) around, what is the grey point?

  2. Based on the plot, does this appear to be a convex problem?

  3. Let’s find the solution without using the stationarity condition first. For a point \(\mathbf{x}=(x_1,x_2)\in U\), write down \(\mathbf{x}\) in terms of just \(x_1\). Then write down \(f(\mathbf{x})\) restricted to \(U\) in terms of just \(x_1\).

  4. Great, this is a function in one variable! Find the minimum. Use this to determine the minimum for the problem. Move the grey point in the desmos plot to check your answer.

  5. Let’s go back and understand the stationarity condition for this problem. First, what is \(\nabla f(x_1,x_2)\)?

  6. We’ll start with a point that isn’t a stationary point and show that the stationarity condition doesn’t hold. For the point \(\mathbf{x}^*=(0,1)\) and some other point \(\mathbf{x}=(x_1,x_2)\in U\), write down the stationarity condition we would check. Put it in terms of only \(x_1\).

  7. To show that \(\mathbf{x}^*=(0,1)\) is not a stationary point, use your calculation above to find a point \(\mathbf{x}\in U\) that does not satisfy the stationarity condition.

  8. Now, we’ll do this for \(\mathbf{x}^*\) which gives the minimum that you found on the first page, which should have been \(\mathbf{x}^*=\left(\tfrac{1}{2},\tfrac{1}{2}\right)\). For \(\mathbf{x}^*\) equal to that point, what is \(\nabla f(\mathbf{x}^*)\)?

  9. Say we have some point \(\mathbf{x}=(x_1,x_2)\in U\). Write the stationarity condition for this problem we would check for \(\mathbf{x}^*\) found above in terms of only \(x_1\). Is there any possible \(\mathbf{x}\in U\) that does not satisfy the stationarity condition?

Worksheet 9-1: Q2#

Let’s extend the example above to the more general case. Consider the optimization problem

\[ \min_{\mathbf{x}} f(\mathbf{x}), \quad \text{s.t. } \mathbf{e}^{\top}\mathbf{x}=1, \text{ where } \mathbf{e}^T=\begin{bmatrix}1 & 1 & \ldots & 1\end{bmatrix}, \]

where \(f\) is a continuously differentiable function over \(\mathbb{R}^n\). The feasible set for the problem is

\[ U=\{\mathbf{x}\in\mathbb{R}^n:\mathbf{e}^{\top}\mathbf{x}=1\} =\left\{\mathbf{x}\in\mathbb{R}^n:\sum_{i=1}^n x_i=1\right\}. \]

We will show that the stationarity condition here, namely

\[ \nabla f(\mathbf{x}^*)^\top(\mathbf{x}-\mathbf{x}^*)\ge 0 \text{ for all } \mathbf{x} \text{ satisfying } \mathbf{e}^\top\mathbf{x}=1 \]

is satisfied when

\[ \frac{\partial f}{\partial x_1}(\mathbf{x}^*) =\frac{\partial f}{\partial x_2}(\mathbf{x}^*) =\cdots =\frac{\partial f}{\partial x_n}(\mathbf{x}^*). \]

  1. First, go back to the previous problem. Check that the solution you found for \(\mathbf{x}^*\) satisfies the second condition above.

  2. Now we will check that if the second condition above is true, then the stationarity condition above is true. Say that every entry in \(\nabla f(\mathbf{x}^*)\) is \(a\) (so this is all the things in the second condition above). Simplify \(\nabla f(\mathbf{x}^*)^\top(\mathbf{x}-\mathbf{x}^*)\) as much as possible.

  3. Why does the result above imply that \(\mathbf{x}^*\) is a stationary point?

Worksheet 9-1: Q3#

Consider the convex optimization problem

\[ \min_{x,y,z}\; 2x^2+3y^2+4z^2+2xy-2xz-8x-4y-2z, \quad \text{s.t. } x,y,z\ge 0. \]

(a) What is the gradient of \(f(\mathbf{x})=2x^2+3y^2+4z^2+2xy-2xz-8x-4y-2z\)?

(b) Fix \(\mathbf{x}^*=\left(\frac{17}{7},0,\frac{6}{7}\right)\). What is \(\nabla f(\mathbf{x}^*)\)? Is \(\mathbf{x}^*\) a stationary point of the function \(f\)?

(c) Show that the vector \(\left(\frac{17}{7},0,\frac{6}{7}\right)\) is a stationary point of the problem.

(d) Find the first iteration of the gradient projection method starting with \(\mathbf{x}_0=(1,1,1)\), and using a constant step size \(0.5\).

Contents