Worksheet 7-1: Convex Functions (with Solutions)

Download: CMSE382-WS7_1.pdf · Solution PDF

Warning

This is an AI-generated transcript of the worksheet and may contain errors or inaccuracies. Please refer to the original course materials for authoritative content.

---

Atom functions (assume convex unless you are proving directly from the definition):

\(f(t) = mt + b\), \(m,b \in \mathbb{R}\)	\(f(t) = t^2\)	\(f(t) = e^t\)
	\(f(t) = t^4\)	\(f(t) = -\ln(t)\), \(t > 0\)

---

Worksheet 7-1: Q1

Justify each step (i)–(vi) in the following proof that \(f(t) = t^2\) is convex. Pick any \(\lambda \in [0,1]\) and \(t_1, t_2 \in \mathbb{R}\). Then:

\[\begin{split} \begin{aligned} f(\lambda t_1+(1-\lambda)t_2) &= (\lambda t_1+(1-\lambda)t_2)^2 \qquad\textit{(i)}\\ &= \lambda^2 t_1^2 + 2\lambda(1-\lambda)t_1 t_2 + (1-\lambda)^2 t_2^2 \qquad\textit{(ii)}\\ &= \lambda t_1^2 - \lambda(1-\lambda)t_1^2 + 2\lambda(1-\lambda)t_1 t_2 + (1-\lambda)t_2^2 - \lambda(1-\lambda)t_2^2 \qquad\textit{(iii)}\\ &= \lambda t_1^2+(1-\lambda)t_2^2-\lambda(1-\lambda)(t_1-t_2)^2 \qquad\textit{(iv)}\\ &\le \lambda t_1^2+(1-\lambda)t_2^2 \qquad\textit{(v)}\\ &= \lambda f(t_1)+(1-\lambda)f(t_2). \qquad\textit{(vi)} \end{aligned} \end{split}\]

Solution

(i) Definition of \(f\).

(ii) Expand the square: \((a+b)^2 = a^2+2ab+b^2\) with \(a=\lambda t_1\) and \(b=(1-\lambda)t_2\).

(iii) Substitute \(\lambda^2 = \lambda-\lambda(1-\lambda)\) and \((1-\lambda)^2=(1-\lambda)-\lambda(1-\lambda)\) into step (ii). Both identities follow from factoring: \(\lambda^2+\lambda(1-\lambda)=\lambda(\lambda+(1-\lambda))=\lambda\).

(iv) Factor \(-\lambda(1-\lambda)\) out of the last three terms in step (iii), using \(t_1^2-2t_1t_2+t_2^2=(t_1-t_2)^2\).

(v) Since \(\lambda\in[0,1]\) we have \(\lambda(1-\lambda)\ge 0\), and \((t_1-t_2)^2\ge 0\), so \(\lambda(1-\lambda)(t_1-t_2)^2\ge 0\); subtracting a non-negative quantity can only decrease the value.

(vi) Definition of \(f\) (applied to \(t_1\) and \(t_2\)).

---

Worksheet 7-1: Q2

Show that each of the following functions is convex.

(a) \(f(x,y) = x^2 + 3y^2\)

Solution

• Let \(\phi(t) = t^2\), which is a convex atom.

• \(x^2 = \phi([1\ 0][x\ y]^\top)\) is convex by composition with an affine function.

• \(y^2 = \phi([0\ 1][x\ y]^\top)\) is convex for the same reason.

• \(3y^2\) is convex by non-negative scalar multiplication (\(3 \ge 0\)).

• \(f(x,y) = x^2 + 3y^2\) is convex by the sum rule.

(b) \(h(x,y) = \exp(x + y)\)

Solution

• Let \(f_1(x,y) = x+y = [1\ 1][x\ y]^\top\), which is affine (hence convex).

• Let \(g(t) = e^t\), which is a convex atom, non-decreasing on \(\mathbb{R}\).

• \(h(x,y) = g(f_1(x,y)) = \exp(x+y)\) is convex by the composition rule (non-decreasing convex \(\circ\) affine).

(c) \(f(x,y) = x^2 + 2xy + 3y^2 + 2x - 3y\)

Solution

• The gradient is \(\nabla f(x,y) = \begin{bmatrix}2x+2y+2 \\ 2x+6y-3\end{bmatrix}\).

• The Hessian is constant: \(\nabla^2 f = \begin{bmatrix}2 & 2 \\ 2 & 6\end{bmatrix}\).

• Eigenvalues: \(\lambda_{1,2} = 4 \pm 2\sqrt{2} > 0\).

• Since \(\nabla^2 f \succeq 0\) everywhere, \(f\) is convex.

(d) \(f(x,y,z) = \exp(x-y+z) + \exp(2y) + x\)

Solution

• \(f_1(x,y,z) = x-y+z\) and \(f_2(x,y,z) = 2y\) are both affine (convex).

• \(g(t) = e^t\) is a convex atom, non-decreasing.

• \(\exp(x-y+z) = g(f_1)\) and \(\exp(2y) = g(f_2)\) are convex by composition.

• \(x\) is affine (convex).

• Sum of convex functions is convex: \(f = \exp(x-y+z) + \exp(2y) + x\) is convex by the sum rule.

(e) \(f(x,y) = -\ln(xy)\), for \(x > 0\), \(y > 0\)

Solution

• Rewrite: \(f(x,y) = -\ln(xy) = -\ln x - \ln y\).

• Let \(\phi(t) = -\ln t\) on \((0,\infty)\), a convex atom.

• \(\phi(x)\) and \(\phi(y)\) are each convex by composition with affine maps (identity).

• \(f(x,y) = \phi(x) + \phi(y)\) is convex by the sum rule.

---

Worksheet 7-1: Q3

Are the following functions convex?

(a) \(f(x,y) = xy\)

Solution

Not convex.

The Hessian is \(\nabla^2 f(x,y) = \begin{bmatrix}0 & 1 \\ 1 & 0\end{bmatrix}\), with eigenvalues \(\lambda_1 = 1\) and \(\lambda_2 = -1\).

The Hessian is indefinite (has a negative eigenvalue), so \(f\) is not convex. Note that the Hessian contains no variables, so it remains indefinite regardless of where we evaluate it — \(f(x,y) = xy\) is not convex on any domain.

(b) \(\|\mathbf{x}\|\) for any norm on \(\mathbb{R}^n\)

Solution

Convex. Use the definition directly.

Pick any \(\lambda \in [0,1]\) and \(\mathbf{x}, \mathbf{y} \in \mathbb{R}^n\):

\[\|\lambda\mathbf{x} + (1-\lambda)\mathbf{y}\| \le \|\lambda\mathbf{x}\| + \|(1-\lambda)\mathbf{y}\| \qquad \text{(triangle inequality)}\]

\[= \lambda\|\mathbf{x}\| + (1-\lambda)\|\mathbf{y}\| \qquad \text{(homogeneity of norms)}\]

So \(\|\lambda\mathbf{x}+(1-\lambda)\mathbf{y}\| \le \lambda\|\mathbf{x}\|+(1-\lambda)\|\mathbf{y}\|\) for all \(\lambda\in[0,1]\) and \(\mathbf{x},\mathbf{y}\in\mathbb{R}^n\). \(\checkmark\)

(c) \(h(\mathbf{x}) = \exp(\|\mathbf{x}\|)\)

Solution

Convex.

• \(\|\mathbf{x}\|\) is convex (shown in (b)).

• \(g(t) = e^t\) is a convex atom and non-decreasing on \(\mathbb{R}\).

• \(h(\mathbf{x}) = g(\|\mathbf{x}\|)\) is convex by the composition rule (non-decreasing convex \(\circ\) convex).

(d) \(\|\mathbf{x}\|^2\) for any norm on \(\mathbb{R}^n\)

Solution

Convex.

• \(\|\mathbf{x}\|\) is convex and takes values \(\ge 0\) (shown in (b)).

• \(g(t) = t^2\) is a convex atom and non-decreasing on \([0, \infty)\).

• \(\|\mathbf{x}\|^2 = g(\|\mathbf{x}\|)\) is convex by the composition rule (non-decreasing convex \(\circ\) convex, with range in \([0,\infty)\)).