Worksheet 4-3: Lipschitz Gradient

Download: CMSE382-WS4_3.pdf

Warning

This is an AI-generated transcript of the worksheet and may contain errors or inaccuracies. Please refer to the original course materials for authoritative content.

---

Useful facts. Let \(f: \mathbb{R}^n \to \mathbb{R}\) be \(C^2\).

• If \(f \in C^{1,1}_L(\mathbb{R}^n)\) (i.e., \(\nabla f\) is \(L\)-Lipschitz), then \(\|\nabla^2 f(\mathbf{x})\| \le L\) for all \(\mathbf{x}\).

• A once-differentiable function is Lipschitz on \(\Omega\) if its gradient is bounded on \(\Omega\).

• For one-dimensional functions, \(\nabla f\) is Lipschitz on \(\Omega\) iff \(f''\) is bounded on \(\Omega\).

---

Worksheet 4-3: Q1

For each of the following functions, determine:

• Is the function Lipschitz on the domain given?

• Is its gradient/derivative Lipschitz? (i.e., does the function belong to \(C^{1,1}\)?)

Function	Domain	Lipschitz?	\(\nabla f\) Lipschitz?
\(f(x) = mx + b\)	\(\mathbb{R}\)
\(f(x) = \sqrt{x}\)	\([0, \infty)\)
\(f(x) = x^2\)	\(\mathbb{R}\)
\(f(x) = \sin(x)\)	\(\mathbb{R}\)
\(f(x) = e^{-x}\)	\([0, \infty)\)
\(f(x,y) = 2\sin(x) - 10.9y^2 + \pi e^{-(x^2+y^2)}\)	\(\mathbb{R}^2\)

---

Worksheet 4-3: Q2

Consider the function \(f(x,y) = x^2 + y^4\) and starting point \(\mathbf{x}_0 = (1, 1)^\top\).

1. Perform 3 iterations of gradient descent with step size \(t = 1/2\).

   (a) \(\mathbf{x}_1 = ?\)

   (b) \(\mathbf{x}_2 = ?\)

   (c) \(\mathbf{x}_3 = ?\)

2. Comment on the convergence behavior observed in the \(x\)- and \(y\)-components. Are they converging at the same rate?

3. Is the \(C^{1,1}\) assumption satisfied for \(f(x,y) = x^2 + y^4\)? Explain.