# Worksheet 4-3: Lipschitz Gradient

Download: [CMSE382-WS4_3.pdf](CMSE382-WS4_3.pdf)

```{warning}
This is an AI-generated transcript of the worksheet and may contain errors or inaccuracies. Please refer to the original course materials for authoritative content.
```

---

**Useful facts.** Let $f: \mathbb{R}^n \to \mathbb{R}$ be $C^2$.

- If $f \in C^{1,1}_L(\mathbb{R}^n)$ (i.e., $\nabla f$ is $L$-Lipschitz), then $\|\nabla^2 f(\mathbf{x})\| \le L$ for all $\mathbf{x}$.
- A once-differentiable function is Lipschitz on $\Omega$ if its gradient is bounded on $\Omega$.
- For one-dimensional functions, $\nabla f$ is Lipschitz on $\Omega$ iff $f''$ is bounded on $\Omega$.

---

## Worksheet 4-3: Q1

For each of the following functions, determine:
- Is the function Lipschitz on the domain given?
- Is its gradient/derivative Lipschitz? (i.e., does the function belong to $C^{1,1}$?)

| Function | Domain | Lipschitz? | $\nabla f$ Lipschitz? |
|---|---|---|---|
| $f(x) = mx + b$ | $\mathbb{R}$ | | |
| $f(x) = \sqrt{x}$ | $[0, \infty)$ | | |
| $f(x) = x^2$ | $\mathbb{R}$ | | |
| $f(x) = \sin(x)$ | $\mathbb{R}$ | | |
| $f(x) = e^{-x}$ | $[0, \infty)$ | | |
| $f(x,y) = 2\sin(x) - 10.9y^2 + \pi e^{-(x^2+y^2)}$ | $\mathbb{R}^2$ | | |

---

## Worksheet 4-3: Q2

Consider the function $f(x,y) = x^2 + y^4$ and starting point $\mathbf{x}_0 = (1, 1)^\top$.

1. Perform 3 iterations of gradient descent with step size $t = 1/2$.

   (a) $\mathbf{x}_1 = ?$

   (b) $\mathbf{x}_2 = ?$

   (c) $\mathbf{x}_3 = ?$

2. Comment on the convergence behavior observed in the $x$- and $y$-components. Are they converging at the same rate?

3. Is the $C^{1,1}$ assumption satisfied for $f(x,y) = x^2 + y^4$? Explain.