Lecture 7-3: Convex Functions: Part 3#
Download the original slides: CMSE382-Lec7_3.pdf
Warning
This is an AI-generated transcript of the lecture slides and may contain errors or inaccuracies. Please refer to the original course materials for authoritative content.
This Lecture#
Topics:
Continuity and differentiability of convex functions
Extended real-valued functions
Maxima of a convex function
Announcements:
Homework 3 due Friday!
The homework uses CVXPY. If you missed last class, make sure you get your CVXPY installation working ASAP!
Continuity and differentiability of convex functions#
Are convex functions always continuous?#
Nope.
Definition: A function \(f: C \to \mathbb{R}\) defined over a convex set \(C \subseteq \mathbb{R}^n\) is convex if for any \(\mathbf{x}, \mathbf{y} \in C\) and \(\lambda \in [0,1]\), we have $\( f(\lambda \mathbf{x} + (1-\lambda) \mathbf{y}) \leq \lambda f(\mathbf{x}) + (1-\lambda) f(\mathbf{y}). \)$
Lipschitz continuity#
Definition (Lipschitz Continuity): A function \(f\colon S \to \mathbb{R}\) where \(S \subseteq \mathbb{R}^n\) is Lipschitz continuous if there exists an \(L > 0\) such that $\( \|f(\mathbf{x}) - f(\mathbf{y})\| \leq L\|\mathbf{x}-\mathbf{y}\|, \quad \forall \mathbf{x}, \mathbf{y} \in S. \)$
Note that \(f\) isn’t assumed to be continuous.
Lipschitz continuity implies continuity, but the converse is not necessarily true.
Local Lipschitz Continuity#
Definition (Local Lipschitz Continuity): A function \(f\colon S \to \mathbb{R}\) where \(S \subseteq \mathbb{R}^n\) is locally Lipschitz continuous if for every \(\mathbf{x}_0 \in S\) there exist \(\varepsilon > 0\) and \(L > 0\) such that \(B(\mathbf{x}_0,\varepsilon) \subseteq S\) and $\( |f(\mathbf{x}) - f(\mathbf{x}_0)| \le L\|\mathbf{x} - \mathbf{x}_0\|, \quad \forall \mathbf{x} \in B(\mathbf{x}_0,\varepsilon). \)$
Continuity of convex functions#
Are convex functions always continuous?
Theorem (Local Lipschitz continuity of convex functions): Let \(f \colon C \to \mathbb{R}\) be a convex function defined over a convex set \(C \subseteq \mathbb{R}^n\). Then \(f\) is locally Lipschitz continuous at every \(\mathbf{x}_0 \in \text{int}(C)\).
Convex functions on \(C \subseteq \mathbb{R}^n\) are continuous on \(\text{int}(C)\).
Convex functions on \(\mathbb{R}^n\) are continous on \(\mathbb{R}^n\).
Directional derivatives for convex functions#
Definition (Recall): The directional derivative of \(f:\mathbb{R}^n\to \mathbb{R}\) at \(\mathbf{x}\) along the direction \(\mathbf{d}\) is defined as $\( f'(\mathbf{x};\mathbf{d})=\nabla f(\mathbf{x})^\top\mathbf{d}. \)$
Theorem (Existence of directional derivatives for convex functions): Let \(f: C \to \mathbb{R}\) be a convex function defined over the convex set \(C\subseteq \mathbb{R}^n\). Let \(\mathbf{x} \in \text{int}(C)\). Then for any \(\mathbf{d} \neq \mathbf{0}\), the directional derivative \(f'(\mathbf{x};\mathbf{d})\) exists.
Extended real-valued functions#
Extended real-valued functions#
Definition (Extended real-valued functions):
Real-valued functions take their values in \(\mathbb{R} = (-\infty, \infty)\).
Extended real-valued functions take their values in \(\mathbb{R} \cup \{\infty\} = (-\infty, \infty]\)
Warning: Some sources define the extended real numbers as \(\mathbb{R} \cup \{\pm\infty\} = [-\infty, \infty]\).
Example: Indicator function $\( \delta_S(\mathbf{x}) = \begin{cases} 0, & \text{ if } \mathbf{x} \in S\\ \infty, & \text{ if } \mathbf{x} \notin S \end{cases} \)$
Arithmetic with \(\infty\):
\(a + \infty = \infty\) for any \(a\in \mathbb{R}\).
\(a \cdot \infty = \infty\) for any \(a \in \mathbb{R}_{++}\).
\(0 \cdot \infty = 0\).
Effective domain of extended real-valued functions#
Definition: The effective domain of an extended real-valued function is the set of vectors for which the function takes a real value: $\( \text{dom}(f ) = \{\mathbf{x} \in \mathbb{R}^n \mid f(\mathbf{x}) < \infty \}. \)$
Convexity of extended real-valued functions#
Theorem: An extended real-valued function \(f\) is convex if and only if \(\text{dom}(f)\) is a convex set and \(f\) is convex over \(\text{dom}(f)\).
Epigraph of a function#
Definition (Epigraph of a function): Let \(f:\mathbb{R}^n \to \mathbb{R} \cup \{\infty\}\). The epigraph set \(\text{epi}(f ) \subseteq \mathbb{R}^{n+1}\) is defined by $\( ext{epi}(f)=\bigg\{\begin{pmatrix}\mathbf{x}\\t \end{pmatrix}:f(\mathbf{x}) \leq t \bigg\}. \)$
It contains all the points \(\begin{pmatrix}\mathbf{x}\\t \end{pmatrix}\) on or above the function graph.
Theorem (Function and epigraph convexity): An extended real-valued (or a real-valued) function \(f\) is convex if and only if its epigraph set \(\text{epi}(f)\) is convex.
Preservation of convexity under maximum for extended real-valued functions#
Theorem: Let \(f_i:\mathbb{R}^n \to \mathbb{R} \cup \{\infty\}\) be an extended real-valued convex function for any \(i \in I\) (I being an arbitrary index set). Then the function $\( f(\mathbf{x}) = \max_{i\in I} f_i(\mathbf{x}) \)$ is an extended real-valued convex function.
Maxima of a convex function#
Recall: Extreme points#
Definition (Recall: Extreme points): Let \(S\) be a convex set. A point \(\textbf{x} \in S\) is an extreme point of \(S\) if there do not exist two distinct points \(\textbf{x}_1, \textbf{x}_2 \in S\) and \(\lambda \in (0,1)\) such that \(\textbf{x} = \lambda\textbf{x}_1+(1-\lambda)\textbf{x}_2\).
It is a point in \(S\) that cannot be represented as a nontrivial convex combination of two different points in \(S\).
The set of all extreme points is denoted \(\mbox{ext}(S)\).
Maxima of convex functions#
Theorem: Let \(f: C \to \mathbb{R}\) be a convex function which is not constant over the convex set \(C\). Then \(f\) does not attain a maximum at a point in \(\text{int}(C)\).
Maximum of non-constant convex function defined on a convex set cannot occur at an interior point in the set.
Maxima of convex functions#
Theorem: Let \(f: C \to \mathbb{R}\) be a convex and continuous function over the convex and compact set \(C \subseteq \mathbb{R}^n\). Then there exists at least one maximizer of \(f\) over \(C\) that is an extreme point of \(C\).
Groups - Round 3#
Group 1 Lowell, Tianjuan, Lauryn, Atticus
Group 2 Alice, Aidan, Dev, Anthony
Group 3 Abigail, Michal, Breena, Andrew
Group 4 Kyle, Vinod, Dori, Joseph
Group 5 Yousif, Jamie, Jay, K.M Tausif
Group 6 Shanze, Saitej, Karen, Jack
Group 7 Arjun, Noah, Luis, Arya
Group 8 Morgan, Jonid, Sanskaar, Jake
Group 9 Quang Minh, Monirul Amin, Daniel, Ha
Group 10 Braedon, Dominic, Zheng, Lora
Group 11 Sai, Brandon, Purvi, Aaron
Group 12 Igor, Scott, Maye, Long