Chapter One - Basics

§1. Basics of Topology on $\mathbb{R}^n$

Metric and Metric Space

A metric $d: X \times X \to \mathbb{R}$ is a binary function that measures the "distance" between two points in $X$, where $X$ is a subset of $\mathbb{R}^n$. Metric functions should have the following properties:

1. Symmetry: $d(x, y) = d(y, x)$
2. Non-Negativity: $d(x, y) \ge 0$, and $d(x, y) = 0$ iff $x = y$
3. Triangle Inequality: $d(x, z) \le d(x, y) + d(y, z)$

A non-empty set $X$ constitutes a metric space when it is equipped with a metric function. More formally, a metric space can be denoted as $(X, d)$ where set $X$ is not empty and $d$ is a metric of $X$. Metric of a metric space is usually omitted for simplification. For example, $(X, d)$ can be expressed as a single $X$ when it is clear that $X$ has a metric $d$.

If $X$ is a metric space and $Y$ is a subset of $X$, then the metric of $Y$ is a narrowed version of the metric of $X$. The difference between these two metrics lies in their domains of definition.

A metric space can have different metrics. For example, $\mathbb{R}^n$ has the following three common metrics ($x, y \in \mathbb{R}^n$):

1. Euclidean Metric: Also known as the 2-norm, it's defined by $d_2(x, y) = \sqrt{\sum_{i = 1}^n (x_i - y_i)^2}$.
2. Manhattan Metric: Also known as the 1-norm or taxicab metric, it's defined by $d_1(x, y) = \sum_{i = 1}^n \big|x_i - y_i\big|$.
3. Infinity Metric: It's defined by the largest component of the difference of two elements, i.e. $d_\infty(x, y) = \max_{i = 1, 2, \dots, n} (x_i - y_i)$.

Subsets and Points

Suppose $X$ is a metric space, then there can be some special sets within $X$. For example, the neighborhood of point $x_0 \in X$ can be defined as $U(x_0, \epsilon) = \{x \in X \big| d(x, x_0) \lt \epsilon\}$ for some $\epsilon \ge 0$. An open set $U \subseteq X$ is such a set that every element $x_0 \in U$ has a neighborhood $U(x_0, \epsilon)$, and a closed set $C \subseteq X$ is such a set whose complementary set $X - C$ is an open set.

Still, let $X$ be a metric space and $S$ be a subset of $X$, a point $x \in X$ is a limit point of $S$ if every open neighborhood of $x$ contains at least one point in $S$. A limit point of $S$ can be either an interior point of $S$ or a boundary point of $S$. The following figure shows the relationship between these points.

Relationship Between Limit Points, Interior Points and Boundary Points

A closure of $X$ is a set that contains every point in $X$ and every limit point in $X$. Denoted as $\bar X$, it is not simply a set of limit points of $X$ since isolated points are regarded as elements in a set while they are not limit points.

Continuity

When we say a function is continuous, we are referring to the feature that the value of this function does not suddenly jump elsewhere as a tiny disturbance is placed on the input. That is to say, when the input changes slightly, the function value also changes as slightly as the input. So consider the following conditions:

1. When the function value $f(x)$ is changing most slightly, the input $x$ also changes slightly.

2. When the function value $f(x)$ is changing most slightly, the input $x$ changes but within a certain range near the original value.

Consider that the function value might remain the same for different inputs, even when these inputs are so "far away" from each other. We only need one neighborhood of the original value of input. If when the input is moving within this however small (and however big) neighborhood, the function value changes slightly, then we can say that this function is continuous at the original value. As for the neighborhood near the input, we demand it to be of whatever size; While for the range of varying function value, we demand it to be however small. Such an idea can be depicted by the following condition.

For all $\epsilon \gt 0$, there exists $\delta \gt 0$, such that if $d_Y(f(x_0), f(x)) \lt \epsilon$, then $d_X(x_0, x) \lt \delta$.

And this is the $\epsilon$-$\delta$ definition for function continuity at point $x_0 \in X$. A function is continuous if and only if it is continuous at every point in its definition domain. A function $f = f(f_1, f_2, \dots, f_n)$ is continuous iff every component $f_i$ is continuous.

Limits

Let $X$ be a metric space, where there's a subset $A$ and function $f: A \to Y$. Let $x_0$ be a limiting point in $X$ (That is, either an interior point or a boundary point of $X$. $x_0$ can be either an element in $X$ or not as well). Function $f$ approaches $y_0$ when $x$ is approaching $x_0$ if for any open set $Y$ that contains $y_0$, there exists an open set $U$ that contains $x_0$ such that $f(x_0)$, where $x \in U \cap X $ and $x \neq x_0$. This can be denoted as $$ \lim_{x \to x_0} f(x) = y_0 $$

Another way to define limits is through metrics. $f(x)$ approaches $y_0$ when $x$ is approaching $x_0$, iff for any $\epsilon \gt 0$, there exists a $\delta \gt 0$ such that when $x \in A$ and $0 \lt d_X(x, x_0) \lt \delta$, $d_Y(f(x), y_0) \lt \epsilon$.

Let $f: X \to Y$. If $x_0$ is an isolated point of $X$, then $f$ is continuous at $x_0$; Otherwise, $f$ is continuous at $x_0$ iff when $x \to x_0$, $f(x) \to f(x_0)$. This theorem reveals the relationship between continuity and limits.

§2. Derivatives

For a function $f$ that maps from $\mathbb{R}^n $ to $\mathbb{R}^m$, we can define a derivative for it.

First, consider the most basic derivatives in monadic calculus. A function has a derivative if the following limit exists: $$ \frac{\mathrm{d}f(x)}{\mathrm{d}x} = \lim_{\Delta x \to 0} \frac{f(x + \Delta x) - f(x)}{\Delta x} $$ Apparently it doesn't work for multi-variable functions such as $f: \mathbb{R}^n \to \mathbb{R}^m$ itself, since it's meaning less to make numbers in vector spaces of different dimenssions be divided by each other. So first let's make our perturbation, i.e. $\Delta x$, a scaler $t \in \mathbb{R}$, and introduce a directional derivative $\boldsymbol{u} \in \mathbb{R}^n$ to make up the major part of perturbation. $$ \frac{\mathrm{d}f}{\mathrm{d}\boldsymbol{x}} = \lim_{\Delta x \to 0} \frac{f(\boldsymbol{x} + t \cdot \boldsymbol{u}) - f(\boldsymbol{x})}{t} \in \mathbb{R}^m$$