27  Glossary

Short, precise definitions for terms used across chapters. Each chapter explains its own core concepts in depth — this page is for the term you half remember from two chapters ago, not a substitute for reading the chapter. Sorted alphabetically for lookup; see Concepts by Type for the same terms grouped by object, structure, theory, or application instead.

Abelian group — a group whose operation is commutative (\(a \ast b = b \ast a\) for all elements). Not required by the group axioms themselves. See Groups.

Affine combination — a weighted sum of points whose weights sum to exactly \(1\); unlike an arbitrary sum of points, this is always well-defined. See Affine Spaces.

Affine map — a linear map plus a translation; generalizes a linear map to a setting with no origin to fix. See Affine Spaces.

Affine space — a set of points, a vector space of displacements, and an action translating points by displacements, such that any two points determine a unique displacement. Formalizes “points don’t add, but their difference is a vector.” See Affine Spaces.

Axiom — a starting statement accepted without proof, from which theorems are derived. Not “true” in isolation — true relative to the choice of axioms. See Axioms and Definitions.

Backpropagation — the chain rule, applied systematically back-to-front through a composition of maps to compute a neural network’s gradient efficiently; not a distinct piece of mathematics from differentiation. See Machine Learning.

Basis — a linearly independent set of vectors that spans a vector space; every vector is then a unique linear combination of basis vectors. See Vector Spaces.

Bayes’ rule — \(P(A\mid B) = \frac{P(B\mid A)P(A)}{P(B)}\); how probability updates belief given new information. See Probability.

Bijective — both injective and surjective; pairs two sets up perfectly, element for element. See Relations and Functions.

Category — a collection of objects and structure-preserving morphisms between them satisfying composition and identity axioms; the maximal abstraction of the homomorphism pattern this site uses repeatedly. See Advanced Topics.

Combinatorics — counting the elements of a set built from others via subsets, functions, or orderings; \(\binom{n}{r}\) counts \(r\)-element subsets, \(n!\) counts bijections of a set with itself. See Discrete Mathematics.

Completeness — a metric space’s property that every Cauchy sequence (one whose terms bunch arbitrarily close together) actually converges to a point in the space. \(\mathbb{R}\) has it; \(\mathbb{Q}\) doesn’t. See Metric Spaces.

Continuous — classically, the \(\varepsilon\)–\(\delta\) definition; topologically, a function whose preimage of every open set is open — the same idea, restated without needing a distance. See Topological Spaces.

Convex — a set containing the line segment between any two of its points; a function is convex when the region above its graph is. Every local minimum of a convex function is a global minimum. See Optimization.

Critical point — a point where a differentiable function’s derivative is zero; a candidate maximum, minimum, or saddle point, decided by the Hessian. See Optimization.

Derivative — \(f'(a) = \lim_{h\to 0} \frac{f(a+h)-f(a)}{h}\); structurally, the linear map that best approximates \(f\) near \(a\). See Differentiation.

Differential equation — an equation relating a function to one or more of its own derivatives; a solution is a function, not a number. See Differential Equations.

Dimension — the number of vectors in a basis of a vector space; the same for every basis of a given space. See Vector Spaces.

Discrete — of objects with definite next/previous neighbors and nothing between (contrast: dense, as \(\mathbb{Q}\) and \(\mathbb{R}\) are). See Discrete Mathematics.

Dynamical system — a differential equation studied for its long-run qualitative behavior (fixed points, stability, oscillation) rather than an explicit formula. See Differential Equations.

Eigenvalue / eigenvector — a nonzero vector \(v\) that a linear map \(T\) only stretches or shrinks (\(T(v) = \lambda v\)), never redirects, and the scalar \(\lambda\) by which it does so. See Linear Algebra.

Embedding — a learned vector representation of data (a word, an image, an item) — an element of a vector space, not a metaphor. See Machine Learning.

Entropy — \(H(X) = -\mathbb{E}[\log P(X)]\); the average “surprise” a distribution’s outcomes carry, an ordinary expectation applied to \(-\log P(X)\). See Machine Learning.

Equivalence relation — a relation that is reflexive, symmetric, and transitive; partitions a set into equivalence classes. Used to construct \(\mathbb{Z}\) and \(\mathbb{Q}\) from more primitive number systems. See Relations and Functions and Number Systems.

Euclidean space — an affine space whose vector space of displacements carries an inner product; the modern, coordinate-based redefinition of the geometry Euclid built from postulates. See Euclidean Geometry.

Expectation — \(\mathbb{E}[X] = \int_\Omega X\,dP\); a random variable’s average value, literally an integral against the probability measure. See Probability.

Field — a ring in which multiplication is commutative and every nonzero element has a multiplicative inverse; the scalars a vector space is defined “over.” \(\mathbb{Q}\), \(\mathbb{R}\), \(\mathbb{C}\) are fields, \(\mathbb{Z}\) is not. See Rings and Fields.

Function — a relation \(f \subseteq A \times B\) in which every element of \(A\) appears as the first coordinate of exactly one pair. See Relations and Functions.

Functor — a structure-preserving map between categories; a homomorphism of homomorphisms. See Advanced Topics.

Fundamental theorem of calculus — differentiation and integration are inverse operations: \(\int_a^b f(x)\,dx = F(b)-F(a)\) for any antiderivative \(F\) of \(f\). See Integration.

Gradient descent — repeatedly stepping opposite a function’s gradient to minimize it; a discretized gradient flow, in the same sense Euler’s method discretizes any differential equation. See Optimization.

Group — a set with one operation satisfying closure, associativity, identity, and inverses. See Groups.

Hilbert space — a complete inner-product space, typically infinite-dimensional; the setting Fourier series and quantum-mechanical states actually live in. See Advanced Topics.

Homomorphism — a map between two structures of the same kind that respects their operations; need not be invertible. See What Is a Mathematical Structure?.

Independence — events \(A\) and \(B\) with \(P(A\cap B) = P(A)P(B)\); a genuinely probabilistic notion with no counterpart in general measure theory. See Probability.

Injective — a function under which distinct inputs always give distinct outputs (“one-to-one”). See Relations and Functions.

Inner product — a symmetric, linear, positive-definite function \(\langle u,v\rangle\) on a vector space; induces a norm (\(\|v\| = \sqrt{\langle v,v\rangle}\)) and an angle between vectors. See Inner Product Spaces.

Isomorphism — an invertible homomorphism; identifies two structures as “the same” up to relabeling their elements. See What Is a Mathematical Structure?.

KL divergence — \(D_{KL}(P\|Q) = \mathbb{E}_P[\log\frac{P}{Q}]\); how different two distributions are. Cross-entropy loss decomposes into an entropy term plus this, which is why minimizing it fits a model’s predicted distribution to the true one. See Machine Learning.

Limit — \(\lim_{x\to a} f(x) = L\): for every \(\varepsilon>0\) there is a \(\delta>0\) such that \(0<|x-a|<\delta\) implies \(|f(x)-L|<\varepsilon\) — the metric-space definition of convergence, specialized to \(\mathbb{R}\). See Limits and Continuity.

Linear combination — \(c_1 v_1 + \cdots + c_k v_k\), for scalars \(c_i\) and vectors \(v_i\). See Vector Spaces.

Linear map — a function \(T: V \to W\) between vector spaces satisfying \(T(u+v) = T(u)+T(v)\) and \(T(cv) = cT(v)\). A matrix is what a linear map looks like once bases are fixed. See Linear Algebra.

Manifold — informally, a topological space that locally resembles \(\mathbb{R}^n\) near every point, even if it doesn’t globally (a sphere’s surface is the standard example). A first pass only — see Euclidean Geometry for the sketch and Advanced Topics for the preview of making it rigorous.

Manifold hypothesis — the idea that real high-dimensional data concentrates near a much lower-dimensional manifold embedded in that space; what dimensionality reduction looks for. See Machine Learning.

Mathematical induction — a proof technique for statements about all natural numbers: prove a base case, prove \(n\) implies \(n+1\), conclude the statement holds for every \(n\). See Discrete Mathematics.

Mathematical structure — a set equipped with operations and/or relations satisfying a fixed list of axioms — e.g. group, vector space, metric space. See What Is a Mathematical Structure?.

Metric / metric space — a function \(d(x,y)\) measuring distance between points of a set, satisfying identity of indiscernibles, symmetry, and the triangle inequality; the set plus that function is a metric space. See Metric Spaces.

Mutual information — \(I(X;Y) = D_{KL}(P(X,Y)\,\|\,P(X)P(Y))\); how far two random variables are from independence, zero exactly when they’re independent. See Machine Learning.

Norm — a length function \(\|v\|\) on a vector space; every inner product induces one, but not every norm comes from an inner product (test: the parallelogram law). See Inner Product Spaces.

Open set — in a metric space, a set where every point has some room around it (an open ball) still inside the set; in a topological space, a member of the designated collection \(\tau\) that defines the topology. See Metric Spaces and Topological Spaces.

Ordinary / partial differential equation (ODE / PDE) — an ODE involves derivatives with respect to one variable; a PDE involves partial derivatives with respect to several. See Differential Equations.

Orthogonal — two vectors \(u,v\) with \(\langle u,v\rangle = 0\); the structural definition behind “perpendicular.” See Inner Product Spaces.

Probability space — a triple \((\Omega, \mathcal{F}, P)\): outcomes, events, and a probability measure satisfying \(P(\Omega)=1\) and countable additivity. See Probability.

Proof — a finite chain of statements, each an axiom or a valid consequence of earlier statements, ending at the claim being established. See Logic and Proof.

Proposition — a statement that is unambiguously true or false. See Logic and Proof.

Random variable — a function \(X:\Omega\to\mathbb{R}\) from a probability space’s outcomes to numbers — not “a variable that is random.” See Probability.

Real analysis — the rigorous \(\varepsilon\)–\(\delta\) foundation for calculus, built roughly two centuries after calculus itself. See Limits and Continuity.

Relation — a subset of a Cartesian product \(A \times B\); the general notion underlying equality, order, divisibility, and functions alike. See Relations and Functions.

Riemann sum — an approximation to \(\int_a^b f(x)\,dx\) built from rectangles over a partition of \([a,b]\); the integral is the limit as the partition gets arbitrarily fine. See Integration.

Ring — a set with two operations, \(+\) and \(\times\), where \((R,+)\) is an abelian group, \(\times\) is associative, and \(\times\) distributes over \(+\). \(\mathbb{Z}\) and the \(n \times n\) matrices are rings that aren’t fields. See Rings and Fields.

Set — a collection of distinct objects, unordered, without repetition. See Sets.

Span — the set of all linear combinations of a given collection of vectors. See Vector Spaces.

Surjective — a function whose range is its entire codomain (“onto”). See Relations and Functions.

Theorem — a statement derivable from a theory’s axioms by valid inference. See Axioms and Definitions.

Topological space — a set with a designated collection of subsets (“open sets”) closed under arbitrary unions and finite intersections, including the whole set and the empty set. Every metric space is one; not every topological space comes from a metric. See Topological Spaces.

Vector space — a set with addition and scalar multiplication satisfying eight axioms; instances include \(\mathbb{R}^n\), polynomials, matrices, and function spaces. See Vector Spaces.

Zero divisor — a nonzero element \(a\) of a ring for which some other nonzero \(b\) satisfies \(ab=0\). Can’t happen in a field; happens routinely among matrices. See Rings and Fields.