Notation
This book uses the following notational conventions. Some entries describe a class of symbols, while others define one particular symbol. The indefinite article “a” marks a convention: for example, “\(x\): a scalar” means that lowercase letters generally denote scalars. The definite article marks a fixed meaning: “\(\mathbb{Z}\): the set of integers” refers specifically to \(\mathbb{Z}\).
Numerical Objects
- \(x\): a scalar
- \(\mathbf{x}\): a vector
- \(\mathbf{X}\): a matrix
- \(\mathsf{X}\): a general tensor
- \(\mathbf{I}\): the identity matrix (of some given dimension), i.e., a square matrix with \(1\) on all diagonal entries and \(0\) on all off-diagonals
- \(x_i\), \([\mathbf{x}]_i\): the \(i^\textrm{th}\) element of vector \(\mathbf{x}\)
- \(x_{ij}\), \(x_{i,j}\), \([\mathbf{X}]_{ij}\), \([\mathbf{X}]_{i,j}\): the element of matrix \(\mathbf{X}\) in row \(i\) and column \(j\)
Set Theory
- \(\mathcal{X}\): a set
- \(\mathbb{Z}\): the set of integers
- \(\mathbb{Z}^+\): the set of positive integers
- \(\mathbb{R}\): the set of real numbers
- \(\mathbb{R}^n\): the set of \(n\)-dimensional vectors of real numbers
- \(\mathbb{R}^{a\times b}\): the set of real matrices with \(a\) rows and \(b\) columns
- \(|\mathcal{X}|\): cardinality (number of elements) of set \(\mathcal{X}\)
- \(\mathcal{A}\cup\mathcal{B}\): union of sets \(\mathcal{A}\) and \(\mathcal{B}\)
- \(\mathcal{A}\cap\mathcal{B}\): intersection of sets \(\mathcal{A}\) and \(\mathcal{B}\)
- \(\mathcal{A}\setminus\mathcal{B}\): the elements of \(\mathcal{A}\) that do not belong to \(\mathcal{B}\)
Functions and Operators
- \(f(\cdot)\): a function
- \(\log(\cdot)\): the natural logarithm (base \(e\))
- \(\log_2(\cdot)\): logarithm to base \(2\)
- \(\exp(\cdot)\): the exponential function
- \(\mathbf{1}(\cdot)\): the indicator function, equal to \(1\) when its Boolean argument is true and \(0\) otherwise
- \(\mathbf{1}_{\mathcal{X}}(z)\): the set-membership indicator, equal to \(1\) when \(z\in\mathcal{X}\) and \(0\) otherwise
- \(\mathbf{(\cdot)}^\top\): transpose of a vector or a matrix
- \(\mathbf{X}^{-1}\): inverse of matrix \(\mathbf{X}\)
- \(\odot\): Hadamard (elementwise) product
- \([\cdot, \cdot]\): concatenation
- \(\|\cdot\|_p\): \(\ell_p\) norm
- \(\|\cdot\|\): \(\ell_2\) norm
- \(\langle \mathbf{x}, \mathbf{y} \rangle\): inner (dot) product of vectors \(\mathbf{x}\) and \(\mathbf{y}\)
- \(\sum\): summation over a collection of elements
- \(\prod\): product over a collection of elements
- \(\stackrel{\textrm{def}}{=}\): an equality asserted as a definition of the symbol on the left-hand side
Calculus
- \(\frac{dy}{dx}\): derivative of \(y\) with respect to \(x\)
- \(\frac{\partial y}{\partial x}\): partial derivative of \(y\) with respect to \(x\)
- \(\nabla_{\mathbf{x}} y\): gradient of \(y\) with respect to \(\mathbf{x}\)
- \(\int_a^b f(x) \;dx\): definite integral of \(f\) from \(a\) to \(b\) with respect to \(x\)
- \(\int f(x) \;dx\): indefinite integral of \(f\) with respect to \(x\)
Probability and Information Theory
- \(X\): a random variable
- \(P\): a probability distribution
- \(X \sim P\): the random variable \(X\) follows distribution \(P\)
- \(P(X=x)\): the probability that random variable \(X\) takes the value \(x\)
- \(P(X \mid Y)\): the conditional probability distribution of \(X\) given \(Y\)
- \(p(\cdot)\): a probability density function (PDF) associated with distribution \(P\)
- \({E}[X]\): expectation of a random variable \(X\)
- \(X \perp Y\): random variables \(X\) and \(Y\) are independent
- \(X \perp Y \mid Z\): random variables \(X\) and \(Y\) are conditionally independent given \(Z\)
- \(\sigma_X\): standard deviation of random variable \(X\)
- \(\textrm{Var}(X)\): variance of random variable \(X\), equal to \(\sigma^2_X\)
- \(\textrm{Cov}(X, Y)\): covariance of random variables \(X\) and \(Y\)
- \(\rho(X, Y)\): the Pearson correlation coefficient, \(\frac{\textrm{Cov}(X, Y)}{\sigma_X \sigma_Y}\)
- \(H(X)\): entropy of random variable \(X\)
- \(D_{\textrm{KL}}(P\|Q)\): the KL divergence (relative entropy) from distribution \(P\) to distribution \(Q\)