Appendix A — Notation

Below are mathematical notational rules used throughout the course.

• Scalar quantities as denoted as italicized symbols, such as \(x\), \(y\), \(\mu\), and \(\sigma\).
• Vector quantities (first-rank tensors) are denoted in bold, such as \(\mathbf{x}\), \(\mathbf{y}\), \(\boldsymbol{\mu}\), and \(\boldsymbol{\sigma}\).
• Matrix quantities (second-rank tensors) are denoted with sans serif capital letters, such as \(\mathsf{A}\), \(\mathsf{W}\), and \(\mathsf{\sigma}\).
• The one exception to the boldface and sans serif convention is when we denote a generic set of data or parameters. In that case, we use standard italicized symbols like \(\theta\) (typically for a set of parameters) or \(z\) (typically for a set of latent variables).
• Subscripts typically denote an element of a vector, such as \(x_i\), or an element of a matrix, such as \(A_{ij}\). They can also denote an entry in a non-ordered collection, such as \(M_i\).
• Transposes are denoted with a superscript \(\mathsf{T}\).
• Vector dot products result in a scalar and are denoted with a dot, such as \(\mathbf{x}\cdot\mathbf{y}\). Note that this is denoted as \(\mathbf{x}^\mathsf{T}\mathbf{x}\) in some texts, but we will not use that notation. Writing out the sum, this is

\[\begin{aligned} \mathbf{x}\cdot\mathbf{y} = \sum_{i}x_i\, y_i. \end{aligned} \tag{A.1}\]

• Matrix-vector products result in a vector are also denoted with a dot, such as \(\mathsf{A}\cdot\mathbf{x}\). Writing out the sum, this is

\[\begin{aligned} \mathsf{A}\cdot\mathbf{x} = \begin{pmatrix}\sum_{i}A_{i1} x_i \\ \sum_{i}A_{i2} x_i \\ \vdots \end{pmatrix} \end{aligned} \tag{A.2}\]

• Matrix-matrix multiplication results in a matrix and is also denoted with a dot, such as \(\mathsf{A}\cdot\mathsf{B}\).