Matrices

This post was machine-translated from the Korean original by Marvin (via Kimi). It may contain errors or awkward phrasing — the Korean original is the source of truth.

In this post we define matrices and the operations between them.

Basic Definitions

Simply put, an \(m\times n\) matrix is a rectangular array of \(mn\) scalars arranged in an \(m\times n\) grid. That is, if we write the entry of \(A\) in the \(i\)th row and \(j\)th column as \(A_{ij}\), then a matrix can be expressed as follows.

\[A=\begin{pmatrix}A_{11}&A_{12}&\cdots&A_{1n}\\A_{21}&A_{22}&\cdots&A_{2n}\\ \vdots&\vdots&\ddots&\vdots\\A_{m1}&A_{m2}&\cdots&A_{mn}\end{pmatrix}\]

In the matrix \(A\) above, the \(m\) vectors

\[(A_{11},A_{12},\cdots, A_{1n}),\;\ldots,\;(A_{m1},A_{m2},\cdots,A_{mn})\]

are called the rows of \(A\), and the \(n\) vectors

\[(A_{11},A_{21},\cdots,A_{m1}),\; \ldots,\; (A_{1n}, A_{2n},\cdots, A_{mn})\]

are called the columns of \(A\). Here the row vectors are all elements of the \(n\)-dimensional space \(\mathbb{K}^n\), and the column vectors are elements of the \(m\)-dimensional space \(\mathbb{K}^m\).

It is often convenient to group certain blocks of a matrix and write it in block form. For example, the following matrix

\[\begin{pmatrix}0&0&1\\ 0&0&3\\ 2&4&0\end{pmatrix}\]

can be written conveniently as in the figure below

Similarly, using the column vectors \(A_1\), \(A_2\), \(\ldots\), \(A_n\) of the above \(m\times n\) matrix \(A\), we can also write

\[A=(A_1\vert A_2\vert \cdots\vert A_n)\]

The set of all \(m\times n\) matrices whose entries are elements of \(\mathbb{K}\) is denoted by \(\Mat_{m\times n}(\mathbb{K})\). In particular, \(\Mat_{m\times m}(\mathbb{K})\) is abbreviated as \(\Mat_m(\mathbb{K})\).

Operations on Matrices

Addition and multiplication are defined for matrices, but not for arbitrary pairs. For example, addition is defined only for matrices of the same shape, as the entrywise sum. Thus, \(\Mat_{m\times n}(\mathbb{K})\) carries a well-defined addition. Furthermore, if we define scalar multiplication for each matrix by the following formula

\[\alpha\begin{pmatrix}A_{11}&A_{12}&\cdots&A_{1n}\\A_{21}&A_{22}&\cdots&A_{2n}\\ \vdots&\vdots&\ddots&\vdots\\A_{m1}&A_{m2}&\cdots&A_{mn}\end{pmatrix}=\begin{pmatrix}\alpha A_{11}&\alpha A_{12}&\cdots&\alpha A_{1n}\\\alpha A_{21}&\alpha A_{22}&\cdots&\alpha A_{2n}\\ \vdots&\vdots&\ddots&\vdots\\\alpha A_{m1}&\alpha A_{m2}&\cdots&\alpha A_{mn}\end{pmatrix}\]

then one can easily verify that \(\Mat_{m\times n}(\mathbb{K})\) has a \(\mathbb{K}\)-vector space structure. This vector space has dimension \(mn\). A basis is most simply given by the following matrices

\[\begin{pmatrix}1&0&\cdots&0\\0&0&\cdots&0\\\vdots&\vdots&\ddots&\vdots\\0&0&\cdots&0\end{pmatrix},\quad\begin{pmatrix}0&1&\cdots&0\\0&0&\cdots&0\\\vdots&\vdots&\ddots&\vdots\\0&0&\cdots&0\end{pmatrix},\quad\cdots,\quad\begin{pmatrix}0&0&\cdots&0\\0&0&\cdots&0\\\vdots&\vdots&\ddots&\vdots\\0&0&\cdots&1\end{pmatrix}\]

Matrix multiplication is somewhat more complicated, so we first define the product of a matrix and a vector.

Definition 1 Let \(A\in\Mat_{m\times n}(\mathbb{K})\) and \(x\in\mathbb{K}^n\). Then the product \(Ax\) of the matrix \(A\) and the vector \(x\) is defined by the formula

\[Ax=\sum_{j=1}^nx_j A_j\]

Here \(x_j\) is the \(j\)th component of \(x\), and \(A_j\) is the \(j\)th column vector of \(A\).

By the above definition, for any vector \(x\) the product \(Ax\) is always a linear combination of the column vectors \(A_j\). Therefore, the set of all \(Ax\) for \(x\in\mathbb{K}^n\) is \(\span\left\{A_1, A_2,\ldots,A_n\right\}\), which we call the column space of \(A\) and denote briefly by \(\col A\). Of course, the row space of \(A\) can be defined similarly, but it is rarely used.

The above formula can be written out in more detail. Let \(A_{ij}\) denote the \(i\)th component of the column vector \(A_j\). Then the first component of the vector \(Ax\) is the sum of the first components of the vectors \(x_jA_j\), so it can be written as

\[x_1A_{11}+x_2A_{12}+\cdots+x_nA_{1n}\]

and, more generally, the \(i\)th component of \(Ax\) is

\[x_1A_{i1}+x_2A_{i2}+\cdots+x_nA_{in}\]

Thus we explicitly obtain the formula

\[\begin{pmatrix}A_{11}&A_{12}&\cdots&A_{1n}\\A_{21}&A_{22}&\cdots&A_{2n}\\\vdots&\vdots&\ddots&\vdots\\A_{m1}&A_{m2}&\cdots&A_{mn}\end{pmatrix}\begin{pmatrix}x_1\\x_2\\\vdots\\x_n\end{pmatrix}=\begin{pmatrix}\sum_{i=1}^nA_{1i}x_i\\\sum_{i=1}^nA_{2i}x_i\\\vdots\\\sum_{i=1}^nA_{mi}x_i\end{pmatrix}\]

Matrix Multiplication

Having defined the product of a matrix and a vector, we can now define the product of two matrices.

Definition 2 For two matrices \(A\in \Mat_{m\times n}(\mathbb{K})\) and \(B\in\Mat_{p\times q}(\mathbb{K})\), a necessary and sufficient condition for the matrix product \(BA\) to be defined is that \(q=m\). In this case, the matrix product \(BA\) is given by the formula

\[BA=(BA_1\vert BA_2\vert \cdots\vert BA_n)\]

(\(A_i\) is the \(i\)th column of \(A\), and \(BA_i\) is the product of the column vector \(A_i\) and the matrix \(B\)¹)

That is, if we write the \(i\)th component of \(A_j\) as \(A_{ij}\), then we can write

\[BA_j=\sum_{k=1}^m A_{kj}B_k\]

Then the \(i\)th component of the column vector \(BA_j\), i.e., the \((i,j)\)-entry \((BA)_{ij}\) of the matrix \(BA\), is given by

\[(BA)_{ij}=\sum_{k=1}^n B_{ik}A_{kj}\]

Matrix multiplication does not satisfy the commutative law. That is, \(AB=BA\) does not hold in general. Not only is there no guarantee that \(AB\) is defined even when \(BA\) is, but even when both are defined, \(AB\) and \(BA\) may have different shapes. Moreover, even if \(m=n=p=q\) so that \(AB\), \(BA\in\Mat_{m\times m}(\mathbb{K})\), the two products may still differ.

On the other hand, matrix multiplication satisfies the associative law. If \(A\), \(B\), and \(C\) are matrices such that \(AB\) and \(BC\) are both defined, then the products \(A(BC)\) and \((AB)C\) are also well defined, and

\[A(BC)=(AB)C\]

holds.

Definition 3 The matrix \(I=(e_1\mid e_2\mid\cdots\mid e_n)\) is called the \(n\times n\) identity matrix.

That is,

\[I=\begin{pmatrix}1&0&\cdots&0\\0&1&\cdots&0\\\vdots&\vdots&\ddots&\vdots\\ 0&0&\cdots&1\end{pmatrix}\]

Similarly, a matrix whose off-diagonal entries are all zero is called a diagonal matrix.

True to its name, if \(I\) is the \(n\times n\) identity matrix, then \(AI=A\) for any \(m\times n\) matrix \(A\), and \(IB=B\) for any \(n\times m\) matrix \(B\).

Definition 4 Let \(A\) be a matrix in \(\Mat_n(\mathbb{K})\). Then \(A\) is called invertible if there exists some \(B\in\Mat_n(\mathbb{K})\) such that \(AB=BA=I\). A matrix that is not invertible is called a singular matrix.

Not every element of \(\Mat_n(\mathbb{K})\) is invertible. For example, the zero matrix \(O\) is not invertible. Thus, \(\Mat_n(\mathbb{K})\) is not generally an abelian group. Instead, if we collect only invertible $n\times n$ matrices, this set carries a group structure.

Definition 5 The group whose underlying set is the collection of \(n\times n\) invertible matrices and whose operation is matrix multiplication is written \(\GL(n,\mathbb{K})\) and is called the general linear group.

As mentioned above, matrix multiplication satisfies the associative law. That is, the operation of \(\GL(n,\mathbb{K})\) is associative. Moreover, the identity element of \(\GL(n,\mathbb{K})\) is the identity matrix \(I\), and the inverse of \(A\in\GL(n,\mathbb{K})\) is the matrix \(B\) appearing in Definition 4. From the uniqueness in §Abelian Groups and Fields, ⁋Proposition 2, we know that the inverse of \(A\) is unique. It is called the inverse matrix of \(A\) and is denoted by \(A^{-1}\). From the observation

\[(B^{-1}A^{-1})AB=B^{-1}IB=B^{-1}B=I\]

we see that the product \(AB\) of two invertible matrices is also invertible, and its inverse is \(B^{-1}A^{-1}\).

Let us consider why the inverse of \(A\) is defined only for \(n\times n\) invertible matrices. If an inverse of an \(m\times n\) matrix \(A\) existed, then since both \(AB\) and \(BA\) must exist, the inverse of \(A\) would necessarily be an \(n\times m\) matrix, and it would have to satisfy

\[AB=I_m,\qquad BA=I_n\]

Such a matrix \(B\) does not exist. This follows immediately from the §Fundamental Theorem of Linear Algebra, but it can also be proved using only the language of matrices.

Definition 6 Let \(A\in\Mat_n(\mathbb{K})\). Then the trace of \(A\), denoted \(\tr(A)\), is defined as the sum of the diagonal entries of \(A\), i.e. \(\tr(A)=\sum_1^n A_{ii}\).

The map \(\tr\) defined in this way is a linear map from \(\Mat_n(\mathbb{K})\) to \(\mathbb{K}\). That is, for any \(A,B\in\Mat_n(\mathbb{K})\) and \(\alpha\in\mathbb{K}\),

\[\tr(A+B)=\tr(A)+\tr(B),\qquad \tr(\alpha A)=\alpha\tr(A)\]

hold.

Now suppose two matrices \(A\in\Mat_{m\times n}(\mathbb{K})\) and \(B\in\Mat_{n\times m}(\mathbb{K})\) are given. Then one can verify that

\[\tr(AB)=\sum_{i=1}^m(AB)_{ii}=\sum_{i=1}^m\sum_{j=1}^nA_{ij}B_{ji}=\sum_{j=1}^n\sum_{i=1}^m B_{ji}A_{ij}=\sum_{j=1}^n(BA)_{jj}=\tr(BA)\]

Therefore, if \(AB=I_m\) and \(BA=I_n\), then from the above formula

\[m=\tr(I_m)=\tr(AB)=\tr(BA)=\tr(I_n)=n\]

must hold, so if \(m\neq n\), no matrix \(B\) satisfies this.

Definition 7 For a given matrix \(A\in\Mat_{m\times n}(\mathbb{K})\), the transpose \(A^t\) of \(A\) is the \(n\times m\) matrix defined by the formula

\[(A^t)_{ij}=A_{ji}\]

Proposition 8 For two matrices \(A\in\Mat_{m\times n}(\mathbb{K}),B\in\Mat_{n\times k}(\mathbb{K})\), the formula

\[(AB)^t=B^tA^t\]

holds.

This can be verified by a direct computation.

---

References

[Goc] M.S. Gockenbach, Finite-dimensional linear algebra, Discrete Mathematics and its applications, Taylor&Francis, 2011.
[Lee] 이인석, 선형대수와 군, 서울대학교 출판문화원, 2005.

---

1. By the above definition, the \(j\)th column of the matrix \(BA\) equals the product of the matrix \(B\) and the column vector \(A_j\); therefore, there is no risk of confusion whether one regards \(BA_j\) as the $j$th column of $BA$ or as the product of \(B\) and \(A_j\). ↩