Summary
ILinear matrix equationsIIInvertible matrices and product matrix equationsADefinitionBCriterion of invertibilityCSolving matrix equationsIIIAugmented matrices and system of equationsAMatrices and systems of linear equationsBSolve a system of linear equationsLinear matrix equations
Linear matrix equation
A linear matrix equation is an equation that can be put in the form:
A+X=B
Where A and B are matrices of the same size and X is a matrix of variables of the same size as A and B.
The following is a linear matrix equation:
\begin{bmatrix}1 &2 \\ 3& 4\end{bmatrix} +\begin{bmatrix}x_{1{,}1} &x_{1{,}2} \\ x_{2{,}1}& x_{2{,}2}\end{bmatrix}=\begin{bmatrix}3 &3 \\ 3& 3\end{bmatrix}
Solving a linear matrix equation requires solving for every component of X.
Consider the following matrix equation:
\begin{bmatrix}1 &2 \\ 3& 4\end{bmatrix} +\begin{bmatrix}x_{1{,}1} &x_{1{,}2} \\ x_{2{,}1}& x_{2{,}2}\end{bmatrix}=\begin{bmatrix}3 &3 \\ 3& 3\end{bmatrix}
It is equivalent to:
\begin{bmatrix}1+x_{1{,}1} &2+x_{1{,}2} \\ 3+x_{2{,}1}& 4+x_{2{,}2}\end{bmatrix} =\begin{bmatrix}3 &3 \\ 3& 3\end{bmatrix}
Which is equivalent to the following four equations:
\begin{cases} 1+x_{1{,}1}=3 \cr \cr 2+x_{1{,}2}=3 \cr \cr 3+x_{2{,}1}=3\cr \cr 4+x_{2{,}2}=3 \end{cases}
Each of these linear equations can be solved by isolating the variables:
\begin{cases} x_{1{,}1}=2 \cr \cr x_{1{,}2}=1 \cr \cr x_{2{,}1}=0 \cr \cr x_{2{,}2}=-1 \end{cases}
Therefore, the solution to the linear matrix equation is:
\begin{bmatrix}x_{1{,}1} &x_{1{,}2} \\ x_{2{,}1}& x_{2{,}2}\end{bmatrix}=\begin{bmatrix}2 &1 \\ 0& -1\end{bmatrix}
The matrices in a matrix linear equation are not necessarily of size 2\times 2, they can be of any size.
Consider the following linear matrix equation consisting of matrices of size 2\times 3 :
\begin{bmatrix}1 &2& 3 \\ 3& 4&-2\end{bmatrix} +\begin{bmatrix}x_{1{,}1} &x_{1{,}2} & x_{1{,}3} \\ x_{2{,}1}& x_{2{,}2}&x_{2{,}3}\end{bmatrix}=\begin{bmatrix}3 &3 &7 \\ 3& 3 &8\end{bmatrix}
Solve the matrix equation by isolating the matrix of variables:
\begin{bmatrix}x_{1{,}1} &x_{1{,}2} & x_{1{,}3} \\ x_{2{,}1}& x_{2{,}2}&x_{2{,}3}\end{bmatrix}=\begin{bmatrix}3 &3 &7 \\ 3& 3 &8\end{bmatrix}-\begin{bmatrix}1 &2& 3 \\ 3& 4&-2\end{bmatrix}
\begin{bmatrix}x_{1{,}1} &x_{1{,}2} & x_{1{,}3} \\ x_{2{,}1}& x_{2{,}2}&x_{2{,}3}\end{bmatrix}=\begin{bmatrix}2 &1 &4 \\ 0& -1 &10\end{bmatrix}
Invertible matrices and product matrix equations
Definition
Identity matrix
The n\times n identity matrix is the n\times n whose diagonal entries are all 1 and every other entry is 0.
The following matrix is the 2\times 2 identity matrix:
\begin{bmatrix}1&0\\0&1\end{bmatrix}
The following matrix is the 3\times 3 identity matrix:
\begin{bmatrix}1&0 & 0 \\ 0 & 1 & 0\\0&0&1\end{bmatrix}
Matrix multiplication with identity matrix
Let I_n be the n\times n identity matrix, A a matrix of size m\times n, and B a matrix of size n\times p. Then:
- AI_n=A
- I_nB=B
Consider the following matrix:
A=\begin{bmatrix}2&3&1\\0&8&1\end{bmatrix}
A is a matrix of size 2\times 3. Observe that multiplying A on the left by the 2\times 2 identity matrix leaves A unchanged:
\begin{bmatrix}1&0\\0&1\end{bmatrix}\begin{bmatrix}2&3&1\\0&8&1\end{bmatrix}=\begin{bmatrix}2&3&1\\0&8&1\end{bmatrix}
Observe further that multiplying A on the right by the 3\times 3 identity matrix leaves A unchanged:
\begin{bmatrix}2&3&1\\0&8&1\end{bmatrix}\begin{bmatrix}1&0&0\\0&1&0\\0&0&1\end{bmatrix}=\begin{bmatrix}2&3&1\\0&8&1\end{bmatrix}
Invertibility
A square matrix A is said to be invertible if there exists a square matrix A^{-1} of the same size, called the inverse of A, such that:
AA^{-1}=A^{-1}A=I_n
Where I_n is the n\times n identity matrix.
Consider the following 2\times 2 matrix:
A=\begin{bmatrix}2&1\\1&1\end{bmatrix}
The matrix A is invertible and its inverse is:
A^{-1}=\begin{bmatrix}1&-1\\-1&2\end{bmatrix}
Because:
\begin{bmatrix}2&1\\1&1\end{bmatrix}\begin{bmatrix}1&-1\\-1&2\end{bmatrix}=\begin{bmatrix}1&0\\0&1\end{bmatrix}
And:
\begin{bmatrix}1&-1\\-1&2\end{bmatrix}\begin{bmatrix}2&1\\1&1\end{bmatrix}=\begin{bmatrix}1&0\\0&1\end{bmatrix}
Not every matrix is invertible.
For example, the matrix A=\begin{bmatrix}1&0\\0&0\end{bmatrix} is not invertible. To see that, consider any 2\times 2 matrix B=\begin{bmatrix}a&b\\c&d\end{bmatrix} and observe that:
AB=\begin{bmatrix}a&b\\0&0\end{bmatrix}
The bottom row of the matrix multiplication consists entirely of 0 's. Therefore, it is impossible to find a matrix B such that AB=\begin{bmatrix}1&0\\0&1\end{bmatrix}.
Non-square matrices are never invertible.
The matrix \begin{bmatrix}2& 3 &1 \\3 & 6 & -7\end{bmatrix} is not square, therefore it is not invertible.
Suppose A and B are square matrices of the same size. It is uncommon for matrix multiplication between A and B to commute. We often have AB\neq BA.
Let A=\begin{bmatrix}0&1\\0&0\end{bmatrix} and B=\begin{bmatrix}1&0\\0&0\end{bmatrix}. Then:
AB=\begin{bmatrix}0&0\\0&0\end{bmatrix}
BA=\begin{bmatrix}0&1\\0&0\end{bmatrix}
We have:
AB\neq BA
Let I_n be the n\times n identity matrix. If A and B are square matrices of the same size and AB=I_n, then:
BA=I_n
Product matrix equation
A product matrix equation is an equation of the form
A\vec{x}=\vec{b}
Where A is a m\times n matrix, \vec{x}=\begin{bmatrix}x_1\\x_2\\\vdots\\x_n\end{bmatrix} is an n\times 1 matrix of variables, and \vec{b} is an m\times 1 column vector.
The following is an example of a product matrix equation:
\begin{bmatrix}2&3&1\\1&-1&7\end{bmatrix}\begin{bmatrix}x_1\\x_2\\x_3\end{bmatrix}=\begin{bmatrix}1\\9\end{bmatrix}
Suppose A is an m\times n matrix and we are studying the following matrix product equation:
A\vec{x}=\vec{b}
The matrix product equation is equivalent to an m\times n system of linear equations.
Consider the following matrix product equation:
\begin{bmatrix}2&3&1\\1&-1&7\end{bmatrix}\begin{bmatrix}x_1\\x_2\\x_3\end{bmatrix}=\begin{bmatrix}1\\9\end{bmatrix}
Solving the above matrix equation is equivalent to solving the following system of 2 linear equations with 3 variables:
\begin{cases} 2x_1+3x_2+x_3=1\cr \cr x_1-x_2+7x_3=9 \end{cases}\\
Criterion of invertibility
Determinant of a 2\times 2 matrix
Let A=\begin{bmatrix}a&b\\c&d\end{bmatrix} be a 2\times 2 matrix. Then the determinant of A is:
\det\left(A\right)=\left|\begin{matrix}a&b\\c&d\end{matrix}\right|=ad-bc
Let A=\begin{bmatrix}2&1\\4&2\end{bmatrix}. Then:
\det\left(A\right)=2\left(2\right)-4\left(1\right)=0
Invertibility of a 2\times 2 matrix
Let A=\begin{bmatrix}a&b\\c&d\end{bmatrix} be a 2\times 2 matrix. Then A is invertible if and only if:
\det\left(A\right)=ad-bc\neq0
In which case:
A^{-1}=\dfrac{1}{\det\left(A\right)}\begin{bmatrix}d&-b\\-c&a\end{bmatrix}
Let A=\begin{bmatrix}2&1\\4&2\end{bmatrix}. Then:
\det\left(A\right)=2\left(2\right)-4\left(1\right)=0
Therefore, the matrix A is not invertible.
Let A=\begin{bmatrix}4&1\\1&3\end{bmatrix}. Then:
\det\left(A\right)=11
Therefore, A is invertible and its inverse is:
A^{-1}=\dfrac{1}{11}\begin{bmatrix}3&-1\\-1&4\end{bmatrix}=\begin{bmatrix}\dfrac{3}{11}& \dfrac{-1}{11}\\\dfrac{-1}{11}&\dfrac{4}{11}\end{bmatrix}
Determinant of a 3\times 3 matrix
Let A=\begin{bmatrix}a_{1{,}1}&a_{1{,}2}& a_{1{,}3}\\a_{2{,}1}&a_{2{,}2}& a_{2{,}3}\\a_{3{,}1}&a_{3{,}2}& a_{3{,}3}\\\end{bmatrix} be a 3\times 3 matrix. Then the determinant of A is:
\det\left(A\right)=a_{1{,}1}\left|\begin{matrix}a_{2{,}2}&a_{2{,}3}\\a_{3{,}2}&a_{3{,}3}\end{matrix}\right|-a_{1{,}2}\left|\begin{matrix}a_{2{,}1}&a_{2{,}3}\\a_{3{,}1}&a_{3{,}3}\end{matrix}\right|+a_{1{,}3}\left|\begin{matrix}a_{2{,}1}&a_{2{,}2}\\a_{3{,}1}&a_{3{,}2}\end{matrix}\right|
Let A= \begin{bmatrix}1&0&0\\3&2&3\\4&4&6\end{bmatrix}. Then:
\det\left(A\right)=1\left|\begin{matrix}2&3\\4&6\end{matrix}\right|-0\left|\begin{matrix}3&3\\4&6\end{matrix}\right|+0\left|\begin{matrix}3&2\\4&4\end{matrix}\right|=1\left(12-12\right)-0\left(18-12\right)+0\left(12-8\right)=0
Invertibility of a 3\times 3 matrix
Let A be 3\times 3 matrix. Then A is invertible if and only if:
\det\left(A\right)\not=0
Let A= \begin{bmatrix}1&0&0\\3&2&3\\4&4&6\end{bmatrix}. Then:
\det\left(A\right)=0
Therefore, the matrix A is not invertible.
Let A= \begin{bmatrix}1&2&1\\3&2&3\\4&4&6\end{bmatrix}. Then:
\det\left(A\right)=-2
Therefore, the matrix is invertible.
Finding the inverse of an invertible 3\times 3 matrix is trickier than finding the inverse of an invertible 2\times 2 matrix. However, below there will be an algorithm discussed which can be used to find the inverse of an invertible 3\times 3 matrix.
Solving matrix equations
To solve the matrix equation AX=B for the matrix X, matrix A needs to be invertible. It is solved as follows:
AX=B
\Leftrightarrow A^{-1}AX=A^{-1}B
\Leftrightarrow X=A^{-1}B
Consider the following matrix equation:
\begin{bmatrix}2&1\\3&2\end{bmatrix}\begin{bmatrix}x_1\\x_2\end{bmatrix}=\begin{bmatrix}1\\1\end{bmatrix}
\begin{bmatrix}2&1\\3&2\end{bmatrix} is invertible with inverse:
\begin{bmatrix}2&1\\3&2\end{bmatrix}^{-1}=\begin{bmatrix}2&-1\\-3&2\end{bmatrix}
Therefore:
\begin{bmatrix}x_1\\x_2\end{bmatrix}=\begin{bmatrix}2&-1\\-3&2\end{bmatrix}\begin{bmatrix}1\\1\end{bmatrix}=\begin{bmatrix}1\\-1\end{bmatrix}
Augmented matrices and system of equations
Matrices and systems of linear equations
Augmented matrix
Suppose A\vec{x}=\vec{b} is a matrix product equation. The augmented matrix of the equation is the matrix obtained by adding \vec{b} as an extra column to A.
Consider the following matrix product equation:
\begin{bmatrix}2&3&7\\1&2&1\end{bmatrix}\begin{bmatrix}x_1\\x_2\\x_3\end{bmatrix}=\begin{bmatrix}3\\4\end{bmatrix}
The corresponding augmented matrix is:
\left[\begin{array}{rrr|r}2&3&7&3\\1&2&1&4\end{array}\right]
The vertical line in the augmented matrix is used to distinguish between the original matrix and the added column.
When solving a system of linear equations, it is often cleaner and easier to solve by studying the augmented matrix.
Elementary row operations
An elementary row operation on a matrix is a manipulation of the matrix in one of the following ways:
- Multiplying a row by a nonzero number
- Adding a multiple of one row to another
- Switching two rows
Consider the following augmented matrix:
\left[\begin{array}{rr|r} 1 & 2 & 3 \\ 4 & 2 & 1 \\ \end{array} \right]
Multiplying the first row by the nonzero number 2 produces the following matrix:
\left[\begin{array}{rr|r} 2 & 4 & 6 \\ 4 & 2 & 1 \\ \end{array} \right]
Adding row one to row two of the new matrix produces the following matrix:
\left[\begin{array}{rr|r} 2 & 4 & 6 \\ 4+2 & 2+4 & 1+6 \\ \end{array} \right]\\= \left[\begin{array}{rr|r} 2 & 4 & 6 \\ 6 & 6 & 7 \\ \end{array} \right]
Swapping the two rows of the new matrix produces the following matrix:
\left[\begin{array}{rr|r} 6 & 6 & 7 \\ 2 & 4 & 6 \\ \end{array} \right]
Elementary row operations on a matrix correspond to manipulating the equations of systems of linear equations:
- Multiplying a row of matrix by a nonzero number corresponds to multiplying an equation of a system of linear equations by the same nonzero number.
- Adding a multiple of one row of a matrix to another row corresponds to adding a multiple of one equation in a system of linear equations with another equation in the system.
- Switching two rows in a matrix corresponds to switching two equations in a system of linear equations.
Consider the following system of linear equations:
2x+3y=4\\x-y=7
The corresponding matrix to the system of linear equations is:
\left[\begin{array}{rr|r} 2 & 3& 4 \\ 1 & -1 & 7 \\ \end{array} \right]
Multiplying the second row by -2 produces the following matrix:
\left[\begin{array}{rr|r} 2 & 3& 4 \\ -2 & 2 & -14 \\ \end{array} \right]
The new matrix corresponds to the following system of linear equations:
\begin{cases} 2x+3y=4\cr \cr 2x-2y=-14 \end{cases}
This system of linear equations is obtained by multiplying the second equation in the original system by -2.
Equivalent systems of linear equations
Suppose A and B are two augmented matrices of the same size and the matrix B can be obtained by performing elementary row operations on A. The two systems of linear equations that matrices A and B represent are equivalent, and have the same set of solutions. In particular, solving one of the systems of linear equations is equivalent to solving the other system of linear equations.
Consider the following matrix:
\left[\begin{array}{rrr|r} 2 & 3 & -1 & 2 \\ 10 & -2 & -1 & 1 \\ 1 & 1 & 1 & 1 \\ \end{array} \right]
If we multiply the first row by 2 and switch the second and third rows, then we obtain the matrix:
\left[\begin{array}{rrr|r} 4 & 6 & -2 & 4 \\ 1 & 1 & 1 & 1 \\ 10 & -2 & -1 & 1 \\ \end{array} \right]
Therefore, the following two systems of linear equations have the same solutions:
\begin{cases} 2x+3y-z=2 \cr \cr 10x-2y-z=1 \cr \cr x+y+z=1 \end{cases}
And:
\begin{cases} 4x+6y-2z=4 \cr \cr x+y+z=1 \cr \cr 10x-2y-z=1\end{cases}
Solve a system of linear equations
The method of Gaussian elimination will allow us to solve systems of linear equations by manipulating augmented matrices with elementary row operations. The goal of Gaussian elimination is to use elementary row operations on an augmented matrix to make it upper triangular.
Consider the following system of linear equations:
\begin{cases} 2x+3y=2 \cr \cr 4x+y=-1 \end{cases}
The corresponding matrix to the system of linear equations is:
\left[\begin{array}{rr|r} 2 & 3& 2 \\ 4 & 1 & -1 \\ \end{array} \right]
We begin the process of Gaussian elimination by making a 1 appear in the upper righthand corner. This can be accomplished by multiplying the first row by \dfrac{1}{2}, which produces the following matrix:
\left[\begin{array}{rr|r} 1 & \dfrac{3}{2}& 1 \\ 4 & 1 & -1 \\ \end{array} \right]
We next want to make a 0 appear in the bottom lefthand corner. This is accomplished by subtracting four times the first row from the second. The resulting matrix is:
\left[\begin{array}{rr|r} 1 & \dfrac{3}{2}& 1 \\ 0 & -5 & -5 \\ \end{array} \right]
To make a 1 appear to the right of the 0, we can multiply the second row by \dfrac{-1}{5}. The resulting matrix is:
\left[\begin{array}{rr|r} 1 & \dfrac{3}{2}& 1 \\ 0 & 1 & 1 \\ \end{array} \right]
Therefore, the original system of equations is equivalent to the following system of linear equations:
\begin{cases} x+\dfrac{3}{2}y=1 \cr \cr y=1 \end{cases}
Therefore, y=1 and substituting in the first equation:
x+\dfrac{3}{2}y=1\\\\x+\dfrac{3}{2}=1\\x=\dfrac{-1}{2}
Using Gaussian elimination, the solution to the original system of linear equations is x=\dfrac{-1}{2} and y=3.
Consider the following system of linear equations:
\begin{cases} y+z=1 \cr \cr x+2y+z=2 \cr \cr 3x+2y+3z=3 \end{cases}
The corresponding matrix is:
\left[\begin{array}{rrr|r} 0 & 1& 1&1\\ 1 & 2& 1& 2\\ 3 & 2 &3 &3 \\ \end{array} \right]
To make a 1 appear in the top right, switch the first and second rows. The resulting matrix is:
\left[\begin{array}{rrr|r} 1 & 2& 1&2\\ 0 & 1& 1& 1\\ 3 & 2 &3 &3 \\ \end{array} \right]
To make 0 appear in all entries below the top right hand corner, we do not need to alter the second row, but to subtract 3 times the first row from the third. The resulting matrix is:
\left[\begin{array}{rrr|r} 1 & 2& 1&2\\ 0 & 1& 1& 1\\ 0 & -4 &0 &-3 \\ \end{array} \right]
We next work on row 2. We only need there to be a 1 in the entry of the second row and second column, which there already is. Therefore, we do not need to alter the second row anymore.
We next need to make a 0 appear below the first nonzero entry of the second row. Add four times the second row to the third row and obtain the matrix:
\left[\begin{array}{rrr|r} 1 & 2& 1&2\\ 0 & 1& 1& 1\\ 0 & 0&4 &1 \\ \end{array} \right]
The last thing we need to do is make a 1 appear where the 4 is located. This is accomplished by multiplying the fourth row by \dfrac{1}{4}. The corresponding matrix is:
\left[\begin{array}{rrr|r} 1 & 2& 1&2\\ 0 & 1& 1& 1\\ 0 & 0&1 &\dfrac{1}{4} \\ \end{array} \right]
The system of linear equations corresponding to the matrix obtained from Gaussian elimination is:
\begin{cases} x+2y+z=1\cr \cr y+z=1 \cr \cr z=\dfrac{3}{4} \end{cases}
Therefore, in the system of linear equations we have z=\dfrac{1}{4}. Substituting into the equation y+z=1, we have:
\begin{cases} y+z=1\cr \cr y+\dfrac{3}{4}=1 \cr \cr y=\dfrac{3}{4}\end{cases}
Lastly, substituting into the first equation:
\begin{cases} x+2y+z=2 \cr \cr x+\dfrac{6}{4}+\dfrac{3}{4}=2 \cr \cr x=\dfrac{1}{4} \end{cases}
Therefore, the solution to the system of linear equations is x=\dfrac{1}{4}, y=\dfrac{3}{4}, and z=\dfrac{1}{4}.
If the matrix corresponding to a system of linear equations is in the following form, then the system of linear equations is readily solved by substitution:
\left[\begin{array}{rrr|r} 1 & *& *&* \\ 0 & 1& *& *\\ 0 & 0 &1 &* \\ \end{array} \right]
Consider the following matrix:
\left[\begin{array}{rr|r} 1 & 3& 7 \\ 0 & 1 & 4 \\ \end{array} \right]
The corresponding system of equations is
\begin{cases} x+3y=7 \cr \cr y=4 \end{cases}
Therefore, y=4 and substituting into the first equation:
x+3y=7
x+3\left(4\right)=7
x+12=7
x=-5
Therefore, the solution to the corresponding system of linear equations is x=-5 and y=4.
The purpose of this example is to illustrate how simple it is to solve a system of linear equations when the corresponding matrix is in the following form:
\left[\begin{array}{rr|r} 1 & *& * \\ 0 & 1 & * \\ \end{array} \right]
The * symbol is a place holder for any real number. The process of Gaussian elimination is the process of performing elementary row operations to a matrix in order for it to have the above form. The purpose of doing so is to be able to solve a system of linear equations.