Section2.2Matrix multiplication and linear combinations¶ permalink

The previous section introduced vectors and linear combinations and demonstrated how they provide a means of thinking about linear systems geometrically. In particular, we saw that the vector \(\bvec\) is a linear combination of the vectors \(\vvec_1,\vvec_2,\ldots,\vvec_n\) if the linear system corresponding to the augmented matrix

Our goal in this section is to introduction matrix multiplication, another algebraic operation that connects linear systems and linear combinations.

Subsection2.2.1Matrices

We first thought of a matrix as a rectangular array of numbers. When the number of rows is \(m\) and columns is \(n\text{,}\) we say that the dimensions of the matrix are \(m\times n\text{.}\) For instance, the matrix below is a \(3\times4\) matrix:

The matrix \(I_n\text{,}\) which we call the identity matrix is the \(n\times n\) matrix whose entries are zero except for the diagonal entries, which are 1. For instance,

As this preview activity shows, both of these operations are relatively straightforward. Some care, however, is required when adding matrices. Since we need the same number of vectors to add and since the vectors must be of the same dimension, two matrices must have the same dimensions as well if we wish to form their sum.

The identity matrix will play an important role at various points in our explorations. It is important to note that it is a square matrix, meaning it has an equal number of rows and columns, so any matrix added to it must be square as well. Though we wrote it as \(I_n\) in the activity, we will often just write \(I\) when the dimensions are clear.

Subsection2.2.2Matrix multiplication and linear combinations

A more important operation will be matrix multiplication as it allows us to compactly express linear systems. For now, we will work with the product of a matrix and vector, which we illustrate with an example.

Example2.2.1

Suppose we have the matrix \(A\) and vector \(\xvec\) as given below.

Let's take note of the dimensions of the matrix and vectors. The two components of the vector \(\xvec\) are weights used to form a linear combination of the columns of \(A\text{.}\) Since \(\xvec\) has two components, \(A\) must have two columns. In other words, the number of columns of \(A\) must equal the dimension of the vector \(\xvec\text{.}\)

In the same way, the columns of \(A\) are 3-dimensional so any linear combination of them is 3-dimensional as well. Therefore, \(A\xvec\) will be 3-dimensional.

We then see that if \(A\) is a \(3\times2\) matrix, \(\xvec\) must be a 2-dimensional vector and \(A\xvec\) will be 3-dimensional.

More generally, we have the following definition.

Definition2.2.2

The product of a matrix \(A\) by a vector \(\xvec\) will be the linear combination of the columns of \(A\) using the components of \(\xvec\) as weights.

If \(A\) is an \(m\times n\) matrix, then \(\xvec\) must be an \(n\)-dimensional vector, and the product \(A\xvec\) will be an \(m\)-dimensional vector. If

If \(A\xvec\) is defined, what is the dimension of the vector \(\xvec\) and what is the dimension of \(A\xvec\text{?}\)

A vector whoses entries are all zero is denoted by \(\zerovec\text{.}\) If \(A\) is a matrix, what is the product \(A\zerovec\text{?}\)

Suppose that \(I = \left[\begin{array}{rrr}
1 \amp 0 \amp 0 \\
0 \amp 1 \amp 0 \\
0 \amp 0 \amp 1 \\
\end{array}\right]\) is the identity matrix and \(\xvec=\threevec{x_1}{x_2}{x_3}\text{.}\) Find the product \(I\xvec\) and explain why \(I\) is called the identity matrix.

Suppose we write the matrix \(A\) in terms of its columns as

Is there a vector \(\xvec\) such that \(A\xvec = \bvec\text{?}\)

Multiplication of a matrix \(A\) and a vector is defined as a linear combination of the columns of \(A\text{.}\) However, there is a shortcut for computing such a product. Let's look at our previous example and focus on the first row of the product.

To find the first component of the product, we consider the first row of the matrix. We then multiply the first entry in that row by the first component of the vector, the second entry by the second component of the vector, and so on, and add the results. In this way, we see that the third component of the product would be obtained from the third row of the matrix by computing \(2(3) + 3(1) = 9\text{.}\)

You are encouraged to evaluate Item a using this shortcut and compare the result to what you found while completing the previous activity.

Activity2.2.3

In addition, Sage can find the product of a matrix and vector using the * operator. For example,

Use Sage to evaluate the product Item a yet again.

What do you find when you evaluate \(A\zerovec\text{?}\)

What do you find when you evaluate \(A(3\vvec)\) and \(3(A\vvec)\) and compare your results?

What do you find when you evaluate \(A(\vvec+\wvec)\) and \(A\vvec + A\wvec\) and compare your results?

If \(I=\left[\begin{array}{rrr}
1 \amp 0 \amp 0 \\
0 \amp 1 \amp 0 \\
0 \amp 0 \amp 1 \\
\end{array}\right]\) is the \(3\times3\) identity matrix, what is the product \(IA\text{?}\)

This activity demonstrates several general properties satisfied by matrix multiplication that we record here.

Proposition2.2.3Linearity of matrix multiplication

If \(A\) is a matrix, \(\vvec\) and \(\wvec\) vectors, and \(c\) a scalar, then

\(A\zerovec = \zerovec\text{.}\)

\(A(c\vvec) = cA\vvec\text{.}\)

\(A(\vvec+\wvec) = A\vvec + A\wvec\text{.}\)

Subsection2.2.3Matrix multiplication and linear systems

So far, we have begun with a matrix \(A\) and a vector \(\xvec\) and formed their product \(A\xvec = \bvec\text{.}\) We would now like to turn this around: beginning with a matrix \(A\) and a vector \(\bvec\text{,}\) we will ask if we can find a vector \(\xvec\) such that \(A\xvec = \bvec\text{.}\) This will naturally lead back to linear systems.

To see the connection between the matrix equation \(A\xvec =
\bvec\) and linear systems, let's write the matrix \(A\) in terms of its columns \(\vvec_i\) and \(\xvec\) in terms of its components.

We know that the matrix product \(A\xvec\) forms a linear combination of the columns of \(A\text{.}\) Therefore, the equation \(A\xvec = \bvec\) is merely a compact way of writing the equation for the weights \(c_i\text{:}\)

We have seen this equation before: Remember that Proposition 2.1.7 says that the solutions of this equation are the same as the solutions to the linear system whose augmented matrix is

This gives us three different ways of looking at the same solution space.

Proposition2.2.4

If \(A=\left[\begin{array}{rrrr}
\vvec_1\amp\vvec_2\amp\ldots\vvec_n
\end{array}\right]\) and \(\xvec=\left[
\begin{array}{r}
x_1 \\ x_2 \\ \vdots \\ x_n \\
\end{array}\right]
\text{,}\) then the following are equivalent.

The vector \(\xvec\) satisfies \(A\xvec = \bvec
\text{.}\)

The vector \(\bvec\) is a linear combination of the columns of \(A\) with weights \(x_j\text{:}\)

and say that we augment the matrix \(A\) by the vector \(\bvec\text{.}\)

We may think of \(A\xvec = \bvec\) as merely giving a notationally compact way of writing a linear system. This form of the equation, however, will allow us to focus on important features of the system that determine its solution space.

Since we originally asked to describe the solutions to the equation \(A\xvec = \bvec\text{,}\) we will express the solution in terms of the vector \(\xvec\text{:}\)

This shows that the solutions \(\xvec\) may be written in the form \(\vvec + x_3\wvec\text{,}\) for appropriate vectors \(\vvec\) and \(\wvec\text{.}\) Geometrically, the solution space is a line in \(\real^3\) through \(\vvec\) moving parallel to \(\wvec\text{.}\)

Suppose \(A\) is an \(m\times n\) matrix. What can you guarantee about the solution space of the equation \(A\xvec
= \zerovec\text{?}\)

Subsection2.2.4Matrix products

In this section, we have developed some algebraic operations on matrices with the aim of simplifying our description of linear systems. We will now introduce a final operation, the product of two matrices, that will become important when we study linear transformations in Section 5.

Given matrices \(A\) and \(B\text{,}\) we will form their product \(AB\) by first writing \(B\) in terms of its columns:

It is important to note that we can only multiply matrices if the dimensions of the matrices are compatible. More specifically, when constructing the product \(AB\text{,}\) the matrix \(A\) multiplies the columns of \(B\text{.}\) Therefore, the number of columns of \(A\) must equal the number of rows of \(B\text{.}\) When this condition is met, the number of rows of \(AB\) is the number of rows of \(A\text{,}\) and the number of columns of \(AB\) is the number of columns of \(B\text{.}\)

Suppose we want to form the product \(AB\text{.}\) Before computing, first explain how you know this product exists and then explain what the dimensions of the resulting matrix will be.

Compute the product \(AB\text{.}\)

Sage can multiply matrices using the * operator. Define the matrices \(A\) and \(B\) in the Sage cell below and check your work by computing \(AB\text{.}\)

Are you able to form the matrix product \(BA\text{?}\) If so, use the Sage cell above to find \(BA\text{.}\) Is it generally true that \(AB = BA\text{?}\)

If \(AB = 0\text{,}\) is it necessarily true that either \(A=0\) or \(B=0\text{?}\)

This activity demonstrated some general properties about products of matrices, which mirror some properties about operations with real numbers.

Properties of Matrix Multiplication

If \(A\text{,}\) \(B\text{,}\) and \(C\) are matrices such that the following operations are defined, it follows that

Associativity:

\(A(BC) = (AB)C\text{.}\)

Distributivity:

\(A(B+C) = AB+AC\text{.}\)

\((A+B)C = AC+BC\text{.}\)

At the same time, there are a few properties that hold for real numbers that do not hold for matrices.

Things to be careful of

The following properties hold for real numbers but not for matrices.

Commutativity:

It is not generally true that \(AB = BA\text{.}\)

Cancellation:

It is not generally true that \(AB = AC\) implies that \(B = C\text{.}\)

Zero divisors:

It is not generally true that \(AB = 0\) implies that either \(A=0\) or \(B=0\text{.}\)

Subsection2.2.5Summary

In this section, we have found an especially simple way to express linear systems using matrix multiplication.

If \(A\) is an \(m\times n\) matrix and \(\xvec\) an \(n\)-dimensional vector, then \(A\xvec\) is the linear combination of the columns of \(A\) using the components of \(\xvec\) as weights. The vector \(A\xvec\) is \(m\)-dimensional.

The solution space to the equation \(A\xvec =
\bvec\) is the same as the solution space to the linear system corresponding to the augmented matrix \(\left[ \begin{array}{r|r} A \amp \bvec \end{array}\right]\text{.}\)

If \(A\) is an \(m\times n\) matrix and \(B\) is an \(n\times p\) matrix, we can form the product \(AB\text{,}\) which is an \(m\times p\) matrix whose columns are the products of \(A\) and the columns of \(B\text{.}\)

Find the matrix \(A\) and vector \(\bvec\) that expresses this linear system in the form \(A\xvec=\bvec\text{.}\)

Give a description of the solution space to the equation \(A\xvec = \bvec\text{.}\)

2

Suppose that \(A\) is a \(135\times2201\) matrix. If \(A\xvec\) is defined, what is the dimension of \(\xvec\text{?}\) What is the dimension of \(A\xvec\text{?}\)

3

Suppose that \(A \) is a \(3\times2\) matrix whose columns are \(\vvec_1\) and \(\vvec_2\text{;}\) that is,

What is the dimension of the vectors \(\vvec_1\) and \(\vvec_2\text{?}\)

What is the product \(A\twovec{1}{0}\) in terms of \(\vvec_1\) and \(\vvec_2\text{?}\) What is the product \(A\twovec{0}{1}\text{?}\) What is the product \(A\twovec{2}{3}\text{?}\)

Find the reduced row echelon form of \(A\) and identify the pivot positions.

Can you find a vector \(\bvec\) such that \(A\xvec=\bvec\) is inconsistent?

For a general 3-dimensional vector \(\bvec\text{,}\) what can you say about the solution space of the equation \(A\xvec = \bvec\text{?}\)

7

The operations that we perform in Gaussian elimination can be accomplished using matrix multiplication. This observation is the basis of an important technique that we will investigate in a subsequent chapter.

Verify that \(SA\) is the matrix that results when the second row of \(A\) is scaled by a factor of 7. What matrix \(S\) would scale the third row by -3?

Verify that \(PA\) is the matrix that results from interchanging the first and second rows. What matrix \(P\) would interchange the first and third rows?

Verify that \(L_1A\) is the matrix that results from multiplying the first row of \(A\) by \(-2\) and adding it to the second row. What matrix \(L_2\) would multiply the first row by 3 and add it to the third row?

When we performed Gaussian elimination, our first goal was to perform row operations that brought the matrix into a triangular form. For our matrix \(A\text{,}\) find the row operations needed to find a row equivalent matrix \(U\) in triangular form. By expressing these row operations in terms of matrix multiplication, find a matrix \(L\) such that \(LA = U\text{.}\)

8

In this exercise, you will construct the inverse of a matrix, a subject that we will investigate more fully in the next chapter. Suppose that \(A\) is the \(2\times2\) matrix:

Find the vectors \(\bvec_1\) and \(\bvec_2\) such that the matrix \(B=\left[\begin{array}{rr} \bvec_1 \amp \bvec_2
\end{array}\right]\) satisfies

\begin{equation*}
AB = I =
\left[\begin{array}{rr}
1 \amp 0 \\
0 \amp 1 \\
\end{array}\right]
\text{.}
\end{equation*}

In general, it is not true that \(AB = BA\text{.}\) Check that it is true, however, for the specific \(A\) and \(B\) that appear in this problem.

Suppose that \(\xvec = \twovec{x_1}{x_2}\text{.}\) What do you find when you evaluate \(I\xvec\text{?}\)

Suppose that we want to solve the equation \(A\xvec =
\bvec\text{.}\) We know how to do this using Gaussian elimination; let's use our matrix \(B\) to find a different way:

In other words, the solution to the equation \(A\xvec=\bvec\) is \(\xvec = B\bvec\text{.}\)

Consider the equation \(A\xvec = \twovec{5}{-2}\text{.}\) Find the solution in two different ways, first using Gaussian elimination and then as \(\xvec = B\bvec\text{,}\) and verify that you have found the same result.

9

Determine whether the following statements are true or false and provide a justification for your response.

If \(A\xvec\) is defined, then the number of components of \(\xvec\) equals the number of rows of \(A\text{.}\)

The solution space to the equation \(A\xvec = \bvec\) is equivalent to the solution space to the linear system whose augmented matrix is \(\left[\begin{array}{r|r} A \amp \bvec
\end{array}\right]\text{.}\)

If a linear system of equations has 8 equations and 5 unknowns, then the dimensions of the matrix \(A\) in the corresponding equation \(A\xvec = \bvec\) is \(5\times8\text{.}\)

If \(A\) has a pivot in every row, then every equation \(A\xvec = \bvec\) is consistent.

If \(A\) is a \(9\times5\) matrix, then \(A\xvec=\bvec\) is inconsistent for some vector \(\bvec\text{.}\)

10

Suppose that \(A\) is an \(4\times4\) matrix and that the equation \(A\xvec = \bvec\) has a unique solution for some vector \(\bvec\text{.}\)

What does this say about the pivots of the matrix \(A\text{?}\) Write the reduced row echelon form of \(A\text{.}\)

Can you find another vector \(\cvec\) such that \(A\xvec = \cvec\) is inconsistent?

What can you say about the solution space to the equation \(A\xvec = \zerovec\text{?}\)

Suppose \(A=\left[\begin{array}{rrrr}
\vvec_1 \amp \vvec_2 \amp \vvec_3 \amp \vvec_4
\end{array}\right]\text{.}\) Explain why every four-dimensional vector can be written as a linear combination of the vectors \(\vvec_1\text{,}\) \(\vvec_2\text{,}\) \(\vvec_3\text{,}\) and \(\vvec_4\) in exactly one way.

Describe the solution space to the homogeneous equation \(A\xvec = \zerovec\text{.}\) What does this solution space represent geometrically?

Describe the solution space to the equation \(\bvec =
\threevec{-3}{-4}{1}\text{.}\) What does this solution space represent geometrically and how does it compare to the previous solution space?

We will now explain the relationship between the previous two solution spaces. Suppose that \(\xvec_h\) is a solution to the homogeneous equation; that is \(A\xvec_h=\zerovec\text{.}\) We will also suppose that \(\xvec_p\) is a solution to the equation \(A\xvec =
\bvec\text{;}\) that is, \(A\xvec_p=\bvec\text{.}\)

Use the Linearity Principle expressed in Proposition 3 to explain why \(\xvec_h+\xvec_p\) is a solution to the equation \(A\xvec
= \bvec\text{.}\) You may do this by evaluating \(A(\xvec_h+\xvec_p)\text{.}\)

That is, if we find one solution \(\xvec_p\) to an equation \(A\xvec = \bvec\text{,}\) we may add any solution to the homogeneous equation to \(\xvec_p\) and still have a solution to the equation \(A\xvec = \bvec\text{.}\) In other words, the solution space to the equation \(A\xvec = \bvec\) is given by translating the solution space to the homogeneous equation by the vector \(\xvec_p\text{.}\)

12

Suppose that a city is starting a bicycle sharing program with bicycles at locations \(B\) and \(C\text{.}\) Bicycles that are rented at one location may be returned to either location at the end of the day. Over time, the city finds that 80% of bicycles rented at location \(B\) are returned to \(B\) with the other 20% returned to \(C\text{.}\) Similarly, 50% of bicycles rented at location \(C\) are returned to \(B\) and 50% to \(C\text{.}\)

where \(B_k\) is the number of bicycles at location \(B\) at the beginning of day \(k\) and \(C_k\) is the number of bicycles at \(C\text{.}\) The information above tells us

Suppose that there are 1000 bicycles at location \(B\) and none at \(C\) on day 1. This means we have \(\xvec_1 = \twovec{1000}{0}\text{.}\) Find the number of bicycles at both locations on day 2 by evaluating \(\xvec_2 = A\xvec_1\text{.}\)

Suppose that there are 1000 bicycles at location \(C\) and none at \(B\) on day 1. Form the vector \(\xvec_1\) and determine the number of bicycles at the two locations the next day by finding \(\xvec_2 = A\xvec_1\text{.}\)

Suppose that one day there are 1050 bicycles at location \(B\) and 450 at location \(C\text{.}\) How many bicycles were there at each location the previous day?

Suppose that there are 500 bicycles at location \(B\) and 500 at location \(C\) on Monday. How many bicycles are there at the two locations on Tuesday? on Wednesday? on Thursday?

13

This problem is a continuation of the previous problem.

Suppose that \(\xvec_1 = c_1 \vvec_1 + c_2 \vvec_2\) where \(c_2\) and \(c_2\) are scalars. Use the Linearity Principle expressed in Proposition 3 to explain why

Suppose that there are initially 500 bicycles at location \(B\) and 500 at location \(c\text{.}\) Write the vector \(\xvec_1\) and find the scalars \(c_1\) and \(c_2\) such that \(\xvec_1=c_1\vvec_1 + c_2\vvec_2\text{.}\)

Use the previous part of this problem to determine \(\xvec_2\text{,}\) \(\xvec_3\) and \(\xvec_4\text{.}\)

After a very long time, how are all the bicycles distributed?