Activity 4.5.3

Consider the matrices

\begin{equation*} A=\left[\begin{array}{rr} 0 \amp 1 \\ 1 \amp 0 \\ \end{array}\right],\qquad B=\left[\begin{array}{rr} 0.4 \amp 0.3 \\ 0.6 \amp 0.7 \\ \end{array}\right]\text{.} \end{equation*}

  1. Verify that both \(A\) and \(B\) are stochastic matrices.

  2. Find the eigenvalues of \(A\) and then find a steady-state vector for \(A\text{.}\)

  3. We will form the Markov chain beginning with the vector \(\xvec_0 = \twovec{1}{0}\) and defining \(\xvec_{k+1} = A\xvec_k\text{.}\) The Sage cell below constructs the first \(N\) terms of the Markov chain with the command markov_chain(A, x0, N). Define the matrix \(A\) and vector \(x0\) and evaluate the cell to find the first 10 terms of the Markov chain.

    What do you notice about the Markov chain? Does it converge to the steady-state vector for \(A\text{?}\)

  4. Now find the eigenvalues of \(B\) along with a steady-state vector for \(B\text{.}\)

  5. As before, find the first 10 terms in the Markov chain beginning with \(\xvec_0 = \twovec{1}{0}\) and \(\xvec_{k+1} = B\xvec_k\text{.}\) What do you notice about the Markov chain? Does it converge to the steady-state vector for \(B\text{?}\)

  6. What condition on the eigenvalues of a stochastic matrix will guarantee that a Markov chain will converge to a steady-state vector?

in-context