Answer Happy

1. For each of the following transition matrices, do the following. (i) Determine whether the Markov chain is irreducible. (ii) For each irreducible one, find its stationary distribution 7. (iii) Determine whether the Markov chain is periodic and give the period if so. (iv) Specify the classes of the Markov chain and determine for each class whether it is recurrent or transient. (v) For irreducible Markov chains, calculate the expected number of steps to return to each state when starting from the state in the long run. (vi) Using any software, find P100 and check if each row matches with for irreducible Markov chains. (a) (b) © 0 0.5 0.5 P=0.5 0 0.5 0.5 0.5 0 P = 0 0 0 0 0.5 0.5 0 0.5 0 0.25 0.5 01 0.5 0.25 0 0 1 1 HOO 00 0 0 0 0 0

(vi) Using any software, find P¹00 and check if each row matches with π for irreducible Markov chains. (a) (b) (c) P = P = 0.5 0.5 0.5 0 0.5 0.5 0.5 0 0 0 1 0 0 0 1 B 0.5 0.5 00 0 0 1 0 P = 0.5 0 0.25 0.5 0.5 0 0 0 0 0 0.5 0.25 0.5 0 0 0 0 0 0.5 0.5 0.5 0.5 0 0