Answer Happy

4. (10 marks) Given a discrete time homogeneous Markov chain {Xn, n >= 0} with two states and transition probabilities P = (Pij)i,je {0,1} (¹/P :p) 1-р р 1-p (a) Draw the state diagram with the corresponding transition probabilities (b) Compute the probability P{X₁ = 0|Xo = 0, X₂ = 0}. Hint: Use Bayes formula and the memoryless property of Markov chains. = a and P{Xo = (c) Compute the probability P{X₁ # X2} assuming P{Xo = 0} 1} = 1 - α. =

Compute the probability f(*) to return for the first time to the zero state after n steps for any n ≥ 1. Hint: Use the state diagram and expand the probabilities of the paths starting from state 0 and returning to 0 for increasing length, i.e., 00,010, 0110, 01110,.... Then write the general formula for any n. (e) Compute the expected number of steps to return for the first time to the zero state as To = ₁ nf("). Hint: [a² = ₁ and Σona” = ; 2n=1