Consider a Markov chain with state space S {1,2,...,5} the following one-step transition matrix: 0 0 P= 0 1/2 1/2 0 0 0

[answerhappygod]

Consider A Markov Chain With State Space S 1 2 5 The Following One Step Transition Matrix 0 0 P 0 1 2 1 2 0 0 0 1 (53.26 KiB) Viewed 29 times

Consider a Markov chain with state space S {1,2,...,5} the following one-step transition matrix: 0 0 P= 0 1/2 1/2 0 0 0 0 0 1/2 0 1/2 1/2 1/2 0 0 1/2 0 0 0 0 0 1/2 1 (a) Let ki = E; [T5] where, for each y ES, Ty min{n > 1: Xn = y). By using a first step analysis, show that 1 1 k3 k4. Determine the corresponding equations for the other ki. Hence determine the numer- ical value of ki. (b) Suppose Xo = 3. What is the probability that the Markov chain hits state 5 before it hits state 1?