5. Consider a Markov chain (Xn) with the state space S = {0, 1, 2,...} of non-negative integers and the one-step transition probabilities given by p(i, i + 1) =p and p(i, 0) = 1-p for i > 0, where p= (0, 1). (a) Identify the closed irreducible sets. (b) Is the Markov chain aperiodic? (c) Is there a stationary distribution? Is it unique? Compute the stationary distribution if it exists. (d) Find the transient states and recurrent states. (e) Compute E; [Ti] = E[Ti|Xo = i] for each i≥ 0, where T; is the (usual) first return time.

5. Consider a Markov chain (Xn) with the state space S = {0, 1, 2,...} of non-negative integers and the one-step transition probabilities given by p(i, i + 1) =p and p(i, 0) = 1-p for i > 0, where p= (0, 1). (a) Identify the closed irreducible sets. (b) Is the Markov chain aperiodic? (c) Is there a stationary distribution? Is it unique? Compute the stationary distribution if it exists. (d) Find the transient states and recurrent states. (e) Compute E; [Ti] = E[Ti|Xo = i] for each i≥ 0, where T; is the (usual) first return time.

Linear Algebra: A Modern Introduction

4th Edition

ISBN:9781285463247

Author:David Poole

Publisher:David Poole

Chapter3: Matrices

Section3.7: Applications

Problem 12EQ: 12. Robots have been programmed to traverse the maze shown in Figure 3.28 and at each junction...

See similar textbooks