Question

Suppose that we have a finite irreducible Markov chain Xn with stationary distribution π on a state space S. (a) Consider the sequence of neighboring pairs, (X0, X1), (X1, X2), (X2, X3), . . . . Show that this is also a Markov chain and find the transition probabilities. (The state space will be S ×S = {(i,j) : i,j ∈ S} and the jumps are now of the form (i, j) → (k, l).) (b) Find the stationary distribution for the new Markov chain.

Suppose that we have a finite irreducible Markov chain Xn with stationary distribution T on a state space S. (a) Consider the sequence of neighboring pairs, (Xo, X1), (X1,X2), (X2, X3), .... Show that this is also a Markov chain and find the transition probabilities. (The state space will be S x S = {(i,j) : i,j e S} and the jumps are now of the form (i, j)(k, l).) (b) Find the stationary distribution for the new Markov chain

Please answer it as soon as possible thanks due in 12 hours

Answer 1

(a) To say that the chain (Xo, X1), (Xi,X2), (X2,X3),... is a Markov Chain we need to show that it follows Markov Property i.e P(Xn+1= k,Xn= [(X0, X1), .. . , (X-1, Xn)) = P(X+1 = k, Xn = 1|(X,-1,X) (1) Now note the following: LHS of (1) P(Xn+1 = k, Xn = |Xo, X1, ... . X,) P(X+1Xo, X1, ... , Xn)P(X, = 1X0, X1,... , X,) P(X+1 k|Xn = l) = P(Xn+1 = k|Xn = l, Xn-1 = i)P(X, = 2|X, = 1, X,,-1 = i) P(X,1k, X,, = lU(X,-1, X)) RHS of (1 Hence it is a Markov Chain with transition probabilities if kj P/t P)kt) Otherwise Where P is the transition probability for the Markov Chain X, from state j to (b) Let the stationary distribution be å. Then t( Στ0) Poa Now if S ,4), then from the above equation S-ΣΤα) ΣΣ103ΡΑ j -Σ5.Pi Now observe that the solution of this equation is r equation of stationary distribution of X). Hence, Sk = m ¥ k. Also from the equation above the previous one, k!) = P1;*(i,k) = PriSk = PiTk (As this represents the