Question

Let {Zn}n=0 be iid. Bernoulli random, vari- + Zn. ables with PZ-0] = p and PZ-1] 1-p. Define Sn-Zo + Which of the following processes is a Markov chain? 1. An S For each process that is a Markov chain, find its transition matriz. For each process Xn E An, Bn, Cn, Dn that is not a Markov chain, find a pair of states i and j so that P[Xn+ 1 = ilXn-j, Xn-1-k] depends on k.

Help please!

Answer 1

A discrete time process {Xn, n = 0, 1, 2, . . .} with discrete state space Xn ∈ {0, 1, 2, . . .} is a Markov chain if it has the Markov property: P[Xn+1= j|Xn= i, Xn−1= in−1, . . . , X0= i0] = P[Xn+1= j|Xn= i] • In words, “the past is conditionally independent of the future given the present state of the process” or “given the present state, the past contains no additional information on the future evolution of the system.” • The Markov property is common in probability models because, by assumption, one supposes that the important variables for the system being modeled are all included in the state space. • We consider homogeneous Markov chains for which P[Xn+1= j | Xn= i] = P[X1= j | X0= i]. 1 Example: physical systems. If the state space contains the masses, velocities and accelerations of particles subject to Newton’s laws of mechanics, the system in Markovian (but not random!) Example: speech recognition. Context can be important for identifying words. Context can be modeled as a probability distrubtion for the next word given the most recent k words. This can be written as a Markov chain whose state is a vector of k consecutive words.