1. Let {Xn, n 2 0 be a Markov Chain with state space S. Show that...
1. Let {Xn,n2 0 be a Markov Chain with state space S. Show that 20 for any n,m-1 and JAn+m , . . . , İn+1,in-1, . . . , io є s.
Let X0,X1,... be a Markov chain whose state space is Z (the
integers).
Recall the Markov property: P(Xn = in | X0 = i0,X1 = i1,...,Xn−1
= in−1) = P(Xn = in | Xn−1 = in−1), ∀n, ∀it. Does the following
always hold: P(Xn ≥0|X0 ≥0,X1 ≥0,...,Xn−1 ≥0)=P(Xn ≥0|Xn−1 ≥0)
?
(Prove if “yes”, provide a counterexample if “no”)
Let Xo,Xi, be a Markov chain whose state space is Z (the integers). Recall the Markov property: P(X,-'n l Xo-io, Xi...
5. Let X n 2 0} be a Markov chain with state space S = {0,1,2,...}. Suppose P{Xn+1 = 0|X,p = 0 3/4, P{Xn+1 = 1\Xn, P{Xn+1 = i - 1|X, 0 1/4 and for i > 0, P{X+1 = i + 1|X2 = i} i} 3/4. Compute the long run probabilities for this Markov chain = 1/4 and =
Q5. Consider a Markov chain {Xn|n ≥ 0} with state space S = {0, 1, · · · } and transition matrix (pij ). Find (in terms QA for appropriate A) P{ max 0≤k≤n Xk ≤ m|X0 = i} . Q6. (Flexible Manufacturing System). Consider a machine which can produce three types of parts. Let Xn denote the state of the machine in the nth time period [n, n + 1) which takes values in {0, 1, 2, 3}. Here...
5. Let (Xn)n be a Markov chain on a state space S with n-step transition probabilities PTy = P(X,= y|Xo = x). Define (n) N x Xn=r n0 and U(G,) ΣΡ. n0 Show that (a) U(x, y)ENy|Xo= x] and (b) U(a, y) P(T, < +o0|X0= x)U(y, y), where Ty = inf {n 2 0 : X y}.
A Markov chain {Xn, n ≥ 0} with state space S = {0, 1, 2} has transition probability matrix 0.1 0.3 0.6 p = 0.5 0.2 0.3 0.4 0.2 0.4 If P(X0 = 0) = P(X0 = 1) = 0.4 and P(X0 = 2) = 0.2, find the distribution of X2 and evaluate P[X2 < X4].
Suppose Xn is a Markov chain on the state space S with transition probability p. Let Yn be an independent copy of the Markov chain with transition probability p, and define Zn := (Xn, Yn). a) Prove that Zn is a Markov chain on the state space S_hat := S × S with transition probability p_hat : S_hat × S_hat → [0, 1] given by p_hat((x1, y1), (x2, y2)) := p(x1, x2)p(y1, y2). b) Prove that if π is a...
2. The Markov chain (Xn, n = 0,1, 2, ...) has state space S = {1, 2, 3, 4, 5} and transition matrix (0.2 0.8 0 0 0 0.3 0.7 0 0 0 P= 0 0.3 0.5 0.1 0.1 0.3 0 0.1 0.4 0.2 1 0 0 0 0 1 ) (a) Draw the transition diagram for this Markov chain.
A Markov chain {Xn, n ≥ 0} with state space S = {0, 1, 2, 3, 4,
5} has transition probability matrix P.
ain {x. " 0) with state spare S-(0 i 2.3.45) I as transition proba- bility matrix 01-α 0 0 1/32/3-3 β/2 0 β/2 0 β/2 β/21/2 0001-γ 0 0 0 0 (a) Determine the equivalence classes of communicating states for any possible choice of the three parameters α, β and γ; (b) In all cases, determine if...
6. Suppose Xn is a two-state Markov chain with transition probabilities (Xn, Xn+1), n = 0, 1, 2, Write down the state space of the Markov chain Zo, Zi, . . . and determine the transition probability matrix.