Question

My Professor of Stochastic Processes gave us this challenge to be able to exempt the subject, but I cant solve it.

Stochastic Processes TOPICS: Asymptotic Properties of Markov Chains May 25, 2019 1.Consider the stochastic process R-fRnh defined as follows: Where {Ynjn is a succession of random variable i.i.d (Independent random variables and identically distributed), with values in {1,2, ...^ with Ro 0 a) Why R is a Markov Chain? Find the state space of R b) Find the transition matrix of the Markov Chain and show that it is irreducible c) Suppose that P(Yi is aperiodic. j) > 0 , j E { 1, 2, },show that the Markov Chain d) ¿Does the Markov Chain have a stationary distribution? ¿Can we assure that the Markov Chain is positive recurrent irreducible? ¿Does the Markov Chain have asymptotic distribution? e) Show that E,-(16) = E(Y)

Answer 1

ANSWER a= R is a handy package called a Markov chain that can handle a vast array of Markov chain type.

the model is based on the finite set of all the possible states called the state space of Markov chain.

ANSWER d= yes the Markov chain have a unique stationary distribution, and absorbing Markov chain have stationary distribution with non zero elements only in absorbing states.

yes we can assume that Markov chain is positive recurrent irreducible .

yes for an irreducible and recurrent Markov chain, every non negative harmonic function is constant