Question

Suppose that X1, … Xn are sample from the following truncated Poisson distribution.

12:35 AM Fri Nov 29 100% 5 TORO+ 0 0 + Bo 2. Suppose that X ., X, are sampled from the following truncated Poisson distribution: PCX = r exp(-)" ! (r) for x=r +1,7 +2,... for some integer r > where *a(r) = Į PM) = 1 - . expe="JA" Such a sample might arise if we were sampling from a Poisson population but were unable to observe data less than or equal to r. The EM algorithm can be employed to estimate from the observed X1,...,x. The key is to think of the observed data as a subset of some larger ("complete") data set X1,..., X., X1+1,..., X+M where M > 0 is a random variable and X2+1,..., Xn+M <r; given M = m, this complete data set is now assumed to be m +n independent observations from a Poisson distribution with mean . The log-likelihood for the complete data is In 2(X) = ln(x) xi-(n + m)X, which depends on two unknowns ; and m. To use the EM algorithm, we need to i=n+1 estimate these two unknowns. (a) The probability distribution of M is P (M = m) = (" ")(1 - Ky(r))"K(r)" for m = 0,1,2,...

Answer 1

Expectation–maximization (EM) algorithm: It is an iterative method to find maximum likelihood or maximum a posteriori (MAP) estimates of parameters in statistical models, where the model depends on unobserved latent variables. The EM iteration alternates between performing an expectation (E) step, which creates a function for the expectation of the log-likelihood evaluated using the current estimate for the parameters, and a maximization (M) step, which computes parameters maximizing the expected log-likelihood found on the E step. These parameter-estimates are then used to determine the distribution of the latent variables in the next E step.

Given the statistical model which generates a set _{$\mathbf {X}$} of observed data, a set of unobserved latent data or missing values _{$\mathbf {Z}$} , and a vector of unknown parameters _{${\boldsymbol {\theta }}$} , along with a likelihood function,${\displaystyle L({\boldsymbol {\theta }};\mathbf {X} ,\mathbf {Z} )=p(\mathbf {X} ,\mathbf {Z} \mid {\boldsymbol {\theta }})}$, the maximum likelihood estimate (MLE) of the unknown parameters is determined by maximizing the marginal likelihood of the observed data

${\displaystyle L({\boldsymbol {\theta }};\mathbf {X} )=p(\mathbf {X} \mid {\boldsymbol {\theta }})=\int p(\mathbf {X} ,\mathbf {Z} \mid {\boldsymbol {\theta }})\,d\mathbf {Z} }$

However, this quantity is often intractable (e.g. if _{$\mathbf {Z}$} is a sequence of events, so that the number of values grows exponentially with the sequence length, the exact calculation of the sum will be extremely difficult).

The EM algorithm seeks to find the MLE of the marginal likelihood by iteratively applying these two steps:

Expectation step (E step): Define _{${\displaystyle Q({\boldsymbol {\theta }}\mid {\boldsymbol {\theta }}^{(t)})}$} as the expected value of the log likelihood function of _{${\boldsymbol {\theta }}$} , with respect to the current conditional distribution of _{$\mathbf {Z}$} given _{$\mathbf {X}$} and the current estimates of the parameters _{$\boldsymbol\theta^{(t)}$} :

${\displaystyle Q({\boldsymbol {\theta }}\mid {\boldsymbol {\theta }}^{(t)})=\operatorname {E} _{\mathbf {Z} \mid \mathbf {X} ,{\boldsymbol {\theta }}^{(t)}}\left[\log L({\boldsymbol {\theta }};\mathbf {X} ,\mathbf {Z} )\right]\,}$

Maximization step (M step): Find the parameters that maximize this quantity:

${\displaystyle {\boldsymbol {\theta }}^{(t+1)}={\underset {\boldsymbol {\theta }}{\operatorname {arg\,max} }}\ Q({\boldsymbol {\theta }}\mid {\boldsymbol {\theta }}^{(t)})\,}$

Deto I Page No Suppose that XX... XD axe Sample from truncated poisson distribution for sc = 8+18+2, (x; + x) = exp(-x X! Ka(x) too some integer $7,0 Where k(o) - 9 / exp(-x) x KA(0) = 1- & exp(-x) xx Che 109 - likelihood to the complete data is, In 1(x) = ln a 3 x; - (nom) x Which depends on two Unknown In) The probability distributions of Mis. D (=m) – (nem-1) (1 Kg (-2))"}, (3)" +08 m=0,1,2, TO calculate m=0 (num-1- ml.

Date : 1 Page No 5; CM) M(nou)! (1-; (*))" MCE)" (n-1)! m! (6-1)! M! in M-0 n+m-1)! (5-1)! (M E,(H)= (1-xx (x))"696) - E (M) = n (1-k (8) consider 1 = 1 ** ant = n lonka (8) KA (2) 1 (**) • E9 (11 (***) = 5(M) Ex(*;]*38) xix:*, - , Xn+#n) = ; (M) Ej (13) X5Y) Therefore E x = 3 10 ... < Xi int