Question

Problem 8. Let XI.Ха» . . , Xn be independent samples taken from some distribution, and X 1:42 is the sample mean. Let random variable X have the same distribution. Prove that

Answer 1

a) starting with LHS

$\begin{align*} E\left[\frac{1}{n}\sum_{i=1}^n(X_i-\bar{X})^2 \right ]&= E\left[\frac{1}{n}\sum_{i=1}^n(X_i^2-2X_i\bar{X}+\bar{X}^2) \right ]\\ &=E\left[\frac{1}{n}\sum_{i=1}^nX_i^2-2\frac{1}{n}\sum_{i=1}^nX_i\bar{X}+\frac{1}{n}\sum_{i=1}^n\bar{X}^2 \right ]\\ &=E\left[\frac{1}{n}\sum_{i=1}^nX_i^2-2\bar{X}^2+\frac{1}{n}n\bar{X}^2 \right ]\\ &=E\left[\frac{1}{n}\sum_{i=1}^nX_i^2-\bar{X}^2\right ]\\ &=\frac{1}{n}E\left[\sum_{i=1}^nX_i^2\right ]-E(\bar{X})^2\\ &=\frac{1}{n}\sum_{i=1}^nEX_i^2-E(\bar{X})^2\\ &=\frac{1}{n}\sum_{i=1}^nEX^2-E(\bar{X})^2\quad\text{Since }X \text{ has same distribution as }X_i\\ &=\frac{1}{n}nEX^2-E(\bar{X})^2\\ &=EX^2-E(\bar{X})^2\\ \end{align*}$

b) starting with LHS

$\begin{align*} \sum_{i=1}^n(X_i-EX)^2&=\sum_{i=1}^n(X_i-\bar{X}+\bar{X}-EX)^2 \quad\text{add and subtract }\bar{X}\\ &=\sum_{i=1}^n\left[ (X_i-\bar{X})^2+2(X_i-\bar{X})(\bar{X}-EX)+(\bar{X}-EX)^2\right ]\\ &=\sum_{i=1}^n(X_i-\bar{X})^2+2(\bar{X}-EX)\sum_{i=1}^n(X_i-\bar{X})+\sum_{i=1}^n(\bar{X}-EX)^2\\ &\text{but }\sum_{i=1}^n(X_i-\bar{X})=0\\ &=\sum_{i=1}^n(X_i-\bar{X})^2+\sum_{i=1}^n(\bar{X}-EX)^2\\ &=\sum_{i=1}^n(X_i-\bar{X})^2+n(\bar{X}-EX)^2\\ \end{align*}$

Now it remains to be shown that $EX=E\bar{X}$ so that we can replace it in the above expression

$\begin{align*}E\bar{X}&=E\left[\frac{1}{n}\sum_{i=1}^nX_i \right ]\\ &=\frac{1}{n} E\left[\sum_{i=1}^nX_i \right ]\\ &=\frac{1}{n}\sum_{i=1}^nEX_i\quad\text{ we can replace }X_i\text{ by }X\text{ and get}\\ &=\frac{1}{n}\sum_{i=1}^nEX\\ &=\frac{1}{n}nEX\\ &=EX \end{align*}$

We can replace $EX=E\bar{X}$

Now back to the main derivation

$\begin{align*} \sum_{i=1}^n(X_i-EX)^2&=\sum_{i=1}^n(X_i-\bar{X})^2+n(\bar{X}-EX)^2\\ &=\sum_{i=1}^n(X_i-\bar{X})^2+n(\bar{X}-E\bar{X})^2\\ \end{align*}$