Question

Suppose Yį = Bo + Bidi + Ei and εi ~ N(0,0%). Also suppose you have n observations of (yi, x;). Consider xi just a set of constants. (a) What is the distribution of Y;? (b) If you wanted to estimate Bo, B1, and oʻ, what likelihood function would you maximize?

Answer 1

a) Given that $\epsilon_i\sim N(0,\sigma^2)$ is normally distributed, we can say that the expected value is

$E(\epsilon_i)=0$ and Variance $Var(\epsilon_i)=\sigma^2$

Since $\beta_0+\beta_1x_i$ is a constant, any linear combination of constants with $\epsilon_i$ will also be normally distributed.

Hence, $Y_i$ is normally distributed

with mean

$\begin{align*} E(Y_i)&=E(\beta_0+\beta_1x_i+\epsilon_i)\\ &=\beta_0+\beta_1x_i+E(\epsilon_i)\quad\text{using the formula }E(aX+b)=aE(X)+b\\ &=\beta_0+\beta_1x_i+0\\ &=\beta_0+\beta_1x_i \end{align*}$

and variance

$\begin{align*} Var(Y_i)&=Var(\beta_0+\beta_1x_i+\epsilon_i)\\ &=Var(\epsilon_i)\quad\text{using the formula }Var(aX+b)=a^2Var(X)\\ &=\sigma^2\\ \end{align*}$

ans: $Y_i\sim N(\beta_0+\beta_1x_i,\sigma^2)$

b) Since $Y_i$ is normally distributed with mean $\mu_i=\beta_0+\beta_1x_i$ and variance $\sigma_i^2=\sigma^2$

we can write the pdf of $Y_i$ using the normal distribution as

$f(y_i\mid \beta_0,\beta_1,\sigma^2)=\frac{1}{\sqrt{2\pi\sigma_i^2}}e^{-\frac{(y_i-\mu_i)^2}{2\sigma_i^2}}=\frac{1}{\sqrt{2\pi\sigma^2}}e^{-\frac{(y_i-\beta_0-\beta_1x_i)^2}{2\sigma^2}}$

Assuming that $Y_1,Y_2,...,Y_n$ are independent, we can write the joint pdf of $Y_1,Y_2,...,Y_n$ as the product of marginal pdfs

$\begin{align*} f(y\mid \beta_0,\beta_1,\sigma^2)&=f(y_1,...,y_n\mid \beta_0,\beta_1,\sigma^2)\\ &=\prod_{i=1}^nf(y_i\mid \beta_0,\beta_1,\sigma^2)\quad\text{as }Y_is\text{ are independent}\\ &=\prod_{i=1}^n\frac{1}{\sqrt{2\pi\sigma^2}}e^{-\frac{(y_i-\beta_0-\beta_1x_i)^2}{2\sigma^2}}\\ &=\frac{1}{(2\pi\sigma^2)^{n/2}}e^{-\frac{\sum_{i=1}^n(y_i-\beta_0-\beta_1x_i)^2}{2\sigma^2}} \end{align*}$

The above joint pdf is the likelihood function written as

$\begin{align*} L(\beta_0,\beta_1,\sigma^2\mid x,y)=f(y\mid \beta_0,\beta_1,\sigma^2 ) \end{align*}$

ans: If we want to estimate $\begin{align*} \beta_0,\beta_1,\sigma^2 \end{align*}$ , the likelihood function we would maximize is

$\begin{align*} L(\beta_0,\beta_1,\sigma^2\mid x,y)=\frac{1}{(2\pi\sigma^2)^{n/2}}e^{-\frac{\sum_{i=1}^n(y_i-\beta_0-\beta_1x_i)^2}{2\sigma^2}} \end{align*}$