Question

J This question relates to the idea of maximum likelihood estimation (MLE). MLE is a commonly used method in statistics, if not a cornerstone, that finds estimates of model parameters by answering the question, "given some observed data, what are the parameter estimates that maximise the likelihood (chance) of observing that data in the first place?" To provide an example, if we observe the values 2.6, 3.2 and 5.1 assumed to be drawn independently from the same distribution, it is more likely that the me an of the distribution is 3.3 than a mean of -10.3, say. MLE is a way of finding an estimated mean that maximises the probability of observing the three values provided In constructing statistical models, we assume observations are random variables from probability distributions, e.g. the Normal distribution which is charac- terised by the probability density function, p(z) = (2x02)-1/2 exp{-(x-4)?/202} > 0 for all r. From this assumption, it is possible to specify a function that represents the combined probability of observing all the data (the likelihood function). The likelihood function is a function of both the model parameters (think variables) and the data itself (think constants). Maximising the likelihood function with respect to the model parameters provides the parameter estimates that make the observed data most likely Now we have covered some background to the relevance of this question (that will become apparent later in your BCA program), suppose that a likelihood function f(0) is differentiable with f(0) > 0 for all 0 and let h(e) be any strictly increasing function with g(e) h(f(0)). (a) Show that f(0) and g(ø) have stationary points at the same values of 0. (b) Show that the stationary points of f(0) and g(0) share the same character, that is, a relative maximum of f(e) is also a relative maximum of g(e) and likewise for a relative minimum (c) Consider the functions P(z,u,0)= (2x02)-1/2 exp-(-/ and 9(z,j4,0)= In p(r,u, o) Find the partial derivatives of p(,4, 0) and g(r,u,0) with respect to Hence, provide a reason why we might prefer to find stationary points of 9(r,H, o) rather than p(r, 4, 0') with respect to u

Answer 1

The answers are given part by part.

Answer a) & (0) is a likelihood funetion with f(o) differentiable and f(0) >o for all o. h (e) be a strictly increasing function with g (0) 2 h (f (0)) Suppore, oo is a stationary point of f. => f (0.) = 0 bifferentiation Now, g'(0) - / h (f(0) 10:00 - h' (f (00). f (0o) [By Chain rule of. So, f' (60) = 0 = h' (f (0)) + (60) = 0 g (oo) on Again, g'lo) = 0 = h' (f (60)) & (0o) = 0 Now, his a strictly increasing funetion. Hence h'(o) o te » h' (f (6.)> o ) f'(6) 20. So, s' (60) =0 g'(o) = o.

point of 4 (0 it and only " Oo is a stationary if co is a stationary point of glo. - stationary points f (0) and g(0) have Sare values. o is a relative maximum of flo. Suppore, = f (0) 0 f'(o) = 0 and f" (100) Lo. g (0o) = 0 (prowed in part (0) 9" (0) - g' (0) / 6.00 - 7 h (f (0)) I' (6) 600 = h ( f (0)) f' (60) + n (+(60)) & " (80) = h' (f (0)) + " (6+) [f' (100)=0] Now, his strictly increasing = h' (f((.)) > 0. So, 9" (60) Lo [since t" (160) Lo] 2 g (6)=0 and 9" (0) Lo relative maximun ut a. 2 Co is a sinilasly g'(o) = 0 2) f'((o)=0 [By past (G)] and 9" (60) Lo 2) h ((0.))."(c) Lo 2) 8" ((o) Lo [n' (f ((.)) > 0] 2) S (0o) = 0 and f" (lo) Lo - Co is a relative maximum of f.

have relathe maximuns at athe the f and og have sare points. In sinilar approach, we can show that f and a have relative minimuns at same points, In this case we can show, {f (60)=0, f "(s) >0} > (6) -0,9" ) > I expf(x-47/2027 p(a, y,0) = (2902) 2) g(2, 4, 5) = log p (a, xo) -log(3053) - Army Silog abalogat logo logbe a clogo loetles on Cam partial derivatives : a ll exp] - (0-3) 1202 ap (avo (2 to 2 1/2 exp I - (ans) .-2169-)..6-) o exo-ca ]

2 g (1,4, of oto – 2a-4) (-1) au [In this calculation, alog Rai " Hence, it can be easily observed that, finding the partial derivative of g (a, e, o it muewor.t. in is much more simpler the simple than finding the partial derivative of p(a, y ,o) w.rot. n. Hence, to sind mle, it is advisable to find the stationary point of g.