Question

Assignment 2 by Dr Dallas to be submitted next week.
1. Describe briefly each of the model performance evaluation techniques; log-likelihood, R Square, Aikake Information criterion, Bayesian Information Criterion
provide their formulae and discuss how you can interpret in each cases.

Answer 1

Solution:

Loglikelihood

In most cases, for various reasons, but often computational convenience, we work with the loglikelihood

l(θ|x)=logL(θ|x)

which is defined up to an arbitrary additive constant.

For example, the binomial loglikelihood is

l(π|x)=xlogπ+(n−x)log(1−π).

In many problems of interest, we will derive our loglikelihood from a sample rather than from a single observation. If we observe an independent sample x1,x2,...,xn from a distribution f(x|θ), then the overall likelihood is the product of the individual likelihoods:

i=1 and the loglikelihood is:

R Square:

R-squared (R²) is a statistical measure that represents the proportion of the variance for a dependent variable that's explained by an independent variable or variables in a regression model. Whereas correlation explains the strength of the relationship between an independent and dependent variable, R-squared explains to what extent the variance of one variable explains the variance of the second variable. So, if the R² of a model is 0.50, then approximately half of the observed variation can be explained by the model's inputs.

The Formula For R-Squared Is:

ErplainedVariance R2-1 TotalV atiance

Aikake Information criterion:

Akaike’s information criterion (AIC) compares the quality of a set of statistical models to each other. For example, you might be interested in what variables contribute to low socioeconomic status and how the variables contribute to that status. Let’s say you create several regression models for various factors like education, family size, or disability status; The AIC will take each model and rank them from best to worst. The “best” model will be the one that neither under-fits nor over-fits.

Although the AIC will choose the best model from a set, it won’t say anything about absolute quality. In other words, if all of your models are poor, it will choose the best of a bad bunch. Therefore, once you have selected the best model, consider running a hypothesis test to figure out the relationship between the variables in your model and the outcome of interest.

Calculations

Akaike’s Information Criterion is usually calculated with software. The basic formula is defined as:
AIC = -2(log-likelihood) + 2K
Where:

• K is the number of model parameters (the number of variables in the model plus the intercept).
• Log-likelihood is a measure of model fit. The higher the number, the better the fit. This is usually obtained from statistical output.

Bayesian Information Criterion:

The Bayesian Information Criterion (BIC) is an index used in Bayesian statistics to choose between two or more alternative models.

The BIC is also known as the Schwarz information criterion (abrv. SIC) or the Schwarz-Bayesian information criteria. It was published in a 1978 paper by Gideon E. Schwarz, and is closely related to the Akaike information criterion (AIC) which was formally published in 1974.

Definition of the Bayesian Information Criterion / Schwarz Criterion

The Bayesian Information Criterion (BIC) is defined as

k log(n)- 2log(L(θ̂)).

Here n is the sample size; the number of observations or number of data points you are working with. k is the number of parameters which your model estimates, and θ is the set of all parameters.

L(θ̂) represents the likelihood of the model tested, given your data, when evaluated at maximum likelihood values of θ. You could call this the likelihood of the model given everything aligned to their most favorable.

Another way of understanding L(θ̂) is that it is the probability of obtaining the data which you have, supposing the model being tested was a given.