Question

11. Model selection and Prediction Error. (a) (2 marks) A sample and two fitted functions are displayed in the graphs below. Based on this which function do you prefer? Why? muhat (degree=2) on S muhat (degree=20) on S with ww 9 1850 1900 1950 2000 1850 1900 1950 2000 (b) (1 marks) Why are many samples preferred than a single sample when calculating the prediction error? (c) (3 marks) Describe k-fold cross-validation.

Answer 1

1)

Note the model in second graph might overfit the data and yield high prediction errors.

While the model in the first graph will do average in both training and testing set.

2)

Because more the data points better will be the training of the model being used and less will be the prediction error.

3)

Cross-validation is a resampling procedure used to evaluate machine learning models on a limited data sample. It has a parameter k. The method involves splitting data set into k parts randomly , parts being approximately of equal sizes, the model is tested on the first part and fitted in the remaining k-1 parts.