Question

3. Consider a labeled data set containing 100 data instances which are randomly partitioned into two sets A and B, each containing 50 instances. We use A as the training set to learn two decision trees T10 with 10 leaf nodes and T100 with 100 leaf nodes. The accuracies of the two decision trees on data sets A and B are shown below: Data Set T100 А. T10 0.86 0.84 B 0.97 0.77 (a) Based on the accuracies shown in the table above, which classification model would you expect to have better performance on unseen instances? (b) Now you've tested T10 and T100 on the entire dataset (A + B) and found that the classification accuracy of T10 on the data set (A + B) is 0.85, whereas the classification accuracy of T 100 on the data set (A + B) is 0.87. Based on this new information and your observations from the table above, which classification model would you finally choose for classification?

Answer 1

Answer:

a) We will choose T10 on unseen data. This is because it has a better accuracy on unseen dataset. It also does not over fit to noise of training dataset unlike T100 which captures even noise on the training dataset to yield an accuracy of 0.97. b) The new information is nothing but just the avergae accuracies on train and test data. The umbers obtained are avergae of earlier numbers. This essentially again means that on an unseen dataset we prefer amodel which performs better on the unseen portion (B). Hence our decision will be the same. We would choose T10 on unseen dataset