Question

Create a new wiki entry from a peer-reviewed research paper that pertains subject below, or provide a summary or substantive commentary on an existing WIKI entry from a classmate.

Admin Notes: Conduct your own research and follow post a short relevant summary of your findings. ( Post current information, not older than five years ). Use not more than three (3) references.

Remember to place your name in your paper

Topic :

Intro to Data Mining Chapter 4 Classification: Basic Concepts.

Task : Select 2 and provide samples of the followings Models for Evaluation :

Holdout

Random subsampling

Cross validation

Stratified sampling

Bootstrap

Random subsampling

Answer 1

The first step in developing a machine learning model is training and validation. In order to train and validate a model, you must first partition your dataset, which involves choosing what percentage of your data to use for the training, cross-validation, and holdout/test sets.

Cross-validation

For a prediction problem, a model is generally provided with a data set of known data, called the training data set, and a set of unknown data against which the model is tested, known as the test data set. The target is to have a data set for testing the model in the training phase and then provide insight on how the specific model adapts to an independent data set. A round of cross-validation comprises the partitioning of data into complementary subsets, then performing analysis on one subset. After this, the analysis is validated on other subsets (testing sets). To reduce variability, many rounds of cross-validation are performed using many different partitions and then an average of the results are taken. Cross-validation is a powerful technique in the estimation of the model performance technique.

Holdout/Test Set

Sometimes referred to as “testing” data, the holdout subset provides a final estimate of the machine learning model’s performance after it has been trained and validated. Holdout sets should never be used to make decisions about which algorithms to use or for improving or tuning algorithms.