Question

SEMMA and CRISP-DM are two methodologies used in the domain of data mining. In your research on these methodologies you have probably noted some similarities in terms of phases represented in both approaches. Do you think that there could be scenarios where a hybrid methodology that uses both these approaches would be beneficial? Why? Describe the scenario and share your opinion/justification.

Answer 1

Data Mining refers to the process of discovering the patterns and trends in data in order to predict future results. There are two most widely recognized methodologies in the field of the Data Mining and they are SEMMA (Sample, Explore, Modify, Model and Assess) and CRISP-DM ( Cross Industry Standard Process for Data Mining).

SEMMA was proposed by the SAS institute which is one of the most popular companies that develop statistical software applications with their software package - Enterprise Miner. This methodology starts with the collection and sampling of the data followed by exploring the data in order to identify the trends and anomalies in the data. This is carried out in order to gain some information about the dataset. In the following step, the data is modified to create, select and transform the variables for the study. In the next step, the data is used to create a model that can be used for predictive analysis. In the end, the methodology involves evaluating the performance of the model. Although this methodology covers the statistical aspects of any data mining applications it does not take care of the implementation, design and analysis phases. It is application dependent and works well for the SAS Enterprise Miner software. This is because this methodology was specifically developed for the SAS Enterprise Miner software.

CRISP-DM is one of the most widely used analytics model. It was conceived in 1996. It tries to break down the process of data mining into six phases. These six phases include Business Understanding, Data Understanding, Data Preparation, Modelling, Evaluation, Deployment. The main advantage of CRISP-DM is that it takes care of the business understanding part of any data mining project very well. This is significantly important as the SEMMA does not take the aspirations of the stakeholder organization into account. It is also an industry, tool, and application-independent methodology.

There might be a scenario when a research problem might not only require good data analytics processing but also has to care for the aspirations of the stakeholder organization. In such a case, it would be better to use a hybrid combination of both SEMMA and CRISP-DM. The SEMMA will take care of the analytics part such as Data Collection, Exploratory Data Analysis, Data Modeling, and Model Evaluation while the CRISP-DM might take care of the implementation and developmental aspects of the application. The role of CRISP-DM is very crucial here as the development of a Data Mining application is a cyclic process and the model has to train after certain intervals of the time in order to ensure the consistency of the Data Mining application, therefore the CRISP-DM provides the implementational superiority to the Data Mining application. Since the CRISP-DM is not software specific and hence it can be used along SEMMA to develop and deploy a Data Mining application to the client.

Here's the solution to your question. Thanks for asking and happy learning!!!