Question

Please answer the following conceptual questions about classification. Assume we have two classes and one predictor, i.e., p = 1 and K = 2.

(1) What is Bayesian Classifier? How does it make prediction?

(2) What is the general idea of logisitic regression? How does it make prediction?

(3) What is the general idea of LDA? How does it make prediction?

Answer 1

Naive Bayes Classifiers

This article discusses the theory behind the Naive Bayes classifiers and their implementation.

Naive Bayes classifiers are a collection of classification algorithms based on Bayes’ Theorem. It is not a single algorithm but a family of algorithms where all of them share a common principle, i.e. every pair of features being classified is independent of each other.

To start with, let us consider a dataset.

Consider a fictional dataset that describes the weather conditions for playing a game of golf. Given the weather conditions, each tuple classifies the conditions as fit(“Yes”) or unfit(“No”) for plaing golf.

Here is a tabular representation of our dataset.

The fundamental Naive Bayes assumption is that each feature makes an:

• independent
• equal

Logistic regression is a statistical method for analyzing a dataset in which there are one or more independent variables that determine an outcome. The outcome is measured with a dichotomous variable (in which there are only two possible outcomes).

One of the most common methods to solve for Binary Classification is called Logistic Regression. The goal of Logistic Regression is to evaluate the probability of a discrete outcome occurring, based on a set of past inputs and outcomes. steps are :

Using Logistic Regression to Predict Probabilities

Classifying Binary Outcomes With a Decision Boundary

Measuring the Accuracy of Different Decision Boundaries

LDA

In natural language processing, latent Dirichlet allocation (LDA) is a generative statistical model that allows sets of observations to be explained by unobserved groups that explain why some parts of the data are similar.

This determines how the parameter estimation is handled. With "plug-in" (the default) the usual unbiased parameter estimates are used and assumed to be correct. With "debiased" an unbiased estimator of the log posterior probabilities is used, and with "predictive" the parameter estimates are integrated out using a vague prior.