Question

fish F 5. Consider the following dataset where the decision attribute is restaurant: mealPreference gender drinkPreference restaurant hamburger M coke mcdonalds M pepsi burgerking chicken coke mcdonalds hamburger coke mcdonalds chicken pepsi wendys fish F coke burgerking chicken M pepsi burger King chicken IF coke wendys fish coke mcdonalds hamburger coke mcdonalds IM M F If we want to make a decision tree for determining restaurant, we must decide which of the three non-decision attributes (mealPreference, gender, or drinkPreference) to use as the root of the tree. a. Set up the equation to compute what in lecture we called entropy Before Split for restaurant. You do not have to actually solve (i.e., calculate the terms in the equation, just set up the equation with the appropriate values. (2 pts.) b. Set up the equation to compute entropy for mealPreference when its value is chicken. That is, a tree with mealPreference at the root would have three branches (one for hamburger, one for chicken, and one for fish), requiring us to compute entropyHamburger, entropy Chicken, and entropy Fish; here we only want you to set up the equation to compute entropyChicken. You do not have to actually solve (i.e., calculate the terms in the equation, just set it up using the appropriate values. (2 pts.) c. Suppose that instead of considering mealPreference to be the root of this decision tree, we had instead considered drinkPreference. Set up the equation to compute information gain for drinkPreference given the variables specified below. (2 pts.) entropy before any split: entropy for drinkPreference = pepsi: entropy for drinkPreference = coke: XAU

Answer 1

Preliminary Theory Concepts

Entropy is the measure of the homogenity of the set of example.Entropy is 0 when all the cases of the sample set belong to one particular class which means totally random classification.Entropy is 1 when all the classes contain equal number of cases of the sample set which means perfectly classified.

The formula used to calculate the same is Entropy(S) = -p_plog₂p_p - p_nlog₂p_n where -

p_p dentoes the proportion of examples belonging to class 1 in S

p_ndentoes the proportion of negative examples belonging to class 2 in S

Similarly it can be extended to n number of classes

Information Gain measures the exepected reduction in entropy caused by partitioning the samples according to a particular attribute.

Information Gain = Entropy before - Entropy After

a.

Total number of samples - 10

Proportion of samples preferring McDonalds:p_m - $\frac{5}{10}$

Proportion of samples preferring Burger King:p_b - $\frac{3}{10}$

Proportion of samples preferring Wendys:p_w - $\frac{2}{10}$

Thus the entropy referring to the above three classes will be -p_mlog₂p_m - p_blog₂p_b -p_wlog₂p_w

Substituting the above values the entropyBeforeSplit is: -$\frac{5}{10}$log₂$\frac{5}{10}$- $\frac{3}{10}$ log₂$\frac{3}{10}$ -$\frac{2}{10}$log₂$\frac{2}{10}$

b.

Total number of samples having chicken as meal preference : 4

When chicken is the meal preference the proprtion of people going to Mcdonalds : p_cm =

14

When chicken is the meal preference the proprtion of people going to Burger King : p_cb=

14

When chicken is the meal preference the proprtion of people going to Wendys : p_cw = $\frac{2}{4}$

entropyChicken = -p_cmlog₂p_cm - p_cblog₂p_cb -p_cwlog₂p_cw

Substituting the above values the entropyChicken is: -
₁₄
log_2
14
-
14
log₂
14
-$\frac{2}{4}$log₂$\frac{2}{4}$

c.

Information Gain = Entropy_before_split - Entropy_after_split

Entropy_before_split = X

Entropy_after_split = -(Proprtion of consumers preferring pepsi)* (entropy for drink Preference Pepsi) - (Proprtion of consumers preferring coke)* (entropy for drink Preference c)

Entropy_after_split = ($\frac{4}{10}$ * P) + ($\frac{6}{10}$ * C)

Information gain for Drink Preference : X - ($\frac{4}{10}$ * P) - ($\frac{6}{10}$ * C)