Question

Suppose that the data for analysis includes the attribute age. The age values for the data tuples are (in increasing order) 13, 15, 16, 16, 19, 20, 20, 21, 22, 22, 25, 25, 25, 25, 30,33, 33, 35, 35, 35, 35, 36, 40, 45, 46, 52, 70.

a.Use smoothing by bin means to smooth these data, using a bin depth of 3. Illustrate your steps. Comment on the effect of this technique for the given data?

b.How might you determine outliers in the data?

c.What other methods are there for data smoothing?

Answer 1

Answers:

Given data contains attribute age values. The given age values for the data tuples are

13,15,16,16,19,20,20,21,22,22,25,25,25,25,30,33,33,35,35,35,35,36,40,45,46,52,70.

The total number of age values of tuples is 27.

a).Use smoothing by bin means to smooth these data, using a bin depth of 3. Illustrate your steps. Comment on the effect of this technique for the given data?

Answer:

Smoothing by bin means: In this method each value in a bin is replaced by the mean value of a bin.

Given depth of bin = 3.

We have to smoothing the given data using bin means method.

The following steps are used to smooth the given data. The given bin depth is 3.

Steps:

Step 1: Sort the data (the given data is already sorted ).

13,15,16,16,19,20,20,21,22,22,25,25,25,25,30,33,33,35,35,35,35,36,40,45,46,52,70.

Step 2: Partitioning the given data into equal frequency of bin size i.e. 3.

Total number of bins = number of tuples / depth of bin

                                = 27 / 3

                                = 9

Bin 1 : 13,15,16                                 Bin 2 : 16,19,20                                 Bin 3 : 20,21,22                    

Bin 4 : 22,25,25                                 Bin 5 : 25,25,30                                 Bin 6 : 33,33,35                    

Bin 7 : 35,35,35                                 Bin 8 : 36,40,45                                 Bin 9 : 46,52,70

Step 3: Calculate the arithmetic mean of each bin.

Calculating smoothed values for bins

Bin 1 = (13+15+16)/3 = 44/3 = 14.67

Bin 2 = (16+19+20)/3 = 55/3 = 18.33        

Bin 3 = (20+21+22)/3 = 63/3 = 21  

Bin 4 = (22+25+25)/3 = 72/3 = 24

Bin 5 = (25+25+30)/3 = 80/3 = 26.67       

Bin 6 = (33+33+35)/3 = 101/3 = 33.67

Bin 7 = (35+35+35)/3 = 105/3 = 35

Bin 8 = (36+40+45)/3 = 121/3 = 40.33

Bin 9 = (46+52+70)/3 = 168/3 = 56

Step 4: Replace the each in each bin by the arithmetic mean of calculated for the bin.

Bin	Values	Smoothed values
1	13,15,16	14.67,14.67,14.67
2	16,19,20	18.33,18.33,18.33
3	20,21,22	21,21,21
4	22,25,25	24,24,24
5	25,25,30	26.67,26.67,26.67
6	33,33,35	33.67,33.67,33.67
7	35,35,35	35,35,35
8	36,40,45	40.33,40.33,40.33
9	46,52,70	56,56,56

b. How might you determine outliers in the data?

Determining the outliers in the data:

Outlier is defined as it as an observation point that is distant from other observations.

Outliers in the data can be identified in many ways:

1. The given data can be clustering the data into groups. Here any data do not fall in any group can be taken as a outliers.
2. Any data points that are deviate (based on some threshold value) from the model can be considered as a outliers.
3. By dividing the given data into equal width of histograms and identifying the outlying histograms.

c. What other methods are there for data smoothing?

The other methods for the data smoothing is:

Smoothing by bin boundaries and Smoothing by bin medians.

Smoothing by bin boundaries

In this method we need to pick the minimum and maximum value, put the minimum value in left side and maximum value on the right side. Middle values in bin boundaries are move to its closet neighbor value with less distance.

Example:

Bin: 8, 9, 15, 17

Here 8 is the minimum value and 17 is the maximum value. 9 is the nearer to 8 and 15 is nearer to 17.so 9 is treated as 8 and 15 as treated as 17.

After bin boundary bin is    Bin: 8, 8, 17, 17.

Methods other than binning are regression techniques and classification techniques.

Regression techniques are used to smooth the data by fitting into a function such as linear or multiple regressions.

Classification techniques are used to be implementing the concept hierarchies that can smooth the data by rolling up lower level concepts to higher level concepts.