Question

Assuming the input is a large collection of simple text documents, and the following MyMapper and MyReducer (written in Hadoop) will be applied to these text documents:

public static class MyMapper extends Mapper<Object, Text, IntWritable, Text> {

    private Text word = new Text();

public void map(Object key, Text value, Context context)

throws IOException, InterruptedException {     

      StringTokenizer itr = new StringTokenizer(value.toString());

      while (itr.hasMoreTokens()) {

      String currentWord = itr.nextToken();

      word.set(currentWord);

      context.write(new IntWritable(currentWord.length()), word);

      }

    }

}

public static class MyReducer

       extends Reducer<IntWritable,Iterable<Text>,IntWritable, IntWritable> {

    private IntWritable result = new IntWritable();

public void reduce(IntWritable key, Iterable<Text> values, Context context)

throws IOException, InterruptedException {

      int sum = 0;

      for (Text val : values) { sum ++; }

      result.set(sum);

      context.write(key, result);

    }

}

1.1 (5 points) What is the goal of the above mapper and reducer? answer very briefly

1.2 (3 points) For an input text document which has the following content,

one of the input text file

write the output from the mapper:

1.3 (4 points) write the result of the grouping stage between mapper and reducer

1.4 (2 points) write the output from the reducer:

Answer 1

Answer:

(1.1) The goal of the above mapreduce program is to "Word Count".

Explanation:

Record Reader
-------------
This is the first phase of MapReduce where the Record Reader reads every line from the input text file as text and yields output as key-value pairs.

Map Phase
----------
The Map phase takes input from the Record Reader, processes it, and produces the output as another set of key-value pairs.

The Map phase reads each key-value pair, divides each word from the value using StringTokenizer, treats each word as key and the count of that word as value.

Combiner Phase
---------------
The Combiner phase takes each key-value pair from the Map phase, processes it, and produces the output as key-value collection pairs.

The Combiner phase reads each key-value pair, combines the common words as key and values as collection. Usually, the code and operation for a Combiner is similar to that of a Reducer.

Reducer Phase
-------------
The Reducer phase takes each key-value collection pair from the Combiner phase, processes it, and passes the output as key-value pairs. Note that the Combiner functionality is same as the Reducer.

Record Writer
-------------
This is the last phase of MapReduce where the Record Writer writes every key-value pair from the Reducer phase and sends the output as text.

(1.2) Given input text document cntains

"one of the input text file"

Output:
-------
<one,1> <of,1> <the,1> <input,1> <text,1> <file,1>

(1.3) Combiner Phase output:

<one,1> <of,1> <the,1> <input,1> <text,1> <file,1>

(1.4) Reducer Otuput:
<one,1> <of,1> <the,1> <input,1> <text,1> <file,1>

In case of any queries please mention in the comment section.

Please do UpVote it!

Thank you!