Question

Assignment 3: Word Frequencies

1. Prepare a text file that contains text to analyze. It could be song lyrics to your favorite song.

2. With your code, you’ll read from the text file and capture the data into a data structure.

3. Using a data structure, write the code to count the appearance of each unique word in the lyrics.

4. Print out a word frequency list.

Example of the word frequency list:

100: frog
94: dog
43: cog
20: bog

Advice:

• You can find song lyrics at https://genius.com/

• Consider whether you want your keys to be case sensitive

• Consider whether you want to include common stopwords such as the, a, is (more can be found at https://www.ranks.nl/stopwords)

• Discuss with a classmate pro/cons of all steps if you’re rather indecisive

• Examples can be found under the Maven folder

• Examples of readme files github.com/matiassingers/awesome-readme

It is okay to start with the example code given in the repo, however please think about why you’re using whichever data structure you’ve selected for the task as you’ll have to write a description of your design rationale.

Example:

package myApp;

        import java.util.*;
        import java.io.*;

/**
 * Hello world!
 *
 */
public class WordCount {
    public static HashMap<String, Integer> sortByValue(HashMap<String, Integer> hm) {

        // Create a list from elements of HashMap
        java.util.List<Map.Entry<String, Integer> > list = new LinkedList<Map.Entry<String, Integer> >(hm.entrySet());

        // Sort the list
        Collections.sort(list, new java.util.Comparator<Map.Entry<String, Integer> >() {
            public int compare(Map.Entry<String, Integer> o1,
                               Map.Entry<String, Integer> o2) {
                return (o2.getValue()).compareTo(o1.getValue());
            }
        });

        // put data from sorted list to hashmap
        HashMap<String, Integer> temp = new LinkedHashMap<String, Integer>();
        for (Map.Entry<String, Integer> aa : list) {
            temp.put(aa.getKey(), aa.getValue());
        }
        return temp;
    }

    public static void main( String[] args ) throws FileNotFoundException  {

        // open the file
        Scanner console = new Scanner(System.in);
        String fileName = args[0];
        Scanner input = new Scanner(new File(fileName));

        // count occurrences
        HashMap<String, Integer> wordCounts = new HashMap<String, Integer>();
        while (input.hasNext()) {
            String next = input.next().toLowerCase();
            String clean = next;
            //next.replaceAll("\\p{Punct}", "").toLowerCase();

            if (!wordCounts.containsKey(clean)) {
                wordCounts.put(clean, 1);
            } else {
                wordCounts.put(clean, wordCounts.get(clean) + 1);
            }
        }


        System.out.println("Total words = " + wordCounts.size());

        HashMap<String, Integer> sortedMapAsc = WordCount.sortByValue(wordCounts);

        // Report frequencies
        for (String word : sortedMapAsc.keySet()) {
            int count = sortedMapAsc.get(word);
            System.out.println(count + ": " + word);
        }
    }
}

Answer 1

Data Structure Used: HASHMAP

Reason:

In hashmap, data is stored in key-value pairs, It uses hashing to store the data, No duplicate keys are allowed in HashMap

Hence, we can use it to keep the word as the KEY and increment the occurence of word as VALUE.

This is the best data Structure to use in this type of cases.

CODE:

import java.util.*;
import java.io.*;

public class Solution{

   private static HashMap<String, Integer> frequency = new HashMap<>();

   //function to add the data into HashMap
   //convert the words to upper case to remove case sensitivity, then replace all spaces with ',' and then split by ','
   //If word exists in the map, increase the count or add it to the map
   public static void countTheWords(String lyrics){

       String[] words = lyrics.toUpperCase().replace(" ", ",").trim().split(",");
       for(String word : words){
           Integer n = frequency.get(word);
   n = (n == null) ? 1 : ++n;
   frequency.put(word, n);
       }
       frequency.remove("");
   }

   //function to print the map
   public static void printTheWords(){
       for(Map.Entry<String, Integer> entry : frequency.entrySet()){
           System.out.println(entry.getKey()+" "+entry.getValue());
       }
   }

   public static void main(String[] args){

       File file = new File("C:\\Users\\venkata.saraswathula\\Documents\\Java\\lyrics.txt");
       try{
           Scanner sc = new Scanner(file);
           String lyrics;
           while(sc.hasNextLine()){
               lyrics = sc.nextLine();
               countTheWords(lyrics);
           }
           printTheWords();
       }
       catch(FileNotFoundException E){
           System.out.println("File not found");
       }
   }
}

OUTPUT: