Question

Task description Baye's Theorem can be used to build many machine learning applications, including spam classifier Spam Classifier in Python from scratch is a tutorial which explains how to use Bave's Theorem and Python to develop a spam classifier step by step To train the spam classifier, one dataset "spam.csv" is used in the program Its description is as follows Context The SMS Spam Collection is a set of SMS tagged messages that have been collected for SMS Spam research. It contains one set of SMS messages in English of 5,574 messages, tagged acording being ham (legitimate) or spam Content The files contain one message per line. Each line is composed by two columns: v1 contains the label (ham or spam) and v2 contains the raw text This corpus has been collected from free or free for research sources at the Internet >A collection of 425 SMS spam messages was manually extracted from the Grumbletext Web site This is a UK forum in which cell phone users make public claims about SMS spam messages, most of them without reporting the very spam message received. The identification of the text of spam messages in the claims is a very hard and time-consuming task, and it involved carefully scanning hundreds of web pages. The Grumbletext Web site is: [Web Linkl. -> A subset of 3,375 SMS randomly chosen ham messages of the NUS SMS Corpus (NSC), which is a dataset of about 10,000 legitimate messages collected for research at the Department of Computer Science at the National University of Singapore. The messages largely originate from Singaporeans and mostly from students attending the University. These messages were collected from volunteers who were made aware that their contributions were going to be made publicly available. The NUS SMS Corpus is avalaible at: [Web Linkl. → A list of 450 SMS ham messages collected from Caroline Tag's PhD Thesis available at [Web Link. → Finally, we have incorporated the SMS Spam Corpus v.0.1 Big. It has 1,002 SMS ham messages and 322 spam messages and it is public available at: [Web Linkl. This corpus has been used in the following academic researches: Acknowledgements The original dataset can be found here. The creators would like to note that in case you find the dataset useful, please make a reference to previous paper and the web page http:/www.dt.fee.unicamp.br/tiago/smsspamcollection/ in your papers, research, etc

I'm using Python 3.7 with Spyder

I need the full code and the same output as the sample above

Resources file: https://drive.google.com/file/d/1e5a21ZKRj2H_jOnWvg7HcjUKjJlY84KE/view -   https://drive.google.com/file/d/1XIA41ra8AaKjFuxO5VpwVkn90bxwDyB5/view

Answer 1

As in the question, it is given to use the code given in GitHub link and use it to classify the given messages so I am using the same code.

from nltk.tokenize import word_tokenize
from nltk.corpus import stopwords
from nltk.stem import PorterStemmer
import matplotlib.pyplot as plt
from wordcloud import WordCloud
from math import log, sqrt
import pandas as pd
import numpy as np
import re
%matplotlib inline

mails = pd.read_csv('spam.csv', encoding = 'latin-1')
mails.head()

mails.drop(['Unnamed: 2', 'Unnamed: 3', 'Unnamed: 4'], axis = 1, inplace = True)
mails.head()

mails.rename(columns = {'v1': 'labels', 'v2': 'message'}, inplace = True)
mails.head()

mails['labels'].value_counts()

mails['label'] = mails['labels'].map({'ham': 0, 'spam': 1})
mails.head()

mails.drop(['labels'], axis = 1, inplace = True)
mails.head()

totalMails = 4825 + 747
trainIndex, testIndex = list(), list()
for i in range(mails.shape[0]):
if np.random.uniform(0, 1) < 0.75:
trainIndex += [i]
else:
testIndex += [i]
trainData = mails.loc[trainIndex]
testData = mails.loc[testIndex]

trainData.reset_index(inplace = True)
trainData.drop(['index'], axis = 1, inplace = True)
trainData.head()

testData.reset_index(inplace = True)
testData.drop(['index'], axis = 1, inplace = True)
testData.head()

trainData['label'].value_counts()

testData['label'].value_counts()

spam_words = ' '.join(list(mails[mails['label'] == 0]['message']))
spam_wc = WordCloud(width = 512,height = 512).generate(spam_words)
plt.figure(figsize = (10, 8), facecolor = 'k')
plt.imshow(spam_wc)
plt.axis('off')
plt.tight_layout(pad = 0)
plt.show()

#training the model using tf_idf word2vector train data set.
sc_tf_idf = SpamClassifier(trainData, 'tf-idf')
sc_tf_idf.train()

preds_tf_idf = sc_tf_idf.predict(testData['message'])
print(metrics(testData['label'], preds_tf_idf))

#training the model using bag-of-word vectors train data set
sc_bow = SpamClassifier(trainData, 'bow')
sc_bow.train()

preds_bow = sc_bow.predict(testData['message'])
print(metrics(testData['label'], preds_bow))

message1 = "Want to change how you recieve these mails? You can update your preferences or unsubscribe from this list at http://guru.phishing.guru/."
message2 = "You are invited to law court by the judge of the law voilation. Case:#3118804 Date: April 5, 2019, sent by [email protected]"

#using tf_idf for predicting message1
pm = process_message(message1)
print(pm)
print(sc_tf_idf.classify(pm))

#using tf_idf for predicting message2
pm = process_message(message2)
print(pm)
print(sc_tf_idf.classify(pm))

Use the code of procss_message function, SpamClassifier function and metrics function same as provided in the GitHub link code.

The required output asked in question images are as follows:

Python [conda em.Anaconda31 Trusted le Edit View Insert l Kenel Widgets Help 凹 in ibJ spam-"oras -- .join(11st (malLSLmalLSL Ladei 」 -- wjL message 」)) 2 spam c Wordcloud(width -512,height 512).generate(spam_words) 3 plt.figure(figsize = (18, 8), facecolor- "k.) 4 plt.imshow (spam_Hc) 5 plt.axis off' 6 plt.tight layout (pad-) 7 plt.show) ove going finish onething even let know free wa co give money E ate ight sleep1ife meeta umber r dear hope talk ar ome Yeah es dat really right ah Pleaseak keep ean find Ssent month PickYup make re ine ight buy one went Happy sendworkk 1ssnight text place want well ThanI 80od today ater saidaround us ouk sarry

le Edit View Insert l Kenel Widgets Help TrustedPython [conda env Anaconda3] O In [20]: | 1 #training the modeL using tf_1df word?vector train dato set 2 sc tf idf SpamClassifier trainData, 'tf-idf' 3 sc tf_idf.train) In [24]:preds_tf_idf - sc_tf_idf.predict (testData'message ]) 2 print metrics(testData'label ], preds tf idf)) Precision 0.9 Recall: 0.7 F-score: .7875 Accuracy: e,9498894620486367 In [21]:training the model using bag-of-word vectors troin doto set 2 sc boWSpamClassifier (trainData, 'bow) 3 sc bow.train) In [25]:1 preds_bowSc_bow.predict (testData [ message']) 2 print metrics(testData['label'], preds bow)) Precision: 9.875 Rec a11: 6.5444444444444444 F-score: e·6712328767123288 Accuracy: 0.9292557111274871 In [22]:1 hange how you recieve these mails? You can update your preferences or unsubscribe from this list at http://guru.phishing 2 .nvited to law court by the judge of the law voilation. Case:#3118804 Date: April 5, 2019, sent by notice(afp-hq.com . au" In [29]: | 1 #using tf_1df for predicting message! 2 pm process_ message(message1) 3 print (pm) 4 print(sc_tf_idf.classify pm)) [.want change", .change how", ‘how you., .you recieve', ‘recieve these', .these mails', .mails you ., .you can., e', .update your", 'your preferences. .preferences unsubscribe', .unsubscribe from. ·from this', 'this list', р., "http //guru.phishing.guru/.] .can updat 'list htt Out [29]: False

le Edit View Insert l Kenel Widgets Help TrustedPython [conda env Anaconda3] O Precision 0.875 Recall: 0.54 F-score: .6712328767123288 Accuracy: е.92925 57111274871 In [22]:1 hange how you recieve these mails? You can update your preferences or unsubscribe from this list at http://guru.phishing 2.nvited to law court by the judge of the law voilation. Case:#3118804 Date: April 5, 2e 19, sent by [email protected]" In [29]: | 1 #using tf-idf for predicting message! 2 pm -process_message(message1) 3 print(pm) 4 print (sc tf_idf.classify(pm)) [ 'want change, "change ho', 'hou you', 'you recieve recieve these, these mails", mails you', 'you can, can updat e upate your',"your preferencesferences unsubscribe subscribe from, from this, 'this list', 'list htt p, "http //guru.phishing.guru/' out [29]: False In [28]: | 1 #using tf-idf for predicting message2 2 pm -process_message(message2) 3 print (pm) 4 print (sc tf_idf.classify(pm)) 'you are', 'are invited', invited law', 'law court court the', 'the judge, "judge the', 'the la 1aw voilation, voilation case', case 3118804, 3118804 date, date april 'april 2019,2019 sent', 'sent notice', notice afp-hq.co m.au out [28] False

I hope you got the answer and understand it. any doubts ask in comments.

Thank you:):)