Classifying spams using spark mllib


Given the Spam data set for classifying an e-mail is either spam (label 1) or not (label 0). The dataset can be downloaded from to an external site.. Compare three ML algorithms: logistic regression, decision tree, and random forest for this binary classification problem. Collect screenshots of critical parts for each question. Type all answers. Submit a pdf report. Use Databrick notebook. The final notebook should be fully documented and uploaded to Canvas as well as the pdf report.

1. (30 points) Find the best model for each ML algorithms. Compute precision, recall, and F1 scores based on ALL data points.  Identify the best ML algorithm based on the statistics computed.

2. (30 points) Use 5 fold (70/30) cross-validation method to find the best model for each ML algorithm using the default ROC criterion. Compare these three ML algorithms and pick the best overall model. Discuss how you reach this conclusion.

3. (20 points) Based on your best model in Q2, identify the most significant features. Discuss how you can take advantage of this finding. 

4. (10 points) Discuss why the best ML algorithm is better than the other two from a theoretical point of view. 

5. (10 points) Does the best model provide a satisfactory classification outcome? Explain. Is there another ML model that may be used to better this result? Discuss and provide the sources of your analysis.

Place your order
(550 words)

Approximate price: $22

Calculate the price of your order

550 words
We'll send you the first draft for approval by September 11, 2018 at 10:52 AM
Total price:
The price is based on these factors:
Academic level
Number of pages
Basic features
  • Free title page and bibliography
  • Unlimited revisions
  • Plagiarism-free guarantee
  • Money-back guarantee
  • 24/7 support
On-demand options
  • Writer’s samples
  • Part-by-part delivery
  • Overnight delivery
  • Copies of used sources
  • Expert Proofreading
Paper format
  • 275 words per page
  • 12 pt Arial/Times New Roman
  • Double line spacing
  • Any citation style (APA, MLA, Chicago/Turabian, Harvard)

Our guarantees

Delivering a high-quality product at a reasonable price is not enough anymore.
That’s why we have developed 5 beneficial guarantees that will make your experience with our service enjoyable, easy, and safe.

Money-back guarantee

You have to be 100% sure of the quality of your product to give a money-back guarantee. This describes us perfectly. Make sure that this guarantee is totally transparent.

Read more

Zero-plagiarism guarantee

Each paper is composed from scratch, according to your instructions. It is then checked by our plagiarism-detection software. There is no gap where plagiarism could squeeze in.

Read more

Free-revision policy

Thanks to our free revisions, there is no way for you to be unsatisfied. We will work on your paper until you are completely happy with the result.

Read more

Privacy policy

Your email is safe, as we store it according to international data protection rules. Your bank details are secure, as we use only reliable payment systems.

Read more

Fair-cooperation guarantee

By sending us your money, you buy the service we provide. Check out our terms and conditions if you prefer business talks to be laid out in official language.

Read more