What is the accuracy of the testing data, and why is it not 100%?

INSTRUCTIONS TO CANDIDATES

ANSWER ALL QUESTIONS

Problem 1. Download the dataset datamining.xlsx from LMS. This dataset contains 2,000 cases. This dataset is to be used to predict whether a person in an MIS program will like a data mining course or not. The fields for each of the 2000 records are as below:

• GMAT: GMAT score of a student

• Bachelor: Field of BS degree (A: Arts, S: Science, E: Engineering)

• Quant, Stats, HBO, Acct: Course rating of the student for each of the courses from 1 (lowest) to 5 (highest)

• E-comm: Flag that is T if student intends to specialize in e-commerce, F otherwise

• Datamine: Course rating of the student for Data Mining

• LikeDM: Flag that is T if course rating for Data Mining is 4 or 5; F otherwise (note that this attribute is derived from “Datamine” attribute, so you should eliminate “Datamine” from exploration and modeling).

Using RapidMiner, answer the following questions. A sample process is provided as a starter.

a) [20 points] Use the entire data (datamining.xlsx) and explore the relationship between LikeDM and each individual field. What effect does each field seem to have on LikeDM? You can use scatterplots and histograms to explore the relationships and show only what seems to be important relations.

b) [25 points] Split the data into 65% for training and 35% for testing using Split Validation operator. Click on the operator and change random seed value to “12345”. Create a Decision Tree (ModelingPredictiveTreeDecision Tree) and make sure to get 100% accuracy on training data. To do this, set criterion to gini index, set the tree depth to high number (e.g. 2000) and uncheck both “apply pruning” and “apply prepruning,” then answer the following:

1. What is the depth of the tree?

2. How many leaves (decision nodes) does it have?

3. What is the accuracy of the testing data, and why is it not 100%?

 

 

c) [15 points] Now change the settings of the decision tree model as follows, then answer the questions:

• Click on the decision tree and choose criterion to “information_gain” and set maximum depth to 8.

• Check “apply prepruning” and set minimal gain to 0.01, minimal leaf size to 2, and minimal size for split to 4 (leave other options as is).

1. What is the accuracy of training and testing? Do you see an improvement in the model? How?

2. Provide two strongest If-Then rules from this decision tree. Please explain why these rules are chosen.

d) [15 points] Try to further improve the performance of the decision tree model by changing the decision tree parameters (you can change tree depth, type of criterion, or minimal size of leaf or split). What is the performance of the tree you created (both training and testing) and what have you changed in the tree settings? Produce at least two (2) different models.

e) [10 points] Use the models developed above to compare between their performance by filling the table provided. Which model is the best, and why?

 

Attachments:

Related Questions

. Introgramming & Unix Fall 2018, CRN 44882, Oakland University Homework Assignment 6 - Using Arrays and Functions in C

DescriptionIn this final assignment, the students will demonstrate their ability to apply two ma

. The standard path finding involves finding the (shortest) path from an origin to a destination, typically on a map. This is an

Path finding involves finding a path from A to B. Typically we want the path to have certain properties,such as being the shortest or to avoid going t

. Develop a program to emulate a purchase transaction at a retail store. This program will have two classes, a LineItem class and a Transaction class. The LineItem class will represent an individual

Develop a program to emulate a purchase transaction at a retail store. Thisprogram will have two classes, a LineItem class and a Transaction class. Th

. SeaPort Project series For this set of projects for the course, we wish to simulate some of the aspects of a number of Sea Ports. Here are the classes and their instance variables we wish to define:

1 Project 1 Introduction - the SeaPort Project series For this set of projects for the course, we wish to simulate some of the aspects of a number of

. Project 2 Introduction - the SeaPort Project series For this set of projects for the course, we wish to simulate some of the aspects of a number of Sea Ports. Here are the classes and their instance variables we wish to define:

1 Project 2 Introduction - the SeaPort Project series For this set of projects for the course, we wish to simulate some of the aspects of a number of