2. Decision Trees (5 Points): There are three parts to this
question. The first two parts are for 2 points and third part is
one point. 1 Figure 1: 2 Dimensional Data for DT Classification (a)
Part 1: In the class, we saw that finding the split which maximizes
the information gain I(Xi ; Y ) [which is also called the mutual
information] was a good strategy to greedily build the decision
tree. Explain why this is a good strategy with an example. What if
instead of the information gain, we were to use the conditional
gain H(Y |Xi). Would it make sense to maximize or minimize
this?
(b) Part 2: Consider the data points shown in Figure 1. Draw the
approximate decision boundry obtained by the a decision tree
algorithm. Also provide an approximate solution obtained by the
decision tree algorithm (provide the final tree which the DT
algorithm would most likely give). I am not expecting you to do
this programmatically. Argue based on the rough position of the
data points, how the tree learning algorithm will grow. Next,
assume you have no limit on the depth of the tree. How would the
solution look in that case?
(c) Part 3: Which of the following is true about ‘max depth‘
hyperparameter in the decision tree: A) Lower is better parameter
in case of same validation accuracy, B) Higher is better parameter
in case of same validation accuracy, C) Increase the value of max
depth may overfit the data or D) Increase the value of max depth
may underfit the data. More than one may be correct.
- 2 Decision Trees 5 Points There Are Three Parts To This Question The First Two Parts Are For 2 Points And Third Par 1 (95.49 KiB) Viewed 46 times
2. Decision Trees (5 Points): There are three parts to this
question. The first two parts are for 2 points and third part is one point. 1 Decision Trees 92 0 o o 6 0 D e 5 0 . 4 3 oOo D 0 > 2 o 1 2 3 4 S 2 Figure 1: 2 Dimensional Data for DT Classification (a) Part 1: In the class, we saw that finding the split which maximizes the information gain I(X,Y) [which is also called the mutual information was a good strategy to greedily build the decision tree. Explain why this is a good strategy with an example. What if instead of the information gain, we were to use the conditional gain H(Y|Xi). Would it make sense to maximize or minimize this? (b) Part 2: Consider the data points shown in Figure 1. Draw the approximate decision boundry obtained by the a decision tree algorithm. Also provide an approximate solution obtained by the decision tree algorithm (provide the final tree which the DT algorithm would most likely give). I am not expecting you to do this programmatically. Argue based on the rough position of the data points, how the tree learning algorithm will grow. Next, assume you have no limit on the depth of the tree. How would the solution look in that case? (c) Part 3: Which of the following is true about 'max depth hyperparameter in the decision tree: A) Lower is better parameter in case of same validation accuracy, B) Higher is better parameter in case of same validation accuracy, C) Increase the value of max depth may overfit the data or D) Increase the value of max depth may underfit the data. More than one may be correct.