Decision Tree

Decision Trees are known for producing robust, interpreable cateogories/splits in which data is classified.

Here, the top box is the root node, containing all the data.

Any nodes at the very bottom are considered the leaves. Any in between are called split nodes.

GINI Impurity

$$G_i=1-\sum _{k=1}^N{p}_{i,k}^2$$

Where:

• $G_i$ is the impurity of the $i^{th}$ node
• $p_{i,k}$ is the ration of class $k$ instances among the training instances in the $i^{th}$ node

If there are two classes in the $i^{th}$ node and 100 instances, then:

$$G_i=1-{\left(\frac{x}{100}\right)}^2-{\left(\frac{100-x}{100}\right)}^2$$