The key takeaways from this section include:
- Decision trees can be used for both categorization and regression tasks
- They are a powerful and interpretable technique for many machine learning problems (especially when combined with ensemble methods)
- Decision trees are a form of Directed Acyclic Graphs (DAGs) - you traverse them in a specified direction, and there are no "loops" in the graphs to go backward
- Algorithms for generating decision trees are designed to maximize the information gain from each split
- A popular algorithm for generating decision trees is ID3 - the Iterative Dichotomiser 3 algorithm
- There are several hyperparameters for decision trees to reduce overfitting - including maximum depth, minimum samples to split a node that is currently a leaf, minimum leaf sample size, maximum leaf nodes, and maximum features