Decision Trees are simple models that can easily be understood by laymen. They are easy to use and visualize, and instead of a black box they can be easily understood as an interpretable white box model, making them suitable for various applications. We discuss their basics, terminology, and how to train them with the CART algorithm. Finally, we discuss how to avoid overfitting of decision trees, and we discuss the pros and cons of decision trees.

After this section you should be able to:

• • Explain the difference between black box and white box models and their pros and cons
  • Explain how the CART algorithm works on a high level
  • Apply the cost function J and Gini-index
  • Explain the terms root, node, leaf, depth
  • Explain two regularization techniques for Decision Trees and give one pro and con for each