Weeks 7 & 8: Decision Trees, Random Forests and Boosting
Decision Tree: Decision Trees are one of the most simple classification methods. Sometimes we do not have discrete features but we rather have continuous features. In continuous space it is convenient to think about decision trees as partitioning the space. Pruning Trees Generative and Discriminative Models Most of what we talked about so far, PCA and k-means and intrinsic dimension, they're all in the generative domain. We talked already about trees that are discriminative models and we're going to talk about boosting that is also a discriminative model. one way that we would describe as generative modeling is to basically try to say what's the distribution of pitches for males and what's the distribution of pitches for females. And so, suppose we have a dataset down there and suppose that we measure, for each set, we measure the mean and the standard deviation, or variance. We have those and now we are making the assumption that because we don't know anyt...