Day 18 - Advanced Ensemble techniques - Stacking

   By Jerin Lalichan 

Advanced Ensemble techniques

Here are the advanced methods in Ensemble techniques:

1. Stacking

    This technique uses prediction from different models (eg. Decision tree, SVM, KNN, etc.) to form a new model. This model is used for making predictions on the test set. The concept used here is that each model can learn different parts of the problem but not the whole problem. 

    So, you can build multiple different learners and you use them to build an intermediate prediction, one prediction for each learned model. Then you add a new model which learns from the intermediate predictions of the same target.

Steps:

1. We split the training data into K-folds like in cross-validation.
   
   
2. A base model is fitted on the K-1 parts and predictions are made for Kth part.
   
   
3. We do this for each part of the training data.
   
   
4. The base model is then fitted on the whole train data set to calculate its performance on the test set.
   
   
5. We repeat the last 3 steps for other base models.
   
   
6. Predictions from the train set are used as features for the second-level model.
   
   
7. Second level model is used to make a prediction on the test set.

   

  I am doing a challenge - #66DaysofData  in which I will be learning something new from the Data Science field for 66 days, and I will be posting daily topics on my LinkedIn, On my GitHub repository, and on my blog as well.

Stay Curious!  

By Jerin Lalichan