Day 19 - Advanced Ensemble techniques - Blending

    By Jerin Lalichan 

Advanced Ensemble techniques

Blending

    Blending follows the same approach as stacking but uses only a validation set from the train set to make predictions. In other words, unlike stacking, the predictions are made on the holdout set only. The holdout set and the predictions are used to build a model which is run on the test set. 

    Blending is a technique derived from Stacking Generalization. The only difference is that in Blending, the k-fold cross validation technique is not used to generate the training data of the meta-model. 

    Blending implements “one-holdout set”, that is, a small portion of the training data (validation) to make predictions which will be “stacked” to form the training data of the meta-model. Also, predictions are made from the test data to form the meta-model test data.

Blending is a similar approach to stacking.

• The train set is split into training and validation sets.
  
  
• We train the base models on the training set.
  
  
• We make predictions only on the validation set and the test set.
  
  
• The validation predictions are used as features to build a new model.
  
  
• This model is used to make final predictions on the test set using the prediction values as features.
  
  
  
  

   

  I am doing a challenge - #66DaysofData  in which I will be learning something new from the Data Science field for 66 days, and I will be posting daily topics on my LinkedIn, On my GitHub repository, and on my blog as well.

Stay Curious!  

By Jerin Lalichan