Day 2 - Gradient descent - #66DaysofData Challenge

-

 By Jerin Lalichan 

1. Gradient Descent

    Gradient descent is an optimization algorithm which is commonly used to train machine learning models and neural networks. Optimization is the process of minimizing the cost function in an algorithm.

Types of Gradient descent 

  1. Batch Gradient descent 
    In this, all the training sample is used for calculations in each iteration, which makes this a time-consuming process if it's a large dataset. ie, It uses all samples for one forward pass and then adjusts weights.

  2. Stochastic Gradient descent
    In this, not all the samples, but one sample selected randomly will undergo forward pass and the weight gets adjusted.

  3. Mini-batch Gradient descent
    In this, a batch of samples, selected randomly will undergo forward pass and weight gets adjusted.


Gradient descent in Linear Regression


In Linear regression, the cost function is the Mean squared error or root mean square error. The parameters(θ) are the Slope and Intercept of the line. Our target is to reduce the cost function. And it can be done by finding the best parameters ie, slope, and intercept. The best value is found using the gradient descent algorithm's iterations.

The Gradient descent algorithm is given by:


  • θj : Weights of the hypothesis.
  • hθ(xi) : predicted y value for ith input. 
  • j : Feature index number (can be 0, 1, 2, ......, n). 
  • α : Learning Rate of Gradient Descent.
After several iterations, the cost function will approach the global minima. So the iterations will stop either by reaching the maximum number of steps or the slope of the cost function reaches very close to zero. The corresponding parameters will be selected and applied to the model 


    I am doing a challenge - #66DaysofData  in which I will be learning something new from the Data Science field for 66 days, and I will be posting daily topics on my LinkedIn, On my GitHub repository, and on my blog as well.


Stay Curious!  




  

Comments

Post a Comment

Popular posts from this blog

Day 17 - Ensemble Techniques in ML - Averaging, Weighted average

-
   By  Jerin Lalichan  Ensemble Techniques in ML      Ensemble methods is a machine learning technique that combines several base models in order to produce one optimal predictive model.      Ensemble methods include building multiple models and combining them to achieve better outcomes. To put it another way, they integrate the conclusions drawn from various models to enhance overall performance. Generally speaking, ensemble methods produce more accurate results than a single model would.     F or example, let's consider the case in which you need to decide if you should go to a particular movie or not. You can infer that a diverse group of people are likely to make better decisions as compared to individuals. So it's best to check online reviews since it is an aggregation of reviews of hundreds of people from different backgrounds when compared to asking a few of your friends.            Similar is true for a diverse set of models in comparison to single models. This diversif
Read more

Day 4 - Performance metrics in Machine Learning - Regression

-
Image
      By  Jerin Lalichan  Performance metrics in ML     Evaluation of the performance of a model is important. Performance metrics are certain measures to quantify the performance of the model during the training and testing phases. In Machine learning, there are generally two kinds of performance metrics in use. For regression models and for classification models. Below are the most popular metrics in use: Regression Metrics Mean Squared Error (MSE) It is simply the average of the sum of the squares of the difference between the actual value and predicted values. Due to the squaring in this equation, small errors are overestimated. Also because for that reason, this is very much prone to outliers.  2 .  Mean Absolute Error (MAE) Mean Absolute Error is the average of the difference between the ground truth(actual value)  and the predicted values . Since there is no squaring, the error estimated is not exaggerated or overestimated. Also, it is better with outliers than MSE. Even though
Read more