Optimization algorithms in machine learning is an iterative process of finding values of parameters in a model that minimizes or maximizes some objective function (usually the cost function). These algorithms are what allows the model to improve its performance by adjusting its parameters to better fit the data.
Common Optimization Algorithms