Batching in machine learning is a method of running an algorithm on multiple feature sets before updating the parameters of the algorithm.

Typically the cost function is run on all outputs of the algorithm and then the value is averaged to update the parameters but other functions may be used like summation or loss aggregation (such as taking the maximum or minimum loss value).

Forms

Batching can take many forms ranging from small batches of just to large batches where the batch is as large as the training set.

• Batch Gradient Descent - Batch Size = Size of Training Set
• Stochastic Gradient Descent - Batch Size = 1
• Mini-Batch Gradient Descent - 1 < Batch Size < Size of Training Set