Time based decay is a type of learning rate decay which decreases the learning rate relative to the last learning rate.

Time based learning rate decay can be defined by the following equation

• Definitions
  • is the learning rate for the iteration
  • is the current iteration step
  • is the "decay parameter" and controls how quickly the learning rate will change.