Marko Kovacevic Blog – Page 2 – Sharing knowledge, amazing things and ideas

Learning Rate Schedules in Deep Learning

Learning rate schedules are techniques used to adjust Learning rate during training by predefined schedule. See what is Learning rate

Learning rate is value by which weights are updated during Artificial neural network training. The value of Learning rate tell

Birds are superior than humans in sleeping because they can sleep and fly but humans can’t sleep and walk or

Nesterov momentum is different version of momentum update. It is also called Nesterov Accelerated Gradient. In practice it works better

Momentum is Deep learning technique that is used along with Stochastic Gradient Descent. Instead of using only the gradient of the