The 1cycle policy

Posted on Sat 07 April 2018 in Experiments • Tagged with Deep Learning, SGD, Learning Rate

Properly setting the hyper-parameters of a neural network can be challenging, fortunately, there are some recipe that can help.

Continue reading

SGD Variants

Posted on Thu 29 March 2018 in Basics • Tagged with SGD, Deep Learning

Let's get a rapid overview and implementations of the common variants of SGD.

Continue reading

How Do You Find A Good Learning Rate

Posted on Tue 20 March 2018 in Basics • Tagged with SGD, Learning Rate

This is the main hyper-parameter to set when we train a neural net, but how do you determine the best value? Here's a technique to quickly decide on one.

Continue reading

What Is Deep Learning?

Posted on Tue 13 March 2018 in Basics • Tagged with Neural Net, SGD, Deep Learning

What is deep learning? It's a class of algorithms where you train something called a neural net to complete a specific task. Let's begin with a general overview and we will dig into the details in subsequent articles.

Continue reading