Join egghead, unlock knowledge.

Want more egghead?

This lesson is for members. Join us? Get access to all 3,000+ tutorials + a community with expert developers around the world.

Unlock This Lesson
1×
Become a member
to unlock all features

Level Up!

Access all courses & lessons on egghead today and lock-in your price for life.

Autoplay

    Change the Learning Rate of the Adam Optimizer on a Keras Network

    Chris AchardChris Achard
    pythonpython
    ^3.0.0

    We can specify several options on a network optimizer, like the learning rate and decay, so we’ll investigate what effect those have on training time and accuracy. Each data sets may respond differently, so it’s important to try different optimizer settings to find one that properly trades off training time vs accuracy for your data.

    Code

    Code

    Become a Member to view code

    You must be a Member to view code

    Access all courses and lessons, track your progress, gain confidence and expertise.

    Become a Member
    and unlock code for this lesson
    Discuss

    Discuss

    Transcript

    Transcript

    Instructor: We're using the Adam optimizer for the network which has a default learning rate of .001. To change that, first import Adam from keras.optimizers. Then, instead of just saying we're going to use the Adam optimizer, we can create a new instance of the Adam optimizer, and use that instead of a string to set the optimizer.

    One of the optional parameters is lr, which stands for learning rate. Setting the learning rate is like saying how large the steps are that the network takes while learning. If the learning rate is too small, the network will never have a chance to get where it's going. The accuracy will always be low, or training will take a really long time.

    If the learning rate is too large, then the network will jump all over the place and will never be able to find the best solution, because it will keep jumping over it. Let's set a learning rate of .005 and see how that works for our problem set.

    After 100 EPoX, we're seeing good progress on the training in validation accuracy, so we'll keep that learning rate. It's important to test different learning rates for your network, because each dataset in network topology will respond slightly differently to different learning rates.