SGD

Choosing a Learning Rate is Hard

Interaction Effects between your Algorithm and your Learning Rate

Adagrad Has a Learning Rate

Adagrad requires a learning rate, and why that can be a good thing.