PyTorch Optimizers Aren't Fast Enough. Try These Instead | Towards Data Science

These 4 advanced optimizers will open your mind.