A Simple Implementation of the Attention Mechanism from Scratch | Towards Data Science

How attention helped models like RNNs mitigate the vanishing gradient problem and capture long-range dependencies among words