Line-By-Line, Let's Reproduce GPT-2: Section 2 - Hardware Optimization | Towards Data Science

This blog post will go line-by-line through the hardware optimizations in Section 2 of Andrej Karpathy’s “Let’s reproduce GPT-2 (124M)”

By · · 1 min read
Line-By-Line, Let's Reproduce GPT-2: Section 2 - Hardware Optimization | Towards Data Science

Source: Towards Data Science

This blog post will go line-by-line through the hardware optimizations in Section 2 of Andrej Karpathy’s “Let’s reproduce GPT-2 (124M)”