New Nvidia Paper Accelerates Large-Scale Language Modelling | Synced
Nvidia’s paper Large Scale Language Modeling: Converging on 40GB of Text in Four Hours introduces a model that uses mixed precision arithmetic and a 32k batch size distributed across 128 Nvid...
Source: Synced | AI Technology & Industry Review
Nvidia’s paper Large Scale Language Modeling: Converging on 40GB of Text in Four Hours introduces a model that uses mixed precision arithmetic and a 32k batch size distributed across 128 Nvidia Tesla V100 GPUs to improve scalability and transfer in Recurrent Neural Networks (RNNs) for Natural Language tasks.