Under-trained and Unused tokens in Large Language Models | Towards Data Science

Existence of under-trained and unused tokens and Identification Techniques using GPT-2 Small as an Example

By · · 1 min read
Under-trained and Unused tokens in Large Language Models | Towards Data Science

Source: Towards Data Science

Existence of under-trained and unused tokens and Identification Techniques using GPT-2 Small as an Example