GPT-2
OpenAI's 2019 language model that demonstrated emergent text generation capabilities, initially withheld from release over misuse concerns and now used as a standard training benchmark.
GPT-2 was trained on 40GB of internet text and showed that scaling transformers produced coherent long-form text generation. OpenAI's staged release due to safety concerns sparked debate about AI publication norms. The model has since become 'the new MNIST'—a cheap, reproducible baseline for testing training optimizations, with modern techniques reducing its training cost from $43,000 to under $20.
Also known as
GPT2, gpt-2, gpt2