Red Pajama 2: The Public Dataset With a Whopping 30 Trillion Tokens

Together, the developer, claims it is the largest public dataset specifically for language model pre-training

Data science recent news

Data science recent news

Red Pajama 2: The Public Dataset With a Whopping 30 Trillion Tokens

ChatGPT / Generative AI recent news, page 3 of 19

RedPajama-Data-v2: An open dataset with 30 trillion tokens for training large language models

NLP recent news, page 7 of 30

cerebras/SlimPajama-627B · Datasets at Hugging Face

Top 10 List of Large Language Models in Open-Source

Ben Wodecki recent news, page 4 of 46

Together AI Releases RedPajama v2: An Open Dataset with 30 Trillion Tokens for Training Large Language Models - MarkTechPost