Researchers show that training on junk data can lead to LLM brain rot

Models trained on short, popular, and/or superficial tweets perform worse on benchmarks.