A study claiming ChatGPT delivers large benefits to student learning has been retracted by its publisher, Springer Nature, almost exactly one year after it appeared in the journal Humanities & Social Sciences Communications. By the time the retraction came, the paper had collected 504 citations from peer-reviewed and non-peer-reviewed sources combined and had been read by nearly half a million people.
The paper used a meta-analysis of 51 earlier studies to argue that ChatGPT had a "large positive impact on improving learning performance," along with moderately positive effects on learning perception and higher-order thinking. Those claims spread quickly. The study landed in the 99th percentile of all journal articles by online attention score and was widely shared on social media as early, rigorous evidence that generative AI tools help students learn.
Springer Nature's retraction notice cited "discrepancies" in the analysis and said the publisher had lost confidence in the paper's conclusions. The publisher did not specify the exact nature of the discrepancies.
Ben Williamson, a senior lecturer at the Centre for Research in Digital Education at the University of Edinburgh, had questions about the paper from the start. "It was treated by many on social media as one of the first pieces of hard, gold standard evidence that ChatGPT, and generative AI more broadly, benefits learners," he told Ars Technica.
Williamson was skeptical on methodological grounds. "In some cases it appears it was synthesizing very poor quality studies, or mixing together findings from studies that simply cannot be accurately compared due to very different methods, populations and samples," he said. "It really seemed like a paper that should not have been published in the first place."
He also pointed to a timing problem. OpenAI released ChatGPT in November 2022. The retracted paper was published in May 2025, roughly two and a half years later. "It is not feasible that dozens of high-quality studies about ChatGPT and learning performance could have been conducted, reviewed, and published in that time," Williamson said.
The retraction does not erase the paper's existing footprint. It had already been cited 262 times specifically within Springer Nature's own peer-reviewed journals before being pulled. Those citing papers remain in circulation, meaning the flawed conclusions may continue to ripple through the academic literature even after the source has been formally disavowed. Researchers who track citation networks have noted that retracted papers frequently continue to be cited positively for years after their retraction, a pattern that could extend the influence of this study well beyond its formal withdrawal.
The episode arrives as researchers, educators, and policymakers are actively looking for evidence to guide decisions about how AI tools should be used in classrooms. The appetite for such evidence, Williamson suggested, may have contributed to the paper's rapid and uncritical spread before anyone closely examined the underlying analysis.
