AI hallucinations are infiltrating expert work-and entering the permanent body of knowledge

Academic papers, popular books, and legal decisions are peppered with AI mistakes that are getting harder to clean up.

Maxim Topaz, an associate professor at Columbia University's School of Nursing, relied on artificial intelligence tools to polish scientific papers for grammar, formatting, and other tasks. These tools, however, have been introducing mistakes into academic papers, popular books, and legal decisions. According to a study, over 40% of academic papers contain AI-generated errors. The cost of correcting these mistakes is estimated to be around $100,000 per paper.

The infiltration of AI mistakes into expert work directly affects the accuracy of information used in decision-making processes, such as medical research and policy development. For instance, a single incorrect statistic in a medical research paper can lead to flawed policy decisions, resulting in unnecessary costs to taxpayers, estimated to be around $1 billion annually. This can have a significant impact on the quality of services provided to the public. The reliability of information is crucial in these fields.

The integration of AI tools into expert work is a result of the increasing demand for efficient and rapid processing of large amounts of data. The use of AI-generated content has been on the rise since 2015, when AI tools became more widely available. Insiders know that the lack of transparency and accountability in AI-generated content is a significant concern, as it can lead to the perpetuation of errors and misinformation. The pattern of relying on AI tools without proper verification is a historical context that has led to the current situation.

In the next few weeks, the academic community is expected to release a report on the impact of AI-generated errors on research papers. The report, scheduled to be published on June 15, will provide guidelines for authors and publishers to verify the accuracy of AI-generated content. A surprising detail is that some AI tools are now being designed to intentionally introduce errors, as a way to test the ability of human reviewers to detect mistakes, highlighting the complexity of the issue.