Why AI invents references, and how to catch the fakes
Language models write references the way they write everything else: as plausible text. What two studies from 2023 found, five ways a reference goes wrong, and how to catch each one.
You ask a chatbot for literature on your topic and get five tidy references. One names an author you recognise, a journal that fits, a year and a DOI. You search for the title and find nothing. The journal exists, the author exists, the paper does not.
That is not a slip in an otherwise reliable search. A language model on its own does not look references up. It writes them.
A reference is a pattern
A language model produces text that is likely to follow what came before. References are an unusually regular kind of text: surnames with initials, a year in brackets, a title, a journal, volume and pages, a DOI that starts with 10.. A model has seen a great many of them, so it can write one that fits every rule of the pattern without belonging to any real paper. Bhattacharyya and colleagues put it like this: a model predicts the next word from statistical patterns learned in training, and so it cannot tell accurate from false information.
Walters and Wilder add a refinement. In their experience the model sometimes seems to copy references rather than paraphrase them, so fabrication may come from failing to recognise which parts of a text are bibliographic data that must be reproduced exactly. A title word swapped for a synonym still reads well, and it makes the title useless for finding the paper.
What two studies found in 2023
- Walters and Wilder (2023), in Scientific Reports, had ChatGPT write 84 short literature reviews on 42 topics in April 2023, half with GPT-3.5 and half with GPT-4, and checked all 636 references. 55% of the GPT-3.5 references and 18% of the GPT-4 references were fabricated. Of the real ones, 43% (GPT-3.5) and 24% (GPT-4) contained substantive errors, most often in the numbers: volume, issue, pages and year. Book chapters fared worst. Even with GPT-4, 70% of the cited chapters were fabricated, and the authors note that many fabricated chapters name books that do not exist either.
- Bhattacharyya et al. (2023), in Cureus, had ChatGPT with GPT-3.5 write 30 short medical papers in April 2023, with 115 references between them. 47% were fabricated, 46% were real but inaccurate, and only 7% were real and accurate. The most frequent error was a wrong PMID, followed by volume, page numbers and year.
Both studies tested the ChatGPT models of spring 2023, so their figures say nothing exact about the models you use today. The kinds of error they describe are still the ones worth checking for, and both point to the same weak spot: numbers and identifiers.
Five ways a reference goes wrong, and how to catch each
- Invented. No such work exists. By hand: search the exact title in quotation marks in your library catalogue or Google Scholar, and look through the issue of the journal it names. In Benthic: Not found, once neither the identifier nor a title search in five registries leads anywhere. The site's example of this kind is Smith, J. (2021), “A systematic audit of citation reliability in large language models”, which is made up.
- A real title with the wrong authors or year. By hand: compare with the publisher's page. In Benthic: Deviates, with a note that names what differs.
- A real DOI attached to another paper. By hand: open the DOI link and read the title that appears. In Benthic: Mismatch, or DOI corrected when the cited paper exists under another DOI. “Deep learning” by LeCun, Bengio and Hinton (2015) is the site's example of the second case.
- Real but retracted. By hand: look for a retraction notice on the publisher's page. In Benthic: Retracted, from the metadata of OpenAlex and Crossref; Crossref has also carried the Retraction Watch database since January 2025. Wakefield et al. (1998) in The Lancet is one such paper.
- Real, but it does not say what is claimed. By hand: read the passage. Benthic does not judge this. It can show you the passages in the full text that come closest to your sentence, with page numbers, and check direct quotations word for word, but whether the source supports the claim is yours to decide.
A routine that holds
Treat every generated reference as a lead, not a source. Check it before you read it, and read it before you cite it. Asking the model whether its own references are real is not a check: Walters and Wilder note that ChatGPT often answers that question wrongly, and Bhattacharyya et al. that it tends to stand by its errors.
For the checking, paste the answer into Benthic. Checking a reading list shows the steps, and the seven verdicts explains what each result means and what to do with it. If your AI assistant is connected to Benthic through MCP, it can check its own sources with Benthic.