Check a reading list from ChatGPT, Claude or a colleague

The list arrives in a chat window: six sources for a seminar talk, neatly formatted, four of them with a DOI. It might come from ChatGPT or Claude, or from a colleague who did not say where they found it. Before any of it ends up in your own paper, you want to know whether each work exists, and whether it really is the work the entry names.

Here is that check, step by step, with the six references from the example on the product page.

What you can paste

Benthic takes the text as it comes, without any tidying first:

Chat answers, including the introduction and the closing remarks. Headings and ordinary prose between the references are skipped.
Reading lists in any common citation style, numbered or with bullet points, also as Markdown with bold and italics.
DOI lists, one per line or separated by commas or semicolons, also as doi.org links.
BibTeX and RIS exported from a reference manager.

Paste it under Check, open a file, or drop it on the window. From another app, select the text and choose Services › “Check References with Benthic”. While you paste, a line under the field says what Benthic recognised, for the example list “6 sources recognised · 4 with DOI · 2 title only”. An entry that appears twice is checked once.

How each reference and its identifier are found

For every entry Benthic first reads what the text claims: authors, year, title and, if there is one, an identifier. The title comes from quotation marks or italics, or from its usual place after the year. This step runs on your Mac, offline.

DOI. Recognised on its own or as a link, also with brackets, as in the Lancet's 10.1016/S0140-6736(97)11096-0.
arXiv ID. An ID such as arXiv:1706.03762 becomes the DOI that arXiv registers for it, 10.48550/arXiv.1706.03762.
PMID. A PubMed ID is resolved to its DOI through Europe PMC.
No identifier. The entry is searched by title, first authors and year.

Which databases Benthic asks

With a DOI, Benthic fetches the real work from OpenAlex, otherwise from Crossref, and otherwise asks doi.org whether the DOI is registered at all. For arXiv papers it also asks arXiv for the date of the first version. Then it compares the claimed title, authors and year with the work it found.

Without a DOI, or with one that leads to a different work, Benthic searches by title in Crossref and OpenAlex. If neither gives a clear match, it goes on to arXiv, Semantic Scholar (which also lists chapters and conference papers) and Open Library for books. Retractions come from the same metadata: the retraction flag in OpenAlex and the update notices in Crossref.

If a database does not answer, Benthic carries on with the others and says so, for example “OpenAlex was not available; checked with the other sources.” A reference is never reported as missing just because one service did not respond. Only the details of the references go to these services, never the text around them (privacy).

What the report shows

The six references from the product page come out like this. Two are invented examples that the site uses on purpose.

Harris et al. (2020), “Array programming with NumPy”, Nature, with its DOI: Verified. The DOI leads to the paper, and title, authors and year match.
LeCun, Bengio and Hinton (2015), “Deep learning”, Nature, cited with 10.1038/nature14538: DOI corrected. That DOI belongs to an item called “Addendum”. A title search finds the paper itself under 10.1038/nature14539, and Benthic puts that one in.
Vaswani et al. (2017), “Attention is all you need”, without any identifier: Verified, with the note “Found by title.” and the arXiv DOI.
Wakefield et al. (1998), The Lancet: Retracted. The Lancet withdrew the paper in 2010, and the registries record it.
An invented example: Smith, J. (2021), “A systematic audit of citation reliability in large language models”: Not found. No database has a work with that title.
Another invented example: Smith, J. (2019), “Deep learning for protein folding at scale”, Science, with a DOI: Mismatch. The DOI exists but belongs to “Cell Proliferation Without Neurogenesis in Adult Primate Neocortex”.

Each row shows the reference as cited and, where the two differ, a line “Found:” with what really exists. “How it was checked” opens the steps behind the verdict: which database was asked, how many results a search returned, and how closely the best title matched. What each of the seven verdicts means is summarised on the product page.

Next step: fetch the real ones

The button “Fetch 3 verified” takes the sources that hold up (Verified, DOI corrected and Deviates) and looks for a free full text: first in your own library, then in eight open-access sources. Each PDF is labelled as publisher version, accepted manuscript or preprint. In the example, the NumPy paper arrives as the publisher version through OpenAlex and “Attention is all you need” as a preprint from arXiv. For “Deep learning” no free copy is known; the result says so and links to the article, through your university library if you entered its proxy, otherwise to the article page at doi.org.

The other three stay out of your list. For Not found, one look in your library catalogue is still worth it: the verdict is a strong signal, not proof, since very new or obscure works are sometimes missing from the registries. And a verified source is only half the work. Whether it says what the chat answer claims it says, you find out by reading it (what Benthic leaves to you).