Checking
Invented sentences in medical transcripts: Whisper's flaw
In October 2024 the AP reported that OpenAI's Whisper speech recogniser sometimes makes up whole sentences, even treatments that do not exist. A tool built on it had already transcribed millions of medical visits.
OpenAI has described its Whisper speech recognition model as approaching human-level accuracy. But according to an Associated Press report published on 26 October 2024, the model is prone to making up chunks of text or even entire sentences. More than a dozen software engineers, developers and researchers described the problem; the invented passages included racial commentary, violent rhetoric and imagined medical treatments. One transcript named a medication that does not exist.
The full extent is hard to measure, but the reports are many. A university researcher studying recordings of public meetings found invented text in eight out of every ten transcripts he examined. A 2024 study by researchers at Cornell University and the University of Virginia found entire made-up phrases or sentences in roughly 1 per cent of transcripts, and 38 per cent of these contained explicit harms, such as references to violence. The fabrications tend to appear during pauses, background noise or music; according to the study, more often for speakers with longer silences.
OpenAI itself advises against using Whisper in high-risk domains. Even so, it has spread in healthcare. According to the AP, more than 30,000 clinicians and 40 health systems use a Whisper-based tool made by Nabla, which has been used for an estimated 7 million medical visits. The tool erases the original audio for data safety reasons, so the transcript cannot later be compared with the recording. The company said clinicians must edit and approve the notes, and OpenAI said it continually studies how to reduce hallucinations.
What can we learn from it?
A machine transcript is fast and mostly accurate, but it is not a witness statement: it can mishear, and occasionally it can invent. Trouble starts where the recording is gone or nobody checks it. Confiella also transcribes with a Whisper model, on your own computer; that keeps the audio on the machine, but it does not protect against invented sentences. Good practice is simple: before anything goes into notes or a report, compare the parts that matter – quotations, numbers, names, medications, anything that points to risk – with the recording.
Sources
- arXiv: Careless Whisper: Speech-to-Text Hallucination Harms
- Associated Press (Anchorage Daily News): Researchers say an AI-powered transcription tool used in hospitals invents things no one ever said
This article is based on the public sources listed. If you find a mistake, write to us: [email protected]