Tech giant OpenAI’s artificial intelligence-powered transcription tool, Whisper, has been touted as having near “human level robustness and accuracy”. However, according to interviews with over a dozen software engineers, developers, and academic researchers, Whisper has a major flaw – it is prone to making up chunks of text or entire sentences. These fabrications, known as hallucinations, can include racial commentary, violent rhetoric, and even imagined medical treatments.
Experts warn that such fabrications are problematic because Whisper is being used in a slew of industries worldwide, including translating and transcribing interviews, generating text in popular consumer technologies, and creating subtitles for videos. The full extent of the problem is difficult to discern, but researchers and engineers have frequently come across Whisper’s hallucinations in their work.
AI Transcription Tool Whispers (Image via Getty)
A University of Michigan researcher found hallucinations in eight out of every 10 audio transcriptions he inspected, while a machine learning engineer discovered hallucinations in about half of the over 100 hours of Whisper transcriptions he analyzed. A third developer found hallucinations in nearly every one of the 26,000 transcripts he created with Whisper. Even in well-recorded, short audio samples, hallucinations were prevalent, with a recent study uncovering 187 hallucinations in over 13,000 clear audio snippets.
Experts warn that such mistakes could have “really grave consequences”, particularly in hospital settings, where incorrect transcriptions could mislead medical professionals and harm patients. Alondra Nelson, a former leader of the White House Office of Science and Technology Policy, emphasized the need for AI regulation. Researchers and advocates have called for the federal government to consider AI regulations, at minimum, requiring OpenAI to address the flaw.
