About 40% of general practitioners in the United Kingdom now use artificial intelligence tools to take notes during patient appointments. A new study suggests those tools carry significant risks alongside their benefits.
Researchers at the University of Edinburgh reviewed cases involving what are called ambient AI scribes, tools that listen to doctor-patient conversations and convert speech to text, generating written notes and letters automatically. Their findings were published in the British Medical Journal's digital health and AI publication, according to the BBC.
The study found the tools can help clinicians focus on complex tasks and reduce administrative workloads. At one clinic in Dudley in the West Midlands, a transcription tool called Heidi was credited with cutting a backlog of patient letters from six months down to 14 days.
But the researchers identified several serious concerns. The tools sometimes miss important elements of consultations that are not spoken aloud, including facial expressions, gestures, and a patient's emotional state. The AI summaries tend to prioritize clinical data, which can come at the expense of what researchers described as the patient's story and personal experience of illness.
Privacy concerns also surfaced. Patients may hold back sensitive information, including disclosures about substance abuse, domestic abuse, or mental health struggles, when they know the conversation is being recorded and processed by an AI system.
The researchers also found that manual notetaking serves functions that go beyond simply capturing information. Writing notes by hand supports a clinician's reasoning and reflection on what a patient has said. When AI takes over that task, the report warned of what it called "cognitive offloading," a process by which doctors rely on the tools to handle mental work they would otherwise do themselves.
The consequences can be significant. The study found instances where clinicians did not recognize their own AI-generated notes or could not remember a patient the next time they came in for an appointment.
The researchers concluded that these systems must be designed to preserve the patient's voice rather than overwrite it, and called for more study on the medium and long-term effects of the tools in real clinical settings.
