Heidi review: the AI scribe tested on 12 consultations

Video transcript

The full narration of Heidi - A Real Review. Each timestamp opens the video at that chapter.

0:00 Introduction

Most AI scribe reviews stop at the screen: the interface, the templates, how polished the note looks. Very few do a deep dive on the quality of the notes and accuracy of the transcripts. Heidi is one of the best-known AI medical scribes and we decided to do a technical review - what it heard, what it wrote, what it missed, and how long a clinician needs to fix it. Then we compared it with five other popular scribes.

0:24 How we tested

Every scribe heard the same twelve consultations, five times each. A blind AI rater graded each note against an inventory of every clinical fact said. Doctor-written figures come from published studies. Our test is stricter than theirs, so that comparison favours the doctors.

0:40 Speed

Heidi's biggest strength is speed. From pressing stop to a finalised note, including review, it averaged three minutes twenty-five seconds. Writing the note by hand takes a median of just over eight minutes, so Heidi saves about five minutes per note.

0:54 Review effort

Reviewing and correcting a Heidi note took just under three minutes. Editing effort came to about 22% of writing the note from scratch. Both ranked second of six.

1:03 Hallucinations

Heidi also invented very little. It averaged one hallucination flag per note, only just behind Hanah's 0.8 and well ahead of CliniScripts at 1.4. That is below the 1.7 reported for doctor-written notes, and Heidi had the fewest flags, or tied for fewest, in five of the twelve consultations.

1:22 Transcription

Its word error rate was 13.6%, just behind CliniScripts and Hanah, and well ahead of Lyrebird, PatientNotes and Freed. But clinically significant transcript errors tell a different story: about 21 per full run, more than double Hanah and CliniScripts, and only narrowly fewer than Freed.

1:40 Transcription errors in context

Those errors matter most when they reach the note. In a heart failure consultation, the doctor asked about metoprolol, and the patient corrected them: bisoprolol. Heidi listed metoprolol, the wrong beta-blocker, in four of its five notes. Lyrebird and PatientNotes made the same mistake. Hanah, Freed and CliniScripts did not.

Not every slip carried through. Heidi's transcript turned furosemide into "fruit" every time, yet every note still named furosemide. But in an allergic reaction, "dizzy" became "busy" in all five transcripts, and none of Heidi's notes recorded the dizziness. CliniScripts caught it in four of five.

2:16 Completeness

The main weakness is completeness. Heidi's notes captured about 81% of the clinical facts, missing roughly one in five. That is just behind Lyrebird and just ahead of CliniScripts, but well short of Freed and Hanah, and Heidi did not lead on coverage in any consultation.

2:31 Note errors

Counting missed facts and hallucinations together, Heidi averaged about nine errors per note, narrowly behind Freed and ahead of CliniScripts. The published study of doctor-written notes found five. Heidi's runs varied from about seven and a half to ten, and none of its notes were ready to finalise without edits.

2:50 Errors reaching the record

If 70% of those errors go uncorrected at review, as Biro et al. reported in 2025, about 630 errors would reach the record per hundred Heidi notes, against 500 for doctor-written notes.

3:02 Best and worst cases

Results varied by specialty. Heidi's best case was the low mood consultation, in mental health. It had the fewest hallucination flags of any scribe, the lowest word error rate, just ahead of CliniScripts, and the second-fewest note errors. It also had the fewest flags in the suspected TIA consultation.

Its weakest area was cardiology. On chest pain, it was fifth of six on both note errors and coverage, ahead of only PatientNotes. And heart failure, where its word error rate was the lowest of all, is where the wrong beta-blocker appeared.

3:35 Resource use

Heidi was also the heaviest scribe we tested. It used about 45 megabytes of data per consultation, the most of any scribe in every consultation, and around 145 on a fifty-minute visit. In our stress test, it averaged the most memory, at about 1.9 gigabytes, a little more than Lyrebird. That matters on a weak connection or an older computer.

3:56 Where Heidi ranks

So where does Heidi sit in the field? It ranked second on speed, review time, editing effort and hallucinations. It was third on transcription and note errors, fourth on coverage, and last on data and memory use.

4:08 Heidi vs Hanah

Against the benchmark leader, Hanah, the gap is clear: Hanah was ahead on every measure we track. The closest were word error rate and hallucination flags. The widest was coverage, 93% against 81, and Hanah made fewer than half as many errors per note. Hanah also finalised notes about a minute sooner, using about a fifth of the data.

4:28 Heidi vs the rest

Against the other four, Heidi was faster to a finalised note than all of them. Freed captured more facts and made slightly fewer errors per note, but took more than a minute longer. CliniScripts edged Heidi on word error rate and had far fewer significant transcript errors, yet left more errors in its notes. Lyrebird and PatientNotes trailed Heidi on both speed and note errors.

4:50 Verdict

Heidi is fast, rarely invents details, and beat writing the note by hand by about five minutes. Its weak spots are completeness, transcription slips that can reach the note, and the heaviest data and memory use we measured. It sits second of six on speed and hallucinations, and third on note errors, narrowly behind Freed and ahead of CliniScripts, Lyrebird and PatientNotes. Like every scribe we tested, its notes still need careful review before they're finalised.

5:18 More testing

We’ll be adding more cases, more scribes, and deeper analysis of each product. Subscribe to stay up to date, and like the video if you found it helpful.

5:26 Full results

Explore Heidi's results, consultation by consultation, at ScribeStandard dot com.