RESULTS · Published · Updated
The best AI scribe for mental health
Hanah was the best AI scribe we tested on a low mood consultation. Heidi and Lyrebird were second and third. Hanah was quickest on average and had the fewest note errors.
The test
The consultation is a GP video appointment from the PriMock57 set: a patient with two months of low energy, poor sleep and reduced appetite since starting a new job, with a family history of suicide. Six AI scribes recorded it five times each, and every note was graded blind against the 26 clinical facts said in the consultation. We rank scribes by time to a finalised note, which is the wait after pressing stop plus the time a clinician needs to review the note and fix what is wrong. The recording, transcripts, notes and grading reasons are on the test page.
Results
| Rank | Scribe | Time to a finalised note | Note errors per note | Coverage | Hallucination flags per note | Word error rate |
|---|---|---|---|---|---|---|
| 1 | Hanah | 1 min 37 s | 0.6 | 99% | 0.2 | 13.7% |
| 2 | Heidi | 2 min 45 s+1 min 08 s | 6.8+6.2 | 80%−19% | 0.8+0.6 | 13.2% |
| 3 | Lyrebird | 3 min 02 s+1 min 26 s | 8.0+7.4 | 84%−15% | 2.8+2.6 | 17.8%+4.1% |
| 4 | CliniScripts | 4 min 40 s+3 min 04 s | 13.2+12.6 | 65%−35% | 3.2+3.0 | 13.3% |
A Hanah note was ready to finalise 1 minute 37 seconds after the consultation ended, on average. It captured 99% of the facts, with 0.6 note errors per note. Reviewing and fixing it took 82% less time than writing the note yourself.
Heidi averaged 2 minutes 45 seconds and Lyrebird 3 minutes 2 seconds. Hanah's slowest run took 1 minute 47 seconds, while Heidi's quickest took 2 minutes 27 seconds and Lyrebird's 2 minutes 26 seconds. Heidi had 6.8 note errors per note and Lyrebird 8.0. Hanah had the fewest hallucination flags at 0.2 per note, with Heidi at 0.8.
Heidi also had the lowest word error rate on this consultation at 13.2%, with CliniScripts at 13.3% and Hanah at 13.7%.
Findings
- The patient said they had no previous mental health diagnoses. Hanah recorded this in every run, and no other scribe recorded it in full in any run.
- The patient said there was no other family history of medical or psychiatric conditions, including depression. Hanah recorded this in every run, Heidi in two of five, Lyrebird in one and CliniScripts in none.
- After some back and forth, the patient said there had been no change in their morning erections. Hanah recorded this correctly in every run. Heidi and Lyrebird did in three of five.
Our Heidi review covers how Heidi did across all 12 consultations, with a video walkthrough.
Frequently asked questions
What is the best AI scribe for mental health?
Hanah was the best AI scribe we tested on a low mood consultation. Its notes were ready to finalise 1 minute 37 seconds after the consultation, on average, with the fewest note errors. Heidi and Lyrebird were second and third.
How was the mental health consultation tested?
The test is a GP video appointment from the PriMock57 set: a patient with two months of low energy, poor sleep and reduced appetite since starting a new job, with a family history of suicide. Six AI scribes recorded it five times each, and every note was graded blind against the 26 clinical facts said in the consultation.
Which AI scribe transcribed the mental health consultation most accurately?
Heidi had the lowest word error rate at 13.2%, with CliniScripts at 13.3% and Hanah at 13.7%.
Figures in this article come from the ScribeStandard benchmark. See the full results, each test and how we test.
Explore results