RESULTS · Published · Updated
The best AI scribe for dermatology
Hanah was the best AI scribe we tested on an eczema consultation. Heidi came second and Lyrebird was third.
The test
The consultation is a GP video appointment from the PriMock57 set: four days of an itchy, sore red rash on the chest, hands and inner elbows that was keeping the patient awake at night. Six AI scribes recorded it five times each, and every note was graded blind against the 39 clinical facts said in the consultation. We rank scribes by time to a finalised note, which is the wait after pressing stop plus the time a clinician needs to review the note and fix what is wrong. The recording, transcripts, notes and grading reasons are on the test page.
Results
| Rank | Scribe | Time to a finalised note | Note errors per note | Coverage | Hallucination flags per note | Word error rate |
|---|---|---|---|---|---|---|
| 1 | Hanah | 2 min 01 s | 1.2 | 98% | 0.4 | 14.3% |
| 2 | Heidi | 3 min 17 s+1 min 16 s | 8.0+6.8 | 84%−14% | 0.4 | 16.9%+2.5% |
| 3 | Lyrebird | 4 min 02 s+2 min 02 s | 12.0+10.8 | 86%−13% | 3.8+3.4 | 18.7%+4.4% |
| 4 | Freed | 4 min 44 s+2 min 43 s | 6.6+5.4 | 89%−9% | 0.6+0.2 | 20.3%+5.9% |
| 5 | CliniScripts | 4 min 58 s+2 min 57 s | 12.8+11.6 | 74%−24% | 1.0+0.6 | 14.9%+0.6% |
A Hanah note was ready to finalise 2 minutes 1 second after the consultation ended, on average. It captured 98% of the facts, with 1.2 note errors and 0.4 hallucination flags per note. Reviewing and fixing it took 78% less time than writing the note yourself.
Heidi averaged 3 minutes 17 seconds. Its quickest run, at 2 minutes 41 seconds, was slower than Hanah's slowest at 2 minutes 13 seconds. Hanah was quicker on average, captured more of the consultation (98% against 84%) and had fewer note errors (1.2 against 8.0). Heidi and Hanah had the fewest hallucination flags on this consultation at 0.4 per note.
Lyrebird was third at 4 minutes 2 seconds and Freed fourth at 4 minutes 44 seconds. Freed had the second fewest note errors at 6.6.
Hanah also had the lowest word error rate on this consultation at 14.3%, with CliniScripts close at 14.9%.
Findings
- The rash affected the chest, hands and inner elbows. Lyrebird heard "elbows" as "eyebrows" in all five transcripts.
- The patient said the rash was not on the face. Hanah recorded this in every run, Heidi in one of five and Lyrebird in none.
- The patient could not go hiking because of the rash. Hanah and Lyrebird recorded this in every run, and Heidi never recorded it in full.
- Most of Hanah's hallucination flags in this test were routine negatives, such as "bowels normal", for questions the doctor never asked.
Our Heidi review covers how Heidi did across all 12 consultations, with a video walkthrough.
Frequently asked questions
What is the best AI scribe for dermatology?
Hanah was the best AI scribe we tested on an eczema consultation. Its notes were ready to finalise 2 minutes 1 second after the consultation, on average, and captured 98% of the facts. Heidi was second and Lyrebird third.
How was the dermatology consultation tested?
The test is a GP video appointment from the PriMock57 set: four days of an itchy, sore red rash on the chest, hands and inner elbows. Six AI scribes recorded it five times each, and every note was graded blind against the 39 clinical facts said in the consultation.
Which AI scribe transcribed the eczema consultation most accurately?
Hanah had the lowest word error rate at 14.3%, with CliniScripts close at 14.9%.
Figures in this article come from the ScribeStandard benchmark. See the full results, each test and how we test.
Explore results