RESULTS · Published · Updated
The best AI scribe for urology
Hanah was the best AI scribe we tested on a urinary symptoms consultation. Lyrebird and Heidi were close to each other in second and third.
The test
The consultation is a GP video appointment from the PriMock57 set: six days of painful urination, foul-smelling urine, blood in the urine and central lower abdominal pain, in a patient with an antibiotic allergy. Six AI scribes recorded it five times each, and every note was graded blind against the 42 clinical facts said in the consultation. We rank scribes by time to a finalised note, which is the wait after pressing stop plus the time a clinician needs to review the note and fix what is wrong. The recording, transcripts, notes and grading reasons are on the test page.
Results
| Rank | Scribe | Time to a finalised note | Note errors per note | Coverage | Hallucination flags per note | Word error rate |
|---|---|---|---|---|---|---|
| 1 | Hanah | 1 min 46 s | 1.2 | 99% | 0.0 | 8.8% |
| 2 | Lyrebird | 3 min 34 s+1 min 48 s | 10.4+9.2 | 88%−10% | 3.2+3.2 | 10.6%+1.8% |
| 3 | Heidi | 3 min 39 s+1 min 53 s | 11.4+10.2 | 78%−21% | 0.8+0.8 | 10.4%+1.6% |
| 4 | CliniScripts | 4 min 17 s+2 min 31 s | 12.2+11.0 | 75%−24% | 0.6+0.6 | 8.8% |
A Hanah note was ready to finalise 1 minute 46 seconds after the consultation ended, on average. It captured 99% of the facts, with 1.2 note errors per note. Reviewing and fixing it took 80% less time than writing the note yourself.
Lyrebird averaged 3 minutes 34 seconds and Heidi 3 minutes 39 seconds. Heidi's quickest run, at 2 minutes 52 seconds, was slower than Hanah's slowest at 1 minute 54 seconds. Hanah was quicker on average and had less than half the note errors of either, 1.2 against 10.4 for Lyrebird and 11.4 for Heidi. Lyrebird had the second highest coverage at 88%.
Hanah had the lowest word error rate on this consultation at 8.8%, level with CliniScripts. Hanah's transcripts had no mis-heard words that change clinical meaning in any run.
Findings
- The doctor said this infection can be treated without testing the urine. Hanah recorded it in every run, and no other scribe recorded it in any run.
- The patient takes mebeverine. Every other scribe mis-heard the drug name in at least one run, and one heard it as "heparin" and listed heparin, a blood thinner, among the patient's medications in every note.
- The prescription was sent to the patient's nominated pharmacy, to start as soon as possible. Hanah recorded this in full in every run, and Heidi recorded part of it in every run.
Our Heidi review covers how Heidi did across all 12 consultations, with a video walkthrough.
Frequently asked questions
What is the best AI scribe for urology?
Hanah was the best AI scribe we tested on a urinary symptoms consultation. Its notes were ready to finalise 1 minute 46 seconds after the consultation, on average, with the fewest note errors. Lyrebird and Heidi were second and third.
How was the urology consultation tested?
The test is a GP video appointment from the PriMock57 set: six days of painful urination, foul-smelling urine, blood in the urine and central lower abdominal pain, in a patient with an antibiotic allergy. Six AI scribes recorded it five times each, and every note was graded blind against the 42 clinical facts said in the consultation.
Do AI scribes mis-hear drug names?
Yes. In the urinary symptoms consultation, the patient takes mebeverine. One scribe heard it as "heparin" and listed heparin, a blood thinner, among the patient's medications in every note.
Figures in this article come from the ScribeStandard benchmark. See the full results, each test and how we test.
Explore results