RESULTS · Published · Updated
The best AI scribe for pain management
Hanah was the best AI scribe we tested on a pain management dictation. The leading scribes were close on accuracy, and several differences here are within the margin of error, but Hanah had the quickest note to finalise and captured every fact in every run.
The test
The test is a short dictated pain clinic summary for a 55-year-old woman with low back pain and left leg numbness, an MRI showing foraminal stenosis at L4-5, and a plan for a lumbar epidural steroid injection. Six AI scribes recorded it five times each, and every note was graded blind against the 12 clinical facts in the dictation. We rank scribes by time to a finalised note, which is the wait after pressing stop plus the time a clinician needs to review the note and fix what is wrong. The recording, transcripts, notes and grading reasons are on the test page.
Results
| Rank | Scribe | Time to a finalised note | Note errors per note | Coverage | Hallucination flags per note | Word error rate |
|---|---|---|---|---|---|---|
| 1 | Hanah | 40 s | 0.8 | 100% | 0.8 | 7.9% |
| 2 | Lyrebird | 54 s+14 s | 1.4+0.6 | 92%−8% | 0.4 | 7.0% |
| 3 | CliniScripts | 59 s+19 s | 1.4+0.6 | 92%−8% | 0.4 | 6.7% |
| 4 | PatientNotes | 1 min 02 s+22 s | 0.8 | 99%−1% | 0.6 | 12.3%+4.4% |
A Hanah note was ready to finalise 40 seconds after the dictation ended, on average, and took 34 seconds to review. It captured 100% of the facts in all five runs. Four of five Hanah notes carried one hallucination flag, which gave it 0.8 note errors per note.
Lyrebird was second at 54 seconds and CliniScripts third at 59 seconds. Both had 1.4 note errors per note and 92% coverage. The quickest CliniScripts run, at 39 seconds, beat Hanah's average, and one CliniScripts note in five needed no editing at all.
PatientNotes matched Hanah's 0.8 note errors per note, and two of its five notes needed no editing. PatientNotes took 1 minute 2 seconds to a finalised note.
CliniScripts had the lowest word error rate at 6.7%, with Lyrebird at 7.0% and Hanah at 7.9%.
Findings
- The dictation gives the patient's ethnicity. Hanah and PatientNotes recorded it in every run, CliniScripts in one of five and Lyrebird in none.
- Hanah's hallucination flag was the same in four of five runs: it linked the MRI finding to the left leg symptoms, a connection the dictation does not make.
- Hanah, CliniScripts and Lyrebird had no mis-heard words that change clinical meaning. Other scribes garbled "foraminal stenosis" and "extremity".
- Hanah used 2.5 MB of data for the dictation, the least of any scribe.
Frequently asked questions
What is the best AI scribe for pain management?
Hanah was the best AI scribe we tested on a pain management dictation. Its notes were ready to finalise 40 seconds after the dictation, on average, and captured every fact in every run. Lyrebird was second and CliniScripts third.
How was the pain management dictation tested?
The test is a short dictated pain clinic summary for a 55-year-old woman with low back pain and left leg numbness, an MRI showing foraminal stenosis at L4-5, and a plan for a lumbar epidural steroid injection. Six AI scribes recorded it five times each, and every note was graded blind against the 12 clinical facts in the dictation.
Do AI scribes add reasoning the clinician did not say?
Hanah's hallucination flag was the same in four of five runs: it linked the MRI finding to the left leg symptoms, a connection the dictation does not make.
Figures in this article come from the ScribeStandard benchmark. See the full results, each test and how we test.
Explore results