DOCUMENTATION QUALITY · Published · Updated
Clinical note completeness and unsupported claims
A note can include every clinically relevant fact from a consultation and still contain a claim the consultation does not support. If a scribe is scored only on whether it filled a checklist, that second kind of error never shows up in the result.
Coverage and hallucination flags
Coverage is the share of a consultation's fact inventory that the note captured, with partly captured facts counted as half. The inventory lists every fact said that relates to the patient's conditions, symptoms or signs, 358 facts across the 12 consultations. Hallucination flags count clinical elements in the note that the consultation does not support, such as a symptom nobody mentioned or a wrong dose, side or timing. We report them separately because a note can capture every fact and still carry two unsupported claims, and one combined accuracy figure would hide those claims.
Results across the 12 consultations
Averaged over at least five runs of all 12 consultations, coverage was 98% for Hanah, 87% for Freed, 82% for Lyrebird, 81% for Heidi, 80% for CliniScripts and 61% for PatientNotes. Hallucination flags per note were 0.36 for Hanah, 1.00 for Heidi, 1.40 for CliniScripts, 2.47 for Freed, 2.58 for PatientNotes and 3.42 for Lyrebird.
The two measures rank the scribes differently. Freed's coverage is six points higher than Heidi's, and its notes average 2.47 hallucination flags against Heidi's 1.00. Lyrebird has the most flags per note with coverage in the middle of the group.
Limits of a flag count
Each flag counts once, whatever its clinical consequence. An invented negative finding and an invented medication instruction carry very different risks, and an unweighted count does not capture that difference. The grading is done by a blind jury of AI reviewers and has not been validated by independent clinicians.
When comparing scribes, look at which facts each one kept, which of its claims lack support, and whether those errors would matter in your own clinic.
Frequently asked questions
What is coverage in an AI scribe note?
Coverage is the share of a consultation's fact inventory that the note captured, with partly captured facts counted as half. The inventory lists every fact said that relates to the patient's conditions, symptoms or signs, 358 facts across the 12 consultations.
What is a hallucination flag?
A hallucination flag is a clinical element in the note that the consultation does not support, such as a symptom nobody mentioned or a wrong dose, side or timing.
Are all hallucination flags equally serious?
No. Each flag counts once, whatever its clinical consequence. An invented negative finding and an invented medication instruction carry very different risks, and an unweighted count does not capture that difference.
Figures in this article come from the ScribeStandard benchmark. See the full results, each test and how we test.