Published on
When using a large language model (LLM) assistant for generating emergency department (ED) discharge notes at an academic 2,400-bed tertiary hospital in South Korea, physicians gained some worthwhile efficiencies, according to a comparative effectiveness study published in JAMA Network Open. Six emergency physicians created 300 manual notes, 300 LLM drafts, and 300 LLM-assisted notes from 50 patient cases, which were judged on a Likert scale ranging from 1 to 5. Compared with manual documentation, LLM-assisted notes scored higher in completeness (4.23 vs 4.03), correctness (4.38 vs 4.20), conciseness (4.23 vs 4.11), and clinical utility (4.17 vs 3.85) (P<.001). Yet the LLM-assisted notes, when compared with the LLM drafts, scored lower in completeness (4.23 vs 4.34; P=.001) and correctness (4.38 vs 4.45; P <.001). Overall, the median time physicians spent on the discharge-note task decreased from 69.5 seconds (95% confidence interval, 65.5-78.0) for manual notes to 32.0 seconds (95% confidence interval, 29.5-36.0) for notes with LLM support.Â
Hands-on humans: Overall, manual notes received the lowest scores comparatively on completeness, correctness, and clinical utility. Manual notes ranked in the middle on conciseness, however. The authors conclude that on-site LLM assistance improved documentation quality and reduced workload without compromising accuracy.