AI-Assisted Triage Systems

Abstract

Emergency department triage determines who receives care first, and getting it wrong has direct consequences - undertriage exposes high-acuity patients to preventable deterioration while overtriage wastes scarce resuscitation resources on patients who did not need them. Human triage is fast but inconsistent: inter-rater agreement on five-level triage scales typically falls between kappa 0.42 and 0.68, and undertriage rates of 5-14% have been reported across major European emergency systems. AI-assisted triage tools promise to reduce this variability by incorporating vital signs, chief complaint language, and historical presentation patterns into real-time acuity predictions. We evaluated five AI-assisted triage systems across 68,244 emergency presentations at 16 European emergency departments in Vienna, Rome, Stockholm, and Lisbon between 2019 and 2023. Systems ranged from rule-augmented early warning scores through natural language processing of chief complaints to multimodal deep learning integrating structured vitals, free-text triage notes, and prior ED visit history. Performance was benchmarked using our Triage Accuracy and Safety Score (TASS), integrating undertriage rate, overtriage rate, time-to-treatment for high-acuity cases, clinician acceptance rate, and demographic equity. The multimodal deep learning system achieved the highest TASS (0.921), reducing undertriage from 9.8% to 2.4% and improving median time-to-treatment for level-1 cases by 11.2 minutes. Clinician acceptance varied from 44% to 87% across systems, driven more by interface design and alert integration than by predictive accuracy, with implications for deployment strategy.