AI Falls Short in Emergency Triage—Nurses and Doctors Still Lead the Way

A new study presented at the European Emergency Medicine Congress confirms what many nurses already know: when it comes to triaging patients in emergency departments, human judgment still leads the way.

While artificial intelligence (AI) tools like ChatGPT show some promise in identifying the most urgent cases, they fall short when used on their own—especially compared to experienced clinical staff.

Study Aims to Support Overworked ED Staff

Dr. Renata Jukneviciene, a postdoctoral researcher at Vilnius University in Lithuania, led the study to explore whether AI could ease the growing burden in emergency departments.

“We conducted this study to address the growing issue of overcrowding in the emergency department and the escalating workload of nurses,” said Dr. Jukneviciene.

Using a mix of paper and digital questionnaires, the researchers asked six doctors and 51 nurses at Vilnius University Hospital Santaros Klinikos to triage real clinical cases pulled from PubMed. Participants categorized each case using the Manchester Triage System, a five-level urgency scale.

The same cases were also processed by ChatGPT-3.5, an AI tool not specifically trained for medical purposes.

Key Findings: AI Underperformed in Most Areas

While AI showed some strengths, it underperformed in most of the triage categories when compared to both doctors and nurses.

🩺 Overall Accuracy:

• Doctors: 70.6%

• Nurses: 65.5%

• AI: 50.4%

🚨 Sensitivity (Recognizing True Urgent Cases):

• Doctors: 83.0%

• Nurses: 73.8%

• AI: 58.3%

Doctors had the highest scores across all levels of urgency, including cases involving surgery and medical therapies.

AI Outperformed Nurses in One Critical Area

Interestingly, AI did better than nurses in the most urgent triage category, with higher scores in both accuracy and specificity.

⚠️ Most Urgent Cases (Category 1):

• Accuracy:
  
  

  • AI: 27.3%

  • Nurses: 9.3%

• Specificity:
  
  

  • AI: 27.8%

  • Nurses: 8.3%

This suggests AI may be more conservative when flagging life-threatening cases—possibly over-triaging patients as a safety buffer.

Therapeutic and Surgical Cases: Doctors Lead, AI Trails

When it came to recommending appropriate care pathways, doctors again performed best.

🔪 Surgical Case Reliability:

• Doctors: 68.4%

• Nurses: 63%

• AI: 39.5%

💊 Therapeutic Case Reliability:

• Doctors: 65.9%

• AI: 51.9%

• Nurses: 44.5%

While AI did outperform nurses in therapeutic cases, it lagged behind both groups in surgical assessments.

AI Has Potential—but Needs Oversight

Despite its limitations, researchers say AI could still play a supporting role in the emergency department, especially for:

• Flagging the most urgent patients
  
  
• Assisting new or less experienced staff
  
  
• Acting as a second-opinion tool in high-volume settings

However, over-triaging could cause new challenges if not carefully monitored.

“Hospitals should approach AI implementation with caution and focus on training staff to critically interpret AI suggestions,” said Dr. Jukneviciene.

Study Limitations and Next Steps

Researchers acknowledge several limitations:

• Small participant pool
  
  
• Single-center study
  
  
• AI used outside a real-time clinical setting
  
  
• No access to live patient data or vitals
  
  
• ChatGPT-3.5 not trained for medical use

Still, the study used real clinical cases and addressed a clinically relevant problem: how to manage ED overcrowding and staff shortages.

Follow-up studies are already in the works. The team plans to:

• Test newer, medically-tuned AI models
  
  
• Involve larger, multi-center groups
  
  
• Include ECG interpretation
  
  
• Explore AI use in nurse triage training, especially for mass casualty incidents

Expert Commentary: Proceed With Caution

Dr. Barbra Backus, emergency physician in Amsterdam and chair of the EUSEM abstract committee, was not involved in the study but weighed in:

“AI has the potential to be a useful tool for many aspects of medical care and it is already proving its worth in areas such as interpreting X-rays. However, it has its limitations, and this study shows very clearly that it cannot replace trained medical staff for triaging patients coming in to emergency departments.”

She emphasized that human oversight remains essential, and AI systems should be thoroughly tested before clinical deployment.

🤔Nurses, what do you think—could AI tools ever become a trusted part of triage in your emergency department, or is human judgment too important to replace? Share your thoughts in the discussion forum below!