For immediate release on Wednesday 13 September 2023

Barcelona, Spain: The artificial intelligence chatbot ChatGPT performed as well as a trained doctor in suggesting likely diagnoses for patients being assessed in emergency medicine departments, in a pilot study to be presented at the European Emergency Medicine Congress, which starts on Saturday [1].

Researchers say a lot more work is needed, but their findings suggest the technology could one day support doctors working in emergency medicine, potentially leading to shorter waiting times for patients.

The study was by Dr Hidde ten Berg, from the department of emergency medicine and Dr Steef Kurstjens, from the department of clinical chemistry and haematology, both at Jeroen Bosch Hospital, 's-Hertogenbosch, The Netherlands.

Dr ten Berg told the Congress: “Like a lot of people, we have been trying out ChatGPT and we were intrigued to see how well it worked for examining some complex diagnostic cases. So, we set up a study to assess how well the chatbot worked compared to doctors with a collection of emergency medicine cases from daily practice.”

The research, which is also published this month in the Annals of Emergency Medicine [2], included anonymised details on 30 patients who were treated at Jeroen Bosch Hospital’s emergency department in 2022. The researchers entered physicians’ notes on patients’ signs, symptoms and physical examinations into two versions of ChatGPT (the free 3.5 version and the subscriber 4.0 version). They also provided the chatbot with results of lab tests, such as blood and urine analysis. For each case, they compared the shortlist of likely diagnoses generated by the chatbot to the shortlist made by emergency medicine doctors and to the patient’s correct diagnosis.

They found a large overlap (around 60%) between the shortlists generated by ChatGPT and the doctors. Doctors had the correct diagnosis within their top five likely diagnoses in 87% of the cases, compared to 97% for ChatGPT version 3.5 and 87% for version 4.0.

Dr ten Berg said: “We found that ChatGPT performed well in generating a list of likely diagnoses and suggesting the most likely option. We also found a lot of overlap with the doctors’ lists of likely diagnoses. Simply put, this indicates that ChatGPT was able suggest medical diagnoses much like a human doctor would.

“For example, we included a case of a patient presenting with joint pain that was alleviated with painkillers, but redness, joint pain and swelling always recurred. In the previous days, the patient had a fever and sore throat. A few times there was a discolouration of the fingertips. Based on the physical exam and additional tests, the doctors thought the most likely diagnosis was probably rheumatic fever, but ChatGPT was correct with its most likely diagnosis of vasculitis.

“It’s vital to remember that ChatGPT is not a medical device and there are concerns over privacy when using ChatGPT with medical data. However, there is potential here for saving time and reducing waiting times in the emergency department. The benefit of using artificial intelligence could be in supporting doctors with less experience, or it could help in spotting rare diseases.”

Professor Youri Yordanov from the St Antoine Hospital emergency department (APHP Paris), France, is Chair of the EUSEM 2023 abstract committee and was not involved in the research. He said: “We are a long way from using ChatGPT in the clinic, but it’s vital that we explore new technology and consider how it could be used to help doctors and their patients. People who need to go to the emergency department want to be seen as quickly as possible and to have their problem correctly diagnosed and treated. I look forward to more research in this area and hope that it might ultimately support the work of busy health professionals.”

(ends)

[1] Abstract no: OA66, “ChatGPT Effectively Generates Differential Diagnosis Using Emergency Department Physician Notes” by Hidde Ten Berg, in the Out-of-the-hospital and the newest technology session, 09:42 hrs CEST, Tuesday 19 September Room 131.

[2] “ChatGPT and Generating a Differential Diagnosis Early in an Emergency Department Presentation”, by H. ten Berg et al, In press, Annals of Emergency Medicine. DOI: https://doi.org/10.1016/j.annemergmed.2023.08.003

Note: When obtaining outside comment, journalists are requested to ensure that their contacts are aware of the embargo on this release.

Funding: No external funding

About European Society of Emergency Medicine (EUSEM)

We are pleased to support the European Resucitation Council's initiative for International CPR Guidelines that consider economic disparities and diverse realities; based on expert recommendations from various low-, middle-, and high resource settings.

Read the full media release here

We are pleased to announce the results of EUSEM Board Election.

We would like to congratulate Dr Robert Leach elected as President-Elect, Dr Diana Cimpoesu elected as Honorary Treasurer, Dr Niccolò Parri elected as Vice president and Dr Tatjana Rajkovic elected as Honorary Secretary.

The newly elected board members will take their positions following the ratification of the General Assembly on 19 September.

For EM-Day 2023 we conducted 3 surveys about safety, for patients/general public, for professionals and for patient organisations. 

The survey (1) included 1256 responses from 101 different countries; 70% of respondents were from Europe.  Among respondents, 80% reported that monitoring devices were available, and 74% reported that protocols for high-risk medication and for triage (66.19%) were available in their ED. The area of greatest concern was the disproportionate imbalance between needs and the availability of staff at times of greatest flow, considered sufficient by only 22%  of doctors and 20%  of nurses. Other critical issues were overcrowding due to boarding and a perceived lack of support from hospital management. Despite these difficult working conditions, 83% of the professionals said they were proud to work in the ED. This survey highlighted that most health professionals identify the ED as an environment with specific safety issues. The main factors appeared to be a shortage of personnel during busy periods, overcrowding due to boarding, and a perceived lack of support from hospital management. The results were published in the European Journal of Emergency Medicine.

Dr Jim Connolly, EUSEM president writes in his editorial (2): "Amongst the headline results the authors found that about 90% of professionals feel that at times the number of patients in their ED exceeded the capacity of the department to provide safe care and that overcrowding was currently a regular, serious problem. Overcrowding is not just an issue of discomfort or loss of dignity but adds to inefficiency and carries a substantial risk of harm and increased mortality"

Read further:

(1) Patient safety in emergency departments: a problem for health care systems? An international survey

(2) With safety in mind