AI Doctor? ChatGPT Nearly Passes US Medical Licensing Exam

ChatGPT has been hitting the headlines since its debut and after passing exams at business and law schools. Now, most recently, the AI has nearly passed the US Medical Licensure Exam (USMLE) necessary to pursue medical practice within the US.

ChatGPT: Revolutionary AI

NDTV reports that Open.AI, which has been an investment recipient of Microsoft and Elon Musk, built the ChatGPT chatbot. ChatGPT specifically stands for "Chat Generative Pre-Trained Transformer" and is a bot that is language-based and that is capable of coming up with responses that mirror those of humans.

Any ChatGPT user can simply ask the bot a question regarding virtually anything. The bot will then offer a fast and detailed paragraph response.

ChatGPT has shown its capabilities recently. NDTV notes how the bot has come up with various instant and difficult essays, marketing pitches, poems, and jokes. The AI even reportedly drafted a speech for a US congressman that he delivered on the House floor.

Aside from these impressive feats, the bot has also successfully passed business and law tests from the Wharton School of Business at the University of Pennsylvania and the University of Minnesota, respectively.

AI Doctor

In the most recent evaluation, the bot was put to test based on 350 queries taken from the USMLE from June 2022. Results were checked by two doctors, while a third expert looked into the discrepancies.

The complete findings have now been reviewed and included in the PLOS Digital Health journal.

According to the Daily Mail, the US Medical Licensing Exam (USMLE) has been utilized since 1992 to gauge knowledge of medical disciplines required for practice. The first step is usually taken at the end of the student's second year in medical school. The second step takes place during their fourth year while the third step is taken once they finish medical school and during their first residency year. Over a hundred thousand students take the USMLE each year.

ChatGPT generated scores that ranged from 52.4% to 75% throughout the three parts of the entire USMLE. The average annual passing threshold is roughly 60%.

These results surpassed those of PubMedGPT, which is a model exclusively designed for the biomedical field. PubMedGPt scored 50.8% on an earlier dataset of questions based on the USMLE.

The study was conducted by a research team from AnsibleHealth. According to them, being able to achieve scores for an extraordinarily difficult specialized test without any human reinforcement is a noteworthy milestone for the clinical maturation of artificial intelligence.

The authors also think that the findings show how ChatGPT could have valuable potential when it comes to medical education.

Clinicians from AnsibleHealth have now started experimenting with the bot by including it in their workflow of constructing reports that are filled with jargon. They even used the AI to write their findings.

While there are limitations and challenges that need to be bypassed with the advancement of AI, Dr. Stuart Armstrong, chief researcher and co-founder of Aligned AI, thinks that this performance is indeed impressive and that people should expect to see more AI successes in the future.

Check out more news and information on Artificial Intelligence in Science Times.