Two artificial intelligence (AI) programmes, including ChatGPT, have passed the U.S. Medical Licensing Examination (USMLE), according to two recent papers.
The papers highlighted different approaches to using large language models to take the USMLE, which is comprised of three exams: Step 1, Step 2 CK, and Step 3.
ChatGPTopens in a new tab or window is an artificial intelligence (AI) search tool that mimics long-form writing based on prompts from human users. It was developed by OpenAI and became popular after several social media posts showed potential uses for the tool in clinical practice, often with mixed results. Â
According to the researchers, “ChatGPT performed at or near the passing threshold for all three assessments without any specialised training or reinforcement.”
Furthermore, ChatGPT’s explanations displayed a high degree of concordance and clarity.
The paper also cites that “these results suggest that large language models (LLMs) may have the potential to assist with medical education and, potentially, clinical decision-making.” The USMLE is a high-stakes, comprehensive, three-step standardised testing programme covering all topics in physicians’ knowledge spanning basic science, clinical reasoning, medical management, and bioethics.
The difficulty and complexity of questions are highly standardised and regulated, making them an ideal input substrate for AI testing. Â
However, this is not the first time the chatbot has aced an examination. A few days back, professors at the University of Pennsylvania’s Wharton School of Business discovered that the ChatGPT could easily successfully complete examinations on a typical MBA core course, operations management.
