Prairie Doc or Prairie Bot?

Artificial Intelligence or A.I. has gone from Science Fiction to a reality.  This technology continues to evolve and find new applications in the world, including the world of medicine.  With any new advancement, there are pros and cons to be considered when implementing it into regular use, especially in medicine.

In 2023 the Journal of Medical Internet Research published an article where they had ChatGPT take 2 of the 3 USMLE exams.  Step 1 is taken by third year medical students and all physicians must pass all three tests in order to become licensed as a physician.  The program was given 4 different practice tests and scored between 44 to 64.4% correct.  Over 60% is considered a passing score.  The headlines read that ChatGPT passes medical boards.

While this headline was shocking, reading the article gives more nuances to that declaration.  There were three different AI programs used, and only one had a passing score for one of the four sample tests, the easiest step 1.  When the exams themselves were broken down, the AI was able to answer the “easy” questions with the greatest accuracy.  Questions were graded on a score from 1 (easy) to 5 (difficult).  However, it was unable to answer any of the level 5 questions.  When the program was given a “hint”, the performance increased to 22% correct of the level 5 questions. Students taking the test do not get hints.

For this article I decided to ask Chat GPT to list “5 ways AI will improve the practice of medicine in the next 5 years” and “5 ways AI may harm the practice of medicine in the next 5 years.”  Here is what the program came up with.

Here are the 5 ways ChatGPT thought AI could improve the practice of medicine

1. Improved diagnostics and early detection
2. Personalized treatment plans
3. Efficient administrative workflows
4. Virtual health assistants and remote monitoring
5. Drug discovery and development

Here are the 5 ways ChatGPT thought AI could worsen the practice of medicine

1. Over-reliance on AI
2. Bias in algorithms
3. Privacy and data security risks
4. Job displacement and role redefinition
5. Increasing costs for AI implementation

AI is a tool that can be used.  Like any tool in medicine, it needs to be tested for safety, accuracy, and effectiveness before widespread implementation.  While ChatGPT could easily write this entire article for me in a matter of seconds, it would not be providing you with the personal connection or with the level of reverence and responsibility that each Prairie Doc feels.  We take our motto to provide you with health information that is based in science and built on trust very seriously.  We will watch as this technology develops and advances.  We embrace progress while holding fast to our prairie roots of connection and community.  You do not need to worry about Prairie Doc being replaced by Prairie Bot.

Jill Kruse, D.O. is part of The Prairie Doc® team of physicians and currently practices as a hospitalist in Brookings, South Dakota. Follow The Prairie Doc® at www.prairiedoc.org and on Facebook, Instagram, and Threads featuring On Call with the Prairie Doc®, a medical Q&A show (most Thursdays at 7pm on streaming on Facebook), 2 podcasts, and a Radio program (on SDPB), providing health information based on science, built on trust for 23 Seasons.