The Role of AI in Medical Affairs: Navigating Reliability and Human Nature
By Peter Seidensticker, MD, PhD, Head of Global Medical Affairs Radiology, Bayer US LLC.
Prologue
Last weekend, I used ChatGPT to generate a list of math problems for my 11-year-old twins. I thought it would save us time and provide them with fresh and engaging challenges. The tool delivered a list quickly and in perfect print out ready formatting. However, as we reviewed the kid’s results together, my daughter pointed out an error in one of the AI-generated solutions. Curious, I looked closer and realized that she was right; the math didn’t add up. I went back to ChatGPT to address the mistake, when the AI responded bluntly, “Sorry, you are correct, I made a mistake.”
It was interesting to observe my kids reacting differently when ChatGPT told them their result was wrong. While my son showed some trust and aimed to recalculate his solution, hoping to match the AI’s conclusion, my daughter doubted the AI’s results and didn’t give in that quickly. Another observation was how ChatGPT reacted to being told it was making simple math mistakes. The blunt acknowledgment of the error, the lack of any visibility into how this mistake could happen, and the missing assurance that it shouldn’t happen again next time, are improvement areas for ChatGPT on its journey to become a more human-friendly and trustworthy tool.
The integration of AI into Medical Affairs presents a great opportunity to enhance efficiency and improve communication and departmental impact.
AI Value in Medical Affairs
Using generative AI functionalities such as Large Language Models (LLMs), e.g., ChatGPT or MS Copilot in Medical Affairs (MA) is generally aimed at enhancing efficiency and supporting decision-making processes.
It is important to make a distinction between AI used in clinical practice, e.g., in Radiology, and the use of generative AI in industry functions like Medical Affairs. In the field of medical imaging, for example, AI provides supports radiologists, in the efficient, accurate, and rapid detection of diseases such as breast and lung cancer, cardio-vascular diseases. The applications used in clinical practice are cleared by the FDA and represent predictive AI models. Generative AI, used in the healthcare industry, does not have the same regulatory framework; therefore, users need to be more aware of risks and potential pitfalls of AI generating novel content.
One of the key advantages LLMs bring to the table is their ability to process large amounts of data quickly. For example, LLMs can efficiently summarize lengthy reports, distilling essential information into concise formats that are easier to digest. This capability is particularly valuable in the context of complex clinical information and data.
LLMs can also assist in generating summaries from complicated meetings, ensuring that critical discussions and decisions are accurately captured. This not only saves time but also minimizes the risk of no- or miscommunication, a common issue in fast-paced high-data environments like MA. By automating administrative tasks, AI allows our high-value professionals to focus on content generation and strategic initiatives, ultimately improving productivity and enhancing the overall effectiveness of the teams.
LLMs excel at research and information retrieval. With its ability to sift through extensive databases and scientific literature, AI can identify relevant studies, guidelines, and emerging clinical trends in real time. This capability is particularly valuable for MA professionals who must always stay up to date in their respective fields.
Finally, AI can optimize real-world evidence generation and study design, patient recruitment, and monitoring with the potential for facilitating patient engagement and education or personalizing patient support materials.
AI Reliability
Despite the evident benefits LLMs offer, there is an inherent challenge to their use, particularly in professional and regulated environments: the reliability of the information generated. While LLMs can analyze data and produce insights at remarkable speeds, the accuracy of these insights is contingent upon the quality, amount, and breadth of underlying data, the recentness of the data, its context, and the way questions or prompts are written. As exemplified in the opening, LLMs are not very good at reflecting on their mistakes or even learning from them at this point. In the context of Medical Affairs, where decisions can impact license to market, patient care, product safety, and treatment outcomes, the stakes are particularly high.
Even more critical than false results, as in the example of the math problems, is that LLMs may inadvertently propagate misinformation, hallucinate and create data that doesn’t exist, or provide outdated data. For example, if an LLM tool is trained on biased or incomplete datasets, it may produce skewed results that do not accurately reflect the current state of medical knowledge. These biases are tough to detect and have the potential to introduce systematic misperception into an organization.
Furthermore, while AI can assist in identifying relevant sources, it lacks the nuanced judgment and reasoning that human professionals bring to the table. In many cases, the offered sources do not perfectly line up with the presented claims or information. MA teams are tasked with interpreting complex data, understanding the context of findings, and evaluating the credibility of sources. These skills are essential for ensuring that the information used in decision-making is reliable, applicable, and defensible. Relying solely on AI-generated insights, at least with today’s performance levels, can lead to critical errors.
Given the challenges associated with AI reliability, it is imperative that MA professionals maintain a proactive approach to fact-checking and source verification. While AI can serve as a powerful tool for enhancing efficiency, it cannot replace critical thinking and expertise that human professionals bring to the process. Instead, AI should be viewed as a complementary resource that augments human capabilities. Human nature leads us to often build trust in sources or systems once we have verified them to be accurate a few times and the challenge is to stay alert to potential AI errors even if 95% of its answers are correct. By fostering a collaborative culture between humans and AI and augmenting the strengths of MA professionals, organizations will be able to maximize the benefits of AI while minimizing the risks associated with mistakes and skewed information.
Conclusion
The integration of AI into Medical Affairs presents a great opportunity to enhance efficiency and improve communication and departmental impact. With its ability to summarize complex information, generate meeting minutes, and conduct thorough searches, AI can significantly improve core areas of MA productivity. However, the reliability of AI-generated insights remains a critical concern. As healthcare organizations embrace AI technologies today, such as LLMs, it is essential to remain vigilant of the produced results and ensure that the information used in decision-making or authority interactions is accurate and trustworthy. A collaborative approach that values both technological advancements and human judgment will be key to achieving success in Medical Affairs. AI will not replace MA professionals, but it is reasonable to assume that it could replace those who are reluctant to embrace technological advancements like AI, missing out on exploring new, maybe more fruitful ways of working.