The Hidden Challenges of AI in Radiology: From Protocol Selection to Cognitive Load
By Elanchezhian Somasundaram, Associate Director Children’s AI Imaging Research Center (CAIIR), Cincinnati Children’s Hospital Medical Center
The conversation around AI in Radiology has matured beyond whether it works to a much harder question: how do we make it work in practice? From convolutional networks to emerging agentic systems, algorithms can now perform complex diagnostic tasks with remarkable accuracy. But as someone who has led AI deployment in a pediatric hospital, I can say the real obstacles do not lie in the algorithms. They lie in validation, integration, and the operational complexities of making AI a trusted partner in the radiologist’s workflow.
Beyond Building Models: Where Real Work Begins
For years, Radiology AI has focused on performance metrics such as AUC and Dice. Today, building a competent model is rarely the bottleneck, given sufficient resources. At Cincinnati Children’s, we deployed several models in shadow mode, running silently alongside workflows without influencing clinical decisions. Despite strong validation numbers, we quickly uncovered hidden gaps: patient population differences, protocol changes, and rare cases that eroded performance. These lessons showed that accuracy on validation datasets does not equal reliability in daily practice. Real progress comes from rigorous local validation, iterative workflows, and thoughtful collaboration with clinicians.
The Myth of One Size Fits All AI
Working at a pediatric institution, we approach adult-trained models with caution. Anatomy evolves dramatically in childhood, and tools built on adult data are often suboptimal. Even among commercial products, only a handful are cleared for pediatric use. Heterogeneity extends further: scanners differ, protocols shift, and even prediction class definitions vary.
We saw this when an abdominal MRI model trained on routine T2 scans faltered on hepatic studies. Adaptability proved essential, not just in fine-tuning weights but also in how outputs were displayed. Some radiologists preferred quantitative measurements, others visual overlays. In domains such as fetal imaging, where variability is high, radiologists even want interactive workflows to guide imperfect models. Success in Radiology AI requires personalization that respects the diversity of patients, equipment, and clinical practice.
Radiology needs a new operating system that treats AI not as an add on but as a core capability.
Why Operational Infrastructure Determines Success
Even the best algorithm fails without robust operations. Getting the right study to the right model at the right time depends on unglamorous but essential details: standardized DICOM (Digital Imaging and Communications in Medicine) tags, consistent protocol names, and clean data pipelines. Early investments here prevent endless downstream workarounds.
In our deployments, automated CI/CD pipelines and monitoring systems were as critical as the models themselves. Radiologists and physicists played a central role in shaping these frameworks. Their input ensured that AI tools aligned with subtle but important workflow expectations. Without such infrastructure, AI risks remaining a proof of concept rather than a clinical asset.
Taming Drift: Continuous Monitoring in the Real World
Model drift is inevitable. In Radiology, it often stems not only from population changes but also from scanner upgrades and protocol shifts. Continuous monitoring must therefore extend beyond accuracy metrics to include clinical feedback loops and system-level changes.
As part of the AAPM (American Association for Physicists in Medicine) Task Group for CAD AI (Computer Aided Diagnosis with AI), I am helping develop recommendations on how routine equipment quality control can be tied into drift monitoring. Phantom studies, calibration checks, and dose metrics may provide early signals of degradation long before radiologists notice. A joint effort, with physicists ensuring equipment stability, engineers monitoring outputs, and clinicians providing feedback, offers a path toward trustworthy long-term deployment.
Reducing, Not Adding, to the Radiologist’s Cognitive Load
The ultimate test of AI is whether it lightens or increases the workload of radiologists. Early pilots demonstrated that, rather than assisting readers, AI outputs that needed frequent double-checking slowed them down. By refining confidence thresholds and presenting results only when reliable, we can reduce friction and earn clinician trust. But this is a collaborative process, and it is imperative to have radiologists who are enthusiastic and incentivized for the initial clinical validation.
AI must blend seamlessly into existing workflows, surfacing information only when it reduces uncertainty. Anything that adds another window or requires extra clicks risks being dismissed, no matter how accurate. The measure of success is whether radiologists end their day with less fatigue, not more.
Rethinking Radiology’s Operating System for the AI Era
The PACS, RIS, and dictation systems that anchor modern Radiology were never designed for an AI-first world. They are very siloed, inflexible and ill-suited for the dynamic, interactive workflows that AI promises. While specialized AI platforms have provided a degree of standardization, it is now clear they are a temporary bridge. A fundamental rethink of enterprise architecture and industry standards is inevitable to support the future of our field, from federated learning to interactive, multimodal agentic copilots.
Fortunately, this work has already begun. Community driven initiatives are leading the charge: MONAI (Medical Open Network for AI) Deploy, an open-source framework I help co-chair, is creating a standardized pathway for model deployment, while OHIF offers a blueprint for a modern, zero footprint DICOM viewer. Our working group is also developing standards for containerized application packages that define inputs, triggers, and outputs consistent with DICOM and IHE (International Health Enterprise) guidelines, allowing models to integrate cleanly into clinical workflows.
Industry partners must join and support these collaborative efforts. A community driven ecosystem, built on open standards, is the most viable foundation for the next generation of Radiology AI infrastructure that paves the way for truly interactive, agent-driven systems.
Conclusion
The path forward is not about building the flashiest algorithm. It is about validation in real populations, adaptable design, operational rigor, continuous monitoring, and human-centered workflows. Radiology needs a new operating system that treats AI not as an add on but as a core capability. Only then can the promise of AI be realized, not by overwhelming radiologists, but by empowering them to focus on what they do best: caring for patients.