Latest:
Maria Morais Research Fellow, Intraoperative Performance Analytics Laboratory (IPAL), Department of General Surgery, Northwell Health and Filippo Filicori, System Chief, Surgical Innovation, Northwell Health | Program Director, MIS Fellowship, Director of Surgical Research, Hosfra/Northwell, Associate Professor of Surgery, Hosfra/Northwell

Artificial Intelligence in the Surgical Landscape: Augmentation Over Automation

By Maria Morais Research Fellow, Intraoperative Performance Analytics Laboratory (IPAL), Department of General Surgery, Northwell Health and Filippo Filicori, System Chief, Surgical Innovation, Northwell Health | Program Director, MIS Fellowship, Director of Surgical Research, Hosfra/Northwell, Associate Professor of Surgery, Hosfra/Northwell

In the modern operating room, data-driven technologies, such as Artificial Intelligence and its derivatives, how surgical performance used to assess and optimize. To the most skeptical, this trend can be threatening and overwhelming. But as procedures become more complex and efficiency remains a priority, understanding and integrating these tools is no longer optional; it’s imperative. Rather than replacing clinical expertise, these technologies are emerging as powerful allies, offering new methods for analyzing operative metrics, supporting technical refinement and improving clinical outcomes, to the point that the biggest threat to the professional in the current scenario might be not minding these changes at all. There is no need for every surgeon in the world to become a data scientist, but it is interesting to get familiar with terms, the current landscape, and the prospects of these technologies.

With collaboration and careful implementation, AI can be a powerful ally in enhancing both surgical education and patient outcomes.

To provide context, Artificial Intelligence (AI) constitutes the ability of a computer to learn, infer, and reason. The ultimate goal is to make the computer perform these tasks as similarly as possible to a human mind. Some derivatives of AI include machine-learning (ML), which is the ability of a computer to learn from an established dataset, and computer vision (CV), which is the machine understanding of images and videos. These terms are prevalent in most ongoing surgical studies in this field.

The idea behind getting AI and robotic systems developed is to use them as analytical support tools that amplify human skills and provide cognitive augmentation. Much like a stethoscope or CT scan, these technologies require expert interpretation and clinical judgment. Their roles are to support standardized assessments, accurate predictions, and education with minimally biased metrics.

These roles are already standing out in surgical practice. For instance, CV models linked to cameras in the operating room (OR) have been used to detect the workflow automatically [1]. This allows for the recognition of surgical phases, to estimate OR times, and enhances the logistical flow of the hospital. This also aids in recognition of error-event sequences, an awareness that may help surgeons in preparing for cases, avoiding new errors and mitigating their consequences. [1] Still in the OR workflow, more integrated software has been developed to capture and compile extensive real-time data, including patient-related, environmental, and intraoperative audiovisual data. This showed feasibility in identifying intraoperative factors that influence performance and patient outcomes, with newer software updates targeted to reduce documentation burden. [2]

Automated analysis of surgical videos was also adopted for surgical video segmentation, automatically identifying different phases of the procedure and anatomic structures, being able to highlight important landmarks during surgery [3]. For instance, such tools could detect when the critical view of safety is achieved during a cholecystectomy. This capability not only can assist surgeons by providing a real-time “green light” to proceed with the dissection but also offers an objective means of documentation, enhancing both medicolegal protection for the provider and safety for the patient.

Another recent application for automated analysis is in surgical performance evaluation. Historically, the assessment of a surgeon’s technical skill relied on subjective methods, often peer evaluations or video reviews, approaches that are time-consuming and inherently prone to bias. With robotic-assisted platforms like the da Vinci Xi Surgical System (Intuitive Surgical, Sunnyvale, CA), a new, objective method has emerged. These systems automatically collect kinematic data – a concept that refers to motion-related information such as how fast and smoothly the instruments move, how often they rotate, and how much energy is used. Kinematic data can then be translated into measurable indicators of a surgeon’s performance in real time [4–6].

By analyzing these patterns, AI and machine-learning tools can help identify what a good technique looks like, and even predict when things might go wrong. For instance, previous studies on cholecystectomy and hernia repair using robotic platforms revealed that certain patterns of hand movements during surgery (known as kinematic signatures) have been associated with surgeon experience and greater technical proficiency [4,5]. These signatures may serve as useful indicators of a surgeon’s progression during training and help establish objective performance benchmarks. Some have also been linked to patient discomfort after surgery [4,5]. If a tool can recognize these patterns in real time, it could prompt the surgeon to adjust, potentially preventing complications before they occur.

These technologies hold exciting potential not just for training, but also for improving patient care. Research has consistently shown that strong technical skills are linked with fewer complications, shorter hospital stays, and even lower mortality [7]. AI-powered systems are being designed to offer real-time guidance during procedures, not to replace the surgeon, but to act as an extra set of expert eyes. These tools could detect subtle issues, like a knot that isn’t tight enough or a stapler that’s slightly misaligned and alert the surgeon before problems arise. This kind of early feedback may help tailor decisions during surgery, reduce risk, and personalize care. On a broader level, more efficient operations can also lead to shorter procedures, less use of resources, and reduced costs. Hospitals using these tools may be able to increase case volume and optimize scheduling, all without compromising safety.

Still, some challenges must be addressed. Data from different hospitals and surgical platforms often don’t “talk to each other,” making widespread adoption difficult. Privacy and ethical concerns also arise when tracking surgical movements. And importantly, some surgeons may worry that such tools feel like surveillance rather than support.

To move forward, it’s essential to actively involve surgeons in designing these systems, ensure transparency in how data is used, and promote a culture of continuous improvement rather than judgment. With collaboration and careful implementation, AI can be a powerful ally in enhancing both surgical education and patient outcomes.

References:

  1. Bonrath, E. M., Gordon, L. E., and Grantcharov, T. P. 2015. “Characterising ‘near miss’ Events in Complex Laparoscopic Surgery through Video Analysis.” BMJ Quality & Safety 24(8): 516–521. https://doi.org/10.1136/bmjqs-2014-003816.
  2. Møller, K. E., Sørensen, J. L., Topperzer, M. K., Koerner, C., Ottesen, B., Rosendahl, M., Grantcharov, T., and Strandbygaard, J. 2023. “Implementation of an Innovative Technology Called the OR Black Box: A Feasibility Study.” Surgical Innovation 30(1): 64–72. https://doi.org/10.1177/15533506221106258.
  3. Hashimoto, D. A., Rosman, G., Rus, D., and Meireles, O. R. 2018. “Artificial Intelligence in Surgery: Promises and Perils.” Annals of Surgery 268(1): 70–76.
  4. Heredia, M. L., Garcia-Peraza-Herrera, L. C., Hashimoto, D. A., et al. 2023. “Objective Assessment of Surgical Skill Using Deep Learning and Robotic Kinematic Data.” Surgical Endoscopy. https://doi.org/10.1007/s00464-023-10481-4.
  5. Fernandes, S., Khan, R., Yadav, M. A., et al. 2023. “Understanding Surgeon Skill and Patient Outcomes Through Instrument Kinematics.” Surgical Endoscopy. https://doi.org/10.1007/s00464-023-10285-6.
  6. Agarwala, A., Palepu, R., Qureshi, S., et al. 2025. “Next-Generation Real-Time Surgical Skill Feedback: Integrating AI and Robotic Data.” Surgical Endoscopy. https://doi.org/10.1007/s00464-025-11599-3.
  7. Fecso, A. B., et al. 2017. “Objective Assessment of Technical Skill in Surgery: A Systematic Review.” Annals of Surgery 267(6): 1041–1047. https://doi.org/10.1097/SLA.0000000000001959.