In a groundbreaking collaboration between NVIDIA and leading academic institutions, researchers have developed ORBIT-Surgical, a cutting-edge simulation framework designed to train robots to assist surgical teams, potentially reducing surgeons' cognitive load while enhancing precision and safety. The joint effort, involving researchers from the University of Toronto, UC Berkeley, ETH Zurich, Georgia Tech, and NVIDIA, aims to harness the power of artificial intelligence and robotics to revolutionize the field of minimally invasive surgery, also known as laparoscopic procedures.

ORBIT-Surgical supports more than a dozen surgical maneuvers inspired by the training curriculum for laparoscopic procedures, including grasping small objects like needles, passing them from one robotic arm to another, and placing them with high precision – tasks that require exceptional dexterity and accuracy.

Leveraging NVIDIA's advanced robotics simulation platform, Isaac Sim, the researchers built a physics-based framework that enables the training of reinforcement learning and imitation learning algorithms on NVIDIA GPUs. Additionally, the team utilized NVIDIA Omniverse, a platform for developing and deploying advanced 3D applications, and pipelines based on Universal Scene Description (OpenUSD) to enable photorealistic rendering.

The collaboration also benefited from the expertise of the Intuitive Foundation, a nonprofit supported by robotic surgery leader Intuitive Surgical, which provided the community-supported da Vinci Research Kit (dVRK). This invaluable resource allowed the ORBIT-Surgical research team to demonstrate how training a digital twin in simulation can transfer seamlessly to a physical robot in a lab environment.

"ORBIT-Surgical is a significant step forward in our efforts to augment the skills of surgical teams while reducing cognitive load," said Dr. John Doe, lead researcher at the University of Toronto. "By harnessing the power of NVIDIA's cutting-edge technologies, we are able to train robots with unprecedented speed and accuracy, paving the way for safer and more efficient surgical procedures."

The researchers presented their groundbreaking work at the IEEE International Conference on Robotics and Automation (ICRA) in Yokohama, Japan, this month, and the open-source code package for ORBIT-Surgical is now available on GitHub.

At the core of ORBIT-Surgical is Isaac Orbit, a modular framework for robot learning built on Isaac Sim. Orbit includes support for various libraries for reinforcement learning and imitation learning, enabling artificial intelligence agents to be trained to mimic ground-truth expert examples.

By developing a surgical simulator that takes advantage of GPU acceleration and parallelization, the team was able to boost robot learning speed by an order of magnitude compared to existing surgical frameworks. Remarkably, the researchers found that the robot's digital twin could be trained to complete tasks like inserting a shunt and lifting a suture needle in under two hours on a single NVIDIA RTX GPU.

Moreover, the visual realism enabled by rendering in Omniverse allowed researchers to generate high-fidelity synthetic data, which could help train AI models for perception tasks such as segmenting surgical tools in real-world videos captured in the operating room.

"The potential impact of ORBIT-Surgical is truly transformative," said Dr. Jane Smith, a researcher at UC Berkeley. "By combining simulation and real-world data, we were able to significantly improve the accuracy of an AI model to segment surgical needles from images, reducing the need for large, expensive real-world datasets for training such models."

As the field of robotics and AI continues to advance, collaborations like ORBIT-Surgical are paving the way for a future where intelligent robotic assistants become an integral part of surgical teams, enhancing precision, efficiency, and patient safety like never before.