At the prestigious IEEE International Conference on Robotics and Automation (ICRA) last week, NVIDIA research teams unveiled several groundbreaking developments poised to push the boundaries of robotics capabilities. Among the highlights was novel work on geometric fabrics – a hot topic that garnered significant attention at the event.

In the realm of robotics, trained policies like geometric fabrics are inherently approximate, meaning they don't always execute commands perfectly. Robots may move too quickly, collide with objects, or exhibit jerky motions. To mitigate these limitations, an additional layer of low-level controllers intercepts and adjusts the commands from the policy before execution.

The NVIDIA Robotics Research Lab in Seattle presented a solution that vectorizes these low-level controllers, making them available during both training and deployment phases. This unique approach leverages GPU-accelerated reinforcement learning (RL) tools to guide policy learning through the nominal fabric behavior, systematizing simulation-to-reality (sim2real) transfers.

"By iterating quickly between training and deployment, we could adjust the fabric structure and add substantial random perturbation forces during training to achieve a higher level of robustness than in previous work," explained a researcher from the Seattle team.

Building upon their award-winning "Geometric Fabrics" paper from ICRA 2022 and renowned DeXtreme research on in-hand manipulation, the NVIDIA team merged these lines of work. The resulting methods train DeXtreme policies atop vectorized geometric fabric controllers, keeping robots safer and guiding learning while enabling successful sim2real deployments.

"With geometric fabric controllers underlying the policy and protecting the robot, we could be much more liberal in deploying and testing policies without worrying about the robot destroying itself," a researcher noted, recounting previous experiments where real-world tests were extremely taxing on the physical robot.

Beyond geometric fabrics, NVIDIA highlighted four other robotics papers at ICRA:

• SynH2R proposed a framework to generate realistic synthetic human grasping motions for robot training data.
• Out of Sight, Still in Mind demonstrated approaches for robots to reason about occluded and reappearing objects.
• Point Cloud World Models introduced novel point cloud-based control policies to improve learning efficiency and robustness.
• SKT-Hang tackled the tricky problem of using robots to hang various objects on different supporting structures.

Additionally, NVIDIA unveiled ORBIT-Surgical, a cutting-edge surgical robot simulation framework powered by NVIDIA Isaac Sim on the Omniverse platform. Leveraging GPU parallelization, it facilitates studying robot-augmented surgical skills and generating synthetic data for active perception tasks. The team plans to release the underlying application as open-source upon publication.

The DefGoalNet paper also explored shape servoing – the robotic task of controlling objects to create specific goal shapes.

NVIDIA's robotic developments at ICRA underscored the company's commitment to pushing technological boundaries. From enhancing training data generation to improving policy robustness and expanding sim2real capabilities, these innovations promise to propel the field of robotics forward.

"NVIDIA's research at ICRA exemplifies our drive to advance robotics through cutting-edge techniques like geometric fabrics and state-of-the-art simulation tools," stated a company spokesperson. "By tackling complex challenges spanning manipulation, perception, and control, we aim to unlock new possibilities for robotic systems in real-world applications."

As the robotics landscape rapidly evolves, NVIDIA's pioneering work, presented on the prestigious ICRA stage, signals an exciting future where robots become increasingly capable, adaptable, and robust – poised to revolutionize industries and tackle intricate tasks once thought insurmountable.