Researchers from Carnegie Mellon University and Meta AI have developed an innovative way of teaching robots multiple skills simultaneously, inspired by how children explore and learn. The team created an AI agent, RoboAgent, with manipulation abilities comparable to a 3-year-old child.

As CMU associate professor Vikash Kumar explained, most robots today are highly specialized for singular tasks. In contrast, their goal was a generalist AI agent that adapts to new situations and expands behaviors over time. Like human children, RoboAgent learns through both passive observation and active experience.

RoboAgent can perform 12 distinct manipulations across various scenes, demonstrating an AI platform adaptable to changing conditions. Crucially, the researchers validated their approach in real-world settings rather than just simulations, while utilizing far less data than previous efforts.

The system combines self-experience with observational learning, akin to a parent demonstrating tasks to guide a child's development. Beyond its own trial-and-error, RoboAgent learns from internet videos much like children observe their environment.

As Mohit Sharma, a CMU robotics graduate student explained, this allows RoboAgent to gain prior knowledge of how people manipulate objects and use skills. Watching similar skills in diverse settings teaches the agent key consistencies and variations for completing tasks.

According to CMU professor Shubham Tulsiani, this training paradigm brings us closer to generalist robots that perform varied tasks in evolving conditions. By leveraging limited self-experience plus abundant external data, RoboAgent can operate in unstructured real-world settings.

The researchers have openly published their models, code, hardware drivers and full dataset. They hope this transparency will enable rapid advancement towards fundamentally capable, generalist robotic agents.

In emulating the multifaceted way children absorb information and explore environments, the CMU and Meta AI teams have opened new possibilities for adaptable, real-world AI. RoboAgent represents a milestone in robotic intelligence that grasps concepts once considered exclusively human.