EPFL researchers used the popular big language ChatGPT3 model to develop a robotic grab for picking tomatoes, demonstrating for the first time the potential of this AI tool to work with humans in designing robots.

In a study published in the journal Nature Machine Intelligence, Josie Hughes, head of the Computational Robot Design and Fabrication Laboratory at the School of Engineering, EPFL graduate student Francesco Stella and Cosimo Della Santina from TU Delft used ChatGPT to develop a tomato picker robot. This research provides the basis for the joint design of such devices by humans and LLM. Based on their experience, the researchers describe the opportunities and risks of using artificial intelligence (AI) tools in robotics, which, in their opinion, "can change the way robots are designed, while enriching and simplifying this process."

"Despite the fact that ChatGPT is a language model and its code generation is text-based, it has provided significant insight and intuition for physical design and has shown great potential as a resonator for stimulating human creativity," says Hughes.

At the first stage, the researchers and LLM held an "ideological" discussion to determine the purpose of the robot, design parameters and technical characteristics. The second stage was devoted to the implementation of the robot in the real world, which included finalizing the created LLM code, manufacturing the device and troubleshooting its operation.

At the first stage, the researchers started from a philosophical moment, discussing with LLM the future problems of humanity and defining robotic harvesting as a solution to the problem of global food supply. Then they used LLM's access to global data from scientific publications, technical manuals, books and the media to give the "most likely" answer to questions such as "what characteristics should a robot collector have?".

After determining the basic format of robotics (motor-driven capture for capturing ripe tomatoes), the researchers could ask more specific questions, for example, "what form should the capture have?", and ask ChatGPT to make technical proposals, including materials and computer code for controlling the device.

"While computing was mainly used to help engineers with technical implementation, for the first time an AI system can create ideas for new systems, thus automating high-level cognitive tasks. This may lead to a change of human roles to more technical ones," says Stella.

But it is noted that there are logical and ethical risks associated with the use of ChatGPT, the researchers warn that the role of LLM should be carefully evaluated in the future. For example, the use of ChatGPT raises questions of bias, plagiarism and intellectual property, since it is unclear whether the design created by AI can be considered new.

"In our study, ChatGPT identified tomatoes as the culture "most worthy" for robotic assembly. However, this may be a biased attitude towards cultures that are more illuminated, as opposed to those where there really is a real need. When decisions are made outside the scope of an engineer's knowledge, this can lead to significant ethical, engineering or factual errors," says Hughes.

Despite these caveats, Hughes and her team, based on their experience, concluded that LLMs have great potential to "become a force for good" if properly managed: "Therefore, the robotics community must determine how to use these powerful tools to accelerate the development of robots in an ethical, sustainable and socially meaningful way".