Researchers at Paris's Pierre and Marie Curie University have created robots that can use experiences from simulated lives to "heal" themselves. This makes robots more autonomous, effective, and robust-and more capable of disaster relief work.
In the experiments the team ran, it took just over one minute for a six-legged robot with two of its legs broken, damaged, or missing to adapt and resume walking. The team was also able to teach a robotic arm with several broken joints or motors to place an object in the right spot.
"One thing we were surprised by was the extent of damage to which the robots could quickly adapt to," study co-author Jean-Baptiste Mouret says. "We subjected these robots to all sorts of abuse, and they always found a way to keep working."
Although robots are tough in the sense that they can survive outer space, deep oceans, and other extreme environments, in other ways they are fragile. This has been a serious barrier to their widespread use outside factory settings. For example, in disaster relief settings, the inability to adapt and continue working despite damage renders them far less useful.
Researchers in this study took their cues from animals, who typically adapt well to damage and injuries. When a mammal loses a limb, for example, they will usually learn how to be ambulatory once more.
"If we send in robots to find survivors after an earthquake, or to put our forest fires, or to shut down a nuclear plant in crisis like Fukushima, we need them to be able to keep working if they become damaged," Mouret says. "In such situations, every second counts, and robots are likely to become damaged because these environments are very unpredictable and hostile. Even in less extreme cases, such as in-home robot assistants that help the elderly or sick, we want robots to keep performing their important tasks even if some of their parts break."
Robots already know how to diagnose their own troubles; however, until now they needed to have a pre-programmed contingency plan for any scenario. But no one can foresee everything, and this is the problem the team set out to solve.
We have reported elsewhere about robots learning by trial-and-error, but often these kinds of processes take more time than a rescue mission has, particularly if the problems are complex. Animals, however, quickly use trial-and-error to resolve problems. The Pierre and Marie Curie team has created a trial-and-error program that lets robots adapt to setbacks in less than two minutes.
"The most important application of these findings is to have robots that can be useful for long periods of time without requiring humans to perform constant maintenance," Mouret says.
The program uses insights about the ways that trial-and-error really works in the brains of animals. They don't start with the blank slate of the robot. "Instead, they have intuitions about different ways to behave," Mouret says. "These intuitions allow them to intelligently select a few, different behaviors to try out and, after these tests, they choose one that works in spite of the injury. We made robots that can do the same."
The team set their robots up with a lifetime of experiences before starting their tasks, and it was this set of intuitions that made the difference. Now, the robot has already gone through simulations that show it the options when something goes wrong. This allows the robot to make predictions about what will work to solve its problem based on its "experience."
"We do not pre-compute anything like 'find a gait that works if a leg is missing,'" Mouret says. "What we do with the simulator is simply to say 'find as many different ways to walk as you can.'"
The robot can draw on what are just like intuitions whenever it faces a real injury; these then steer its trial-and error experiments and find a viable solution.
"Once damaged, the robot becomes like a scientist," lead author of the study, Antoine Cully says. "It has prior expectations about different behaviors that might work, and begins testing them. However, these predictions come from the simulated, undamaged robot. It has to find out which of them work, not only in reality, but given the damage."
The program allows robots to run experiments with patterns of behavior, ruling out those that are ineffective.
"For example, if walking, mostly on its hind legs, does not work well, it will try walking mostly on its front legs," Cully says. "What's surprising is how quickly it can learn a new way to walk. It's amazing to watch a robot go from crippled and flailing around to efficiently limping away in about two minutes."
The researchers believe this technique will help robots adapt in a variety of environments that present a plethora of obstacles. "Our approach can work with any robot," Mouret says.
Study co-author Danesh Tarapore, a roboticist, discussed potential applications in the statement:
"It could enable the creation of robots that can help rescuers without requiring their continuous attention," Tarapore says. "It also makes easier the creation of personal robotic assistants that can continue to be helpful even when a part is broken."
The researchers highlight the broad-ranging implications for this work.
"They could in principle be applied to having robots learn almost anything," Mouret says. "Until now, nearly all approaches for having robots learn took many hours, which is why videos of robots doing anything are often extremely sped up. Watching them learn in real-time was excruciating, much like watching grass grow. Now we can see robots learning in real-time, much like you would watch a dog or child learn a new skill. Thus, for the first time, we have robots that learn something useful after trying a few different things, just like animals and humans."
At first blush it seems that providing the lifetime of experience for robots would be time-consuming and costly, but the researchers explain, "our approach is actually very cost-effective, because it does not require complex internal sensors," Mouret said. "The robot only needs to know how well it performs its task. It does not need to know the precise reason why it cannot perform the task as expected. That allows tremendous cost savings, because a robot does not need to have a suite of expensive self-diagnosing sensors woven throughout its body."
The team's next step is to test their strategy in simulated real-world situations with more advanced robots. The researchers will immediately explore how this technique could assist disaster-relief robots like those that will compete in the upcoming Defense Advanced Research Projects Agency (DARPA) Robotics Challenge.
