Imagine asking Alexa or Siri or Cortana what's the weather like that day 'face to face', somewhat like talking to a living and breathing human being. This was what Doug Roble, senior director of software of research and development at Digital Domain, an Oscar-winning visual effects studio in California, presented in a 15 minute Ted Talk, during Ted's annual Ted Conference hosted in Vancouver this week. It seems that Roble's team was able to merge 3D motion capture and machine learning to generate hyper-realistic, human models.
These models' perceptiveness will he heightened, and they can mimic subtle facial mannerisms, map eye-lash movement, and even recreate the appearance of blood flow to make its cheeks look flushed.
Past psychological studies have provided evidence that certain facial expressions elicit emotional responses, a facet of communication that voice assistants simply can't access, at least for now. But Roble and his team at Digital Domain have created a method to give Siri and its counterparts a mug.
Using a device called the "light stage" at the University of Southern California, Roble captured thousands of images of his face under various lighting conditions. This allowed him to recreate a strikingly realistic 3D model of his face, which a deep neural network maniupates using real-time motion capture data.
Roble gave a demonstration of the technology by stepping on stage fully suited in motion capture gear, which he then used to render a mirror image of himself, called DigiDoug, on the screen behind him.
"This is going to be used to give virtual assistants a body and a face, a humanity," he said. "I already love it that when I talk to virtual assistants they answer back in a soothing human-like voice, now they'll have a face and you'll get all of those non-verbal cues that make communication so much easier."
Roble said being able to recreate such lifelike humans could revolutionize the film industry, bring faraway friends, family, and colleagues closer together, and give voice assistants digital flesh and blood.
So far, the database is only made up of Roble's face, but with more volunteers, it could be used to render a body for Alexa. This way the assistant can connect with users on another level and potentially even be able to provide emotional support further down the line.
"You're going to be able to tell when a virtual assistant is busy, confused, or concerned about something," said Roble.
In other words, we might all soon be able to meet our virtual friends face-to-face, a step that could enable more intimate and fulfilling interactions with our technology than ever before.
