This robot predicts when you’re going to smile – and smiles back

A humanoid robot can predict whether someone will smile a second before they do, and match the smile on its own face. The creators hope the technology could make interactions with robots more lifelike.

Although artificial intelligence can now mimic human language to an impressive degree, interactions with physical robots often fall into the “uncanny valley”, in part because robots can’t replicate the complex non-verbal cues and mannerisms that are vital for communication.

Now, Hod Lipson at Columbia University in New York and his colleagues have created a robot called Emo that uses AI models and high-resolution cameras to predict people’s facial expressions and try to replicate them. It can anticipate whether someone will smile about 0.9 seconds before they do, and smile itself in sync. “I’m a jaded roboticist, but I smile back at this robot,” says Lipson.

Emo consists of a face with cameras in its eyeballs and flexible plastic skin that has 23 separate motors attached to it by magnets. The robot uses two neural networks: one to look at human faces and predict their expressions and another to work out how to produce expressions on its own face.

The first network was trained on YouTube videos of people making faces, while the second network was trained by having the robot watch itself make faces on a live camera feed. “It learns what its face is going to look like when it’s going to pull all these muscles,” says Lipson. “It’s sort of like a person in front of a mirror, when even if you close your eyes and smile, you know what your face is going to look like.”

Lipson and his team hope that Emo’s technology will improve human-robot interactions, but they first need to broaden the range of expressions the robot can make. They also hope to train it to make expressions in response to what people are saying, rather than simply mimicking another person, says Lipson.