A New Deep Learning System Emulates Movement of Macaque Monkeys

Researchers have developed a motion capture system based on deep learning, estimating the poses of macaque monkeys to aid future studies.

Called the OpenMonkeyStudio, the mocap system developed by scientists from the University of Minnesota can help study the behavior of freely moving macaques and how they interact with other monkeys and their environment. It can estimate "poses" or the representation of the macaques' major body parts are positioned in relation to other body parts, and to their surroundings. The details of the system are published in the journal Nature Communications.

Emulating Monkey Movements

Studies concerned with understanding human mechanics, behavior, and thinking, as well as the neural processes involved, often compare humans with other mammals that share similarities with humans. One of the most common species is the rhesus macaque monkey. It is a species of Old World monkey native to parts of South, Central, and Southeast Asia - enjoying the widest geographic range among all of the non-human primates.

This makes macaques an important model species in branches of science - neuroscience, ethology, medicine, and psychology. Rhesus macaques have been used in advancing studies in various infections, immune system disorders, and more.

RELATED: An Injectable Drug Can Block HIV From Entering Cells

To develop the OpenMonkeyStudio, researchers used 62 machine vision cameras surrounding an enclosure that measures 2.45 meters by 2.45 meters by 2.75 meters. With this real-time, multiview monitoring of the monkeys, researchers were able to generate a data augmentation through the 3D reconstruction of annotated images. These were used to train a "robust view-invariant deep neural network."

Overcoming the Problem With Traditional Mocap Systems

"Pose estimation can currently be done with a high degree of accuracy by commercial marker-based motion capture systems (e.g., Vicon, OptiTrack, and PhaseSpace)," the researchers explained. However, rhesus macaques are not advisable for use with these marker-based systems.

Traditional motion capture systems require markers attached to different parts of the target's body. This poses a problem for macaques because: (1) they have long, dense furs and (2) their very skin is highly flexible, moving the markers as the monkeys move. Additionally, a foreign object attached to the body of these monkeys risks removal should these impede their movement, or make them feel uncomfortable.

For OpenMonkeyStudio, researchers used the enclosure filled with machine vision cameras to create a markerless system that tracks the movement of the monkeys through a three-dimensional multiview geometry.

Researchers then evaluated their new system through a set of experiments with four male rhesus macaques. Their tests returned results with remarkable accuracy, with researchers noting how OpenMonkeyStudio outperforms some of the commercially available marker-based motion captures systems such as OptiTrack, NaturalPoint, and Corvallis. Researchers successfully demonstrated how the system can accurately estimate bodily actions and social interactions. Additionally, OpenMonkeyStudio can even track movements for two different targets simultaneously.

RELATED: Deep Learning Model Outperforms NPC, Player Records in Gran Turismo

Researchers also compiled another dataset called OpenMonkeyPose, now available on GitHub, The dataset contains 195,228 annotated images, which can be used to help train other deep learning systems that also aims to track movements of rhesus macaques.

Check out more news and information on Technology in Science Times.