There’s an algorithm that can take low-quality video footage of a crowd of people, and detect the group’s emotions. Developed by National Research University Higher School of Economics (HSE) researchers Alexander Tarasov and Andrey Savchenko, the algorithm is faster than similar technologies, rendering results in one hundredth of a second.
The system is also “comparable with the existing group-level emotion recognition techniques in terms of recognition accuracy (75.5 percent),” and needs 5 MB of memory, according to HSE.
Image analysis first takes place with the MTCNN neural network, which is generally utilized for detecting small faces. The process then moves on to a fully convolutional network that extracts each face’s features. The network, at this stage, according to HSE, was “preliminarily trained to classify emotions of faces with very low resolution, no bigger than a profile picture on social media.”
Next, the system makes its concluding verdict on the group’s emotions. “The final decision on the emotion (negative, positive, or neutral) of the whole group is made by an ensemble of known classifiers (random forest and support vector machines) applied to the weighted sum of feature vectors of all detected faces,” according to HSE.
Researchers have been able to achieve adequate group-level emotion recognition, but only with high resolution video and images with close-up shots of faces, according to HSE. However, typical surveillance cameras usually take low-resolution footage, and the face close-ups are few and far between since they’re typically mounted in high places.
As a result, the newly developed algorithm can lend its services to many surveillance systems, and avoid conflict by spotting emotion changes at a variety of public events, such as concerts, sports games, and rallies. In addition to security purposes, supermarkets can also analyze customers’ reactions to different product promotions.
The research was described in the article, “Emotion Recognition of a Group of People in Video Analytics Using Deep Off-the-Shelf Image Embeddings,” published in Analysis of Images, Social Networks and Texts.