PARTICIPANTS EMOTION RECOGNITION IN VIDEO CONFERENCES USING MULTIMODAL ANALYSIS

Abstract

The paper describes the relevance of the task of analyzing emotions in video communications, emphasizing the importance of understanding the moods of participants to improve the quality of interaction in the modern world, where video meetings are becoming the norm in business, education, and personal contacts. Effective understanding of emotions helps to adapt communication, resolve conflicts at early stages, and improve the overall perception of interaction. Despite the availability of powerful emotion recognition software such as FaceReader and Microsoft Oxford Project, their effectiveness is limited due to their focus on analyzing facial expressions alone. The accuracy of such systems is often inferior due to shortcomings in emotion recognition, which requires improved analysis methods. This paper proposes a novel approach to recognizing emotions of video conference participants through multimodal analysis that combines the processing of physical voice characteristics and facial expressions. The use of convolutional neural networks allows for high accuracy in identifying emotional states, taking into account various distortions of the input data. The technique involves analyzing voice data, normalizing it, and converting it into spectrograms for further processing by a neural network. Special attention is paid to the process of training the network based on the gradient descent method to improve the accuracy of emotion recognition. Experimental results demonstrate the advantage of the proposed method over existing software tools, with an increase in emotion recognition accuracy of up to 79%, which is a significant improvement.