Audio-visual Human Emotion Recognition Using Hierarchical Approach

Audio-visual Human Emotion Approach

  • Imran Khan Department of Electronics, University of Peshawar, Peshawar-25120, Pakistan
  • Sana Ul Haq Department of Electronics, University of Peshawar, Peshawar-25120, Pakistan
  • Muhammad Imran Majid Department of Electrical Engineering, Institute of Business Management, Karachi-75190, Pakistan
  • Imtiaz Rasool Department of Electronics, University of Peshawar, Peshawar-25120, Pakistan
  • Muhammad Saeed Shah Department of Electronics, University of Peshawar, Peshawar-25120, Pakistan
Keywords: audio-visual emotion recognition, feature extraction, hierarchical classification, mahalanobis distance, human-computer interaction

Abstract

This paper presents automatic human emotion recognition from audio-visual data. Both the hierarchical and flat approaches were implemented to obtain higher classification performance. The  hierarchical approach was based on Mahalanobis distance. The Interactive Emotional dyadic Motion Capture database (IEMOCAP) was acquired and six different emotions, i.e., anger, excited, frustration, sadness, happiness and neutral state were used for the analysis. The method consisted of feature extraction, normalization, different feature selection and classification techniques. For flat approach, the best accuracy of 95.60% was obtained with Support Vector Machine (SVM) classifier and Info Gain feature selection. In the case of hierarchical approach, the best accuracy of 97.53% was achieved with Random Forest  classifier and Correlation-based Feature Selection (CFS).

Published
2024-10-28
Issue
Vol 67 No 3 (2024): Pakistan Journal of Scientific and Industrial Research Series A: Physical Sciences
Section
Articles