Fuzzy Systems and its Applications

Fuzzy Systems and its Applications

Fuzzy Fusion of Abstraction and Decision Fusion Architecture - based Machine Learning Models for Emotion Recognition

Document Type : Original Article

Authors
Somaye Moghari 1
Mohammad Safaei 2
Maryam Jalali 2
1 Shahrood University of Technology
2 Faculty of Mathematical Sciences, Shahrood University of Technology
10.22034/jfsa.2025.506704.1262
Abstract
Emotion recognition in emotional computing plays an important role in improving human-machine interaction, monitoring mental health, and creating personalized user experiences. Given the growing importance of this field, this paper introduces a machine learning model based on the Abstraction and Decision Fusion Architecture for emotion recognition. The architecture is defined in three layers: abstraction, computation, and fusion. This research introduces two types of abstraction for video data summarization. Then, in the computation layer, two lightweight base models including a fully connected neural network and a shallow convolutional neural network are designed. Finally, in the fusion layer, the decisions of the base models are combined using a proposed fuzzy controller so that the model can make the final decision. Also, this model is tested on the L-SVD dataset for classifying emotion recognition videos and achieves an accuracy of more than 96%. Experimental results show that the proposed model is superior to the existing state-of-the-art models not only in recognition accuracy but also in terms of processing load and memory consumption.
Keywords
Affective Computing
Machine Learning
Abstraction and Decision Fusion Architecture
Fuzzy Fusion
Subjects
[2]    Ahmed, Naveed, Zaher Al Aghbari, and Shini Girija. (2023). A systematic survey on multimodal emotion recognition using learning algorithms. Intelligent Systems with Applications, 17, 200171.
[3]    Bai, Zhongli, Junjie Liu, Fazheng Hou, Yirui Chen, Meiyi Cheng, Zemin Mao, Yu Song, and Qiang Gao. (2023). Emotion recognition with residual network driven by spatial-frequency characteristics of EEG recorded from hearing-impaired adults in response to video clips. Computers in Biology and Medicine, 152, 106344.
[4]    Chen, Jin, Tony Ro, and Zhigang Zhu. (2022). Emotion recognition with audio, video, EEG, and EMG: a dataset and baseline approaches. IEEE Access, 10, 13229-13242.
 
[5]    Chen, Yen-Liang, Chia-Ling Chang, and Chin-Sheng Yeh. (2017). Emotion classification of YouTube videos. Decision Support Systems, 101, 40-50.
[6]    Fallah, Mohammad K., Najafi, Mohammadreza, Gorgin, Saeid, and Lee, Jeong-A. (2024). Abstraction and decision fusion architecture for resource-aware image understanding with application on handwriting character classification. Applied Soft Computing, 2024, 111813.
[7]    Fallah, Mohammad K., Najafi, Mohammadreza, Gorgin, Saeid, and Lee, Jeong-A. (2024). An ultra- low-computation model for understanding sign languages. Expert Systems with Applications, 249, 123782.
[8]    Haider, Irfan, Hyung-Jeong Yang, Guee-Sang Lee, and Soo-Hyung Kim. (2023). Robust human face emotion classification using triplet-loss-based deep CNN features and SVM. Sensors, 23(10), 4770.
[9]    Huang, Yahe, and Dongying Bo. (2023). Emotion classification and achievement of students in distance learning based on the knowledge state model. Sustainability, 15(3), 2367.
[10]    Khanna, Deepanshu, Neeru Jindal, Prashant Singh Rana, and Harpreet Singh. (2024). Enhanced spatio-temporal 3D CNN for facial expression classification in videos. Multimedia Tools and Applications, 83(4), 9911-9928.
[11]    Karatay, Buşra, Deniz Beştepe, Kashfia Sailunaz, Tansel Özyer, and Reda Alhajj. (2024). CNN- Transformer based emotion classification from facial expressions and body gestures. Multimedia Tools and Applications, 83(8), 23129-23171.
[12]    Khare, Smith K., Victoria Blanes-Vidal, Esmaeil S. Nadimi, and U. Rajendra Acharya. (2024). Emotion recognition and artificial intelligence: A systematic review (2014–2023) and research recommendations. Information fusion, 102, 102019.
[13]    Lek, Jeniffer Xin-Ying, and Jason Teo. (2023). Academic emotion classification using fer: A systematic review. Human Behavior and Emerging Technologies, 2023(1), 9790005.
[14]    Loewenstern, Yocheved, Noa Benaroya-Milshtein, Katya Belelovsky, and Izhar Bar-Gad. (2024). Automatic Identification of Facial Tics Using Selfie-Video. IEEE Journal of Biomedical and Health Informatics.
[15]    Ma, Zhuang, Ao Li, Jiehao Tang, Jianhua Zhang, and Zhong Yin. (2025). Multimodal emotion recognition by fusing complementary patterns from central to peripheral neurophysiological signals across feature domains. Engineering Applications of Artificial Intelligence, 143, 110004.
[16]    Miranda Calero, Jose A., Laura Gutiérrez-Martín, Esther Rituerto-González, Elena Romero-Perales, Jose M. Lanza-Gutiérrez, Carmen Peláez-Moreno, and Celia López-Ongil. (2024). WEMAC: Women and emotion multi-modal affective computing dataset. Scientific Data, 11(1), 1182.
[17]    Moontaha, Sidratul, Franziska Elisabeth Friederike Schumann, and Bert Arnrich. (2023). Online learning for wearable EEG-based emotion classification. Sensors, 23(5), 2387.
 
[18]    Nguyen, Van-Ho, Nghia Nguyen, Thuy-Hien Nguyen, Yen-Nhi Nguyen, Mai-Thu Dinh, and Dung Doan. (2025). Customer emotion detection and analytics in hotel and tourism services using multi- label classificational models based on ensemble learning. Annals of Operations Research, 1-31.
[19]    Pandeya, Yagya Raj, and Joonwhoan Lee. (2021). Deep learning-based late fusion of multimodal information for emotion classification of music video. Multimedia Tools and Applications, 80(2), 2887-2905.
[20]    Pei, Guanxiong, Qian Shang, Shizhen Hua, Taihao Li, and Jia Jin (2024). EEG-based affective computing in virtual reality with a balancing of the computational efficiency and recognition accuracy. Computers in Human Behavior, 152, 108085.
[21]    Peiran L, Linbo T, Xizheng Y. University of Wisconsin Madison. (2023) L-SVD: A Comprehensive Video Dataset for Emotion Recognition, Available: https://github.com/PeiranLi0930/
emotionnet.
[22]    Radzi, Nor Haizan Mohamed, and Haslina Hashim. (2024). Research on Emotion Classification Based on Multi-modal Fusion. Baghdad Science Journal, 21(2 (SI)), 0548-0548.
[23]    Song, Peipei, Dan Guo, Xun Yang, Shengeng Tang, and Meng Wang. (2024). Emotional video cap- tioning with vision-based emotion interpretation network. IEEE Transactions on Image Processing.
[24]    Wei, Jie, Guanyu Hu, Xinyu Yang, Anh Tuan Luu, and Yizhuo Dong. (2024). Learning facial ex- pression and body gesture visual information for video emotion recognition. Expert Systems with Applications, 23
Fuzzy Systems and its Applications
Volume 8, Issue 2 - Serial Number 17
Open Access Statement
December 2025
Pages 43-63

  • Receive Date 14 February 2025
  • Revise Date 22 September 2025
  • Accept Date 01 October 2025