Please use this identifier to cite or link to this item: https://hdl.handle.net/20.500.11851/8625
Title: A Multi-Modal Emotion Recognition System Based on CNN-Transformer Deep Learning Technique
Authors: Karatay B.
Bestepe D.
Sailunaz K.
Ozyer T.
Alhajj R.
Keywords: CNN
Deep Learning
emotion
emotion classi-fication
Transformer
Classification (of information)
CNN
Deep learning
Detection methods
Emotion
Emotion classi-fication
Emotion detection
Emotion recognition
Learning models
Multi-modal
Transformer
Deep learning
Issue Date: 2022
Publisher: Institute of Electrical and Electronics Engineers Inc.
Source: Karatay, B., Bestepe, D., Sailunaz, K., Ozyer, T., & Alhajj, R. (2022, March). A Multi-Modal Emotion Recognition System Based on CNN-Transformer Deep Learning Technique. In 2022 7th International Conference on Data Science and Machine Learning Applications (CDMA) (pp. 145-150). IEEE.
Abstract: Emotion analysis is a subject that researchers from various fields have been working on for a long time. Different emotion detection methods have been developed for text, audio, photography, and video domains. Automated emotion detection methods using machine learning and deep learning models from videos and pictures have been an interesting topic for researchers. In this paper, a deep learning framework, in which CNN and Transformer models are combined, that classifies emotions using facial and body features extracted from videos is proposed. Facial and body features were extracted using OpenPose, and in the data preprocessing stage 2 operations such as new video creation and frame selection were tried. The experiments were conducted on two datasets, FABO and CK+. Our framework outperformed similar deep learning models with 99% classification accuracy for the FABO dataset, and showed remarkable performance over 90% accuracy for most versions of the framework for both the FABO and CK+ dataset. © 2022 IEEE.
Description: 7th International Conference on Data Science and Machine Learning Applications, CDMA 2022 -- 1 March 2022 through 3 March 2022 -- -- 177931
URI: https://doi.org/10.1109/CDMA54072.2022.00029
https://hdl.handle.net/20.500.11851/8625
ISBN: 9781665410144
Appears in Collections:Bilgisayar Mühendisliği Bölümü / Department of Computer Engineering
Scopus İndeksli Yayınlar Koleksiyonu / Scopus Indexed Publications Collection
WoS İndeksli Yayınlar Koleksiyonu / WoS Indexed Publications Collection

Show full item record

CORE Recommender

SCOPUSTM   
Citations

1
checked on Sep 23, 2022

WEB OF SCIENCETM
Citations

1
checked on Sep 24, 2022

Page view(s)

32
checked on Dec 26, 2022

Google ScholarTM

Check

Altmetric


Items in GCRIS Repository are protected by copyright, with all rights reserved, unless otherwise indicated.