EEG-Based Emotional State Recognition using Deep Learning Network

Zahedi Haghighi, Seyedeh Saeideh; Sakhaei, Sayed Mahmoud; Daliri, Mohammadreza

doi:10.22041/ijbme.2019.97535.1419

Document Type : Full Research Paper

Authors

¹ M.Sc. Student, Bioelectric Department, Electrical & Computer Engineering Faculty, Babol Noshirvani University of Technology, Babol, IranNoshirvani University of Iran, Babol, Iran

² Assistant Professor, Bioelectric Department, Electrical & Computer Engineering Faculty, Babol Noshirvani University of Technology, Babol, Iran

³ Electrical Engineering Department, Iran University of Science and Technology

https://doi.org/10.22041/ijbme.2019.97535.1419

Abstract

Emotion is one of the most important human quality that plays an important role in life. Also, one way of communicating between human and computer is based on emotion recognition. One way of emotion recognition is based on electroencephalographic signal (EEG). In this paper, according to the non-stationary property of EEG, intrinsic mode functions (IMF) extracted by using empirical mode decomposition (EMD) and then first 3 IMFs selected. Each IMF converts into smaller pieces with a one-second window and power feature has been extracted from each piece. Then, by using a suitable mapping, the position of the electrodes in the 10-20 system becomes the position of the pixels in the picture. The extracted features are considered as pixel color components. To determine the class of valence, the set of all generated pictures is given as input to a deep learning network and output determine the high or low class of valence. The same process is used to determine the class of arousal. To examining the method, the DEAP dataset is used. By choosing 17×17 for the image size, the mean accuracy and standard deviation were obtained of 78.58% and 3.9 for the valence and 78.66% and 3.1 for the arousal which that shows a significant improvement compared to similar tasks.

Keywords

References

[1] S. Brave and C. Nass, “Emotion in human-computer interaction,” human-computer Interact. Handb. Fundam. Evol. Technol. Emerg. Appl., pp. 81–96, 2003.

[2] G. Garcia-Molina, T. Tsoneva, and A. Nijholt, “Emotional brain–computer interfaces,” Int. J. Auton. Adapt. Commun. Syst., vol. 6, no. 1, pp. 9–25, 2013.

[3] Y. Liu, O. Sourina, and M. K. Nguyen, “Real-time EEG-based emotion recognition and its applications,” in Transactions on computational science XII, Springer, 2011, pp. 256–277.

[4] B. Fehr and J. A. Russell, “Concept of emotion viewed from a prototype perspective.,” J. Exp. Psychol. Gen., vol. 113, no. 3, p. 464, 1984.

[5] H. Spencer, The principles of psychology, vol. 1. Appleton, 1895.

[6] M. M. Bradley and P. J. Lang, “Measuring emotion: the self-assessment manikin and the semantic differential,” J. Behav. Ther. Exp. Psychiatry, vol. 25, no. 1, pp. 49–59, 1994.

[7] S. Koelstra et al., “Deap: A database for emotion analysis; using physiological signals,” IEEE Trans. Affect. Comput., vol. 3, no. 1, pp. 18–31, 2012.

[8] P. C. Petrantonakis and L. J. Hadjileontiadis, “Adaptive emotional information retrieval from EEG signals in the time-frequency domain,” IEEE Trans. Signal Process., vol. 60, no. 5, pp. 2604–2616, 2012.

[9] A. Mert and A. Akan, “Emotion recognition from EEG signals by using multivariate empirical mode decomposition,” Pattern Anal. Appl., vol. 21, no. 1, pp. 81–89, 2018.

[10]R. Fontugne, J. Ortiz, D. Culler, and H. Esaki, “Empirical mode decomposition for intrinsic-relationship extraction in large sensor deployments,” in Workshop on Internet of Things Applications, IoT-App, 2012, vol. 12.

[11]N. E. Huang et al., “The empirical mode decomposition and the Hilbert spectrum for nonlinear and non-stationary time series analysis,” in Proceedings of the Royal Society of London A: mathematical, physical and engineering sciences, 1998, vol. 454, no. 1971, pp. 903–995.

[12]D. J. Hand, H. Mannila, and P. Smyth, Principles of data mining (adaptive computation and machine learning). MIT press Cambridge, MA, 2001.

[13]D. S. Naser and G. Saha, “Recognition of emotions induced by music videos using DT-CWPT,” in Medical Informatics and Telemedicine (ICMIT), 2013 Indian Conference on, 2013, pp. 53–57.

[14]M. Chen, J. Han, L. Guo, J. Wang, and I. Patras, “Identifying valence and arousal levels via connectivity between EEG channels,” in Affective Computing and Intelligent Interaction (ACII), 2015 International Conference on, 2015, pp. 63–69.

[15]S. Jirayucharoensak, S. Pan-Ngum, and P. Israsena, “EEG-based emotion recognition using deep learning network with principal component based covariate shift adaptation,” Sci. World J., vol. 2014, 2014.

[16]F. Bahari and A. Janghorbani, “Eeg-based emotion recognition using recurrence plot analysis and k nearest neighbor classifier,” in Biomedical Engineering (ICBME), 2013 20th Iranian Conference on, 2013, pp. 228–233.

[17]P. Arnau-González, M. Arevalillo-Herráez, and N. Ramzan, “Fusing highly dimensional energy and connectivity features to identify affective states from EEG signals,” Neurocomputing, vol. 244, pp. 81–89, 2017.

[18]M. S. Özerdem and H. Polat, “Emotion recognition based on EEG features in movie clips with channel selection,” Brain informatics, vol. 4, no. 4, p. 241, 2017.

[19]Z. Liang, S. Oba, and S. Ishii, “An unsupervised EEG decoding system for human emotion recognition,” Neural Networks, 2019.

[20]Z. Lan, O. Sourina, L. Wang, R. Scherer, and G. R. Müller-Putz, “Domain adaptation techniques for EEG-based emotion recognition: a comparative study on two public datasets,” IEEE Trans. Cogn. Dev. Syst., vol. 11, no. 1, pp. 85–94, 2019.

[21]W.-L. Zheng and B.-L. Lu, “Investigating critical frequency bands and channels for EEG-based emotion recognition with deep neural networks,” IEEE Trans. Auton. Ment. Dev., vol. 7, no. 3, pp. 162–175, 2015.

[22]Y. Li, J. Huang, H. Zhou, and N. Zhong, “Human Emotion Recognition with Electroencephalographic Multidimensional Features by Hybrid Deep Neural Networks,” Appl. Sci., vol. 7, no. 10, p. 1060, 2017.

[23]K. Simonyan and A. Zisserman, “Very deep convolutional networks for large-scale image recognition,” arXiv Prepr. arXiv1409.1556, 2014.

[24]A. Krizhevsky, I. Sutskever, and G. E. Hinton, “Imagenet classification with deep convolutional neural networks,” in Advances in neural information processing systems, 2012, pp. 1097–1105.

[25]N. Srivastava, G. Hinton, A. Krizhevsky, I. Sutskever, and R. Salakhutdinov, “Dropout: a simple way to prevent neural networks from overfitting,” J. Mach. Learn. Res., vol. 15, no. 1, pp. 1929–1958, 2014.

[26]X. Glorot and Y. Bengio, “Understanding the difficulty of training deep feedforward neural networks,” in Proceedings of the thirteenth international conference on artificial intelligence and statistics, 2010, pp. 249–256.

Iranian Journal of Biomedical Engineering

EEG-Based Emotional State Recognition using Deep Learning Network

References

References

Volume 13, Issue 2
August 2019
Pages 95-104

EEG-Based Emotional State Recognition using Deep Learning Network

References

References

Volume 13, Issue 2August 2019Pages 95-104

Volume 13, Issue 2
August 2019
Pages 95-104