نشریه علمی مهندسی پزشکی زیستی

Arousal and Valence Classification of Music Emotion using Music and Demographic Features

Document Type : Full Research Paper

Author

Assistant Professor, Biomedical Engineering Group, School of Engineering Science, College of Engineering, University of Tehran, Tehran, Iran

Abstract
Two of the most prominent human emotions are arousal and valence. In this article, the aim is to answer the question whether predicting arousal and valence emotions arising from listening to music without using physiological signals and only using demographic and musical characteristics can provide appropriate results?. For this purpose, 48 30-second music with very high and very low levels of arousal and valence were selected from the DEAM music collection. Then, each of these music was separately labeled in terms of arousal and valence emotions by 175 Iranian participants with an age range of 14-35 years. These integer labels were from 1 (the lowest rate) to 5 (the highest rate). The root mean square energy, tempo, zero-crossing, spectral flatness, spectral centroid, spectral flux, spectral rolloff, rhythmic Complexity, and chromagram features were extracted from each music. The demographic features were age, gender, education level, economic level, ethnicity, zip code, and the hours of listening to music in each day. Observations related to label 3 (middle rate) were discarded due to the very low number of occurrences of this label compared to other labels, and 8051 observations were used for classification. The entire data was divided into 4 equal, nonoverlapping parts and classified 4 times so that each time one of the parts was used for testing and the rest parts were used for training the model. This process was repeated 10 times and the average results of the test data were calculated for the classification criteria. The arousal and valence emotions were analyzed separately. For classification performance comparison, five different classifiers including neural network, K nearest neighbors, support vector machine, decision tree, and random forest were taken into account. The neural network offered the best classification performance for arousal emotion by 77% accuracy, 90.3% specificity, 77% sensitivity and valence emotion by 79.7% accuracy, 91.2% specificity, 79.7% sensitivity. The results offer that the neural network can be an appropriate classifier for classification of the musical emotions of Iranian society using the music and demographic features.

Keywords

Subjects


  1. Cui, X., Wu, Y., Wu, J., You, Z., Xiahou, J., & Ouyang, M. (2022). A review: Music-emotion recognition and analysis based on EEG signals. Frontiers in Neuroinformatics, 16, 997282.
  2. Aljanaki, A., Yang, Y. H., & Soleymani, M. (2017). Developing a benchmark for emotional analysis of music. PloS one, 12(3), e0173392.
  3. Han, D., Kong, Y., Han, J., & Wang, G. (2022). A survey of music emotion recognition. Frontiers of Computer Science, 16(6), 166335.
  4. Kim, Y. E., Schmidt, E. M., Migneco, R., Morton, B. G., Richardson, P., Scott, J., ... & Turnbull, D. (2010, August). Music emotion recognition: A state of the art review. In Proc. ismir (Vol. 86, pp. 937-952).
  5. Hu, X., & Yang, Y. H. (2017). Cross-dataset and cross-cultural music mood prediction: A case on western and chinese pop songs. IEEE Transactions on Affective Computing, 8(2), 228-240.
  6. Panda, R., Malheiro, R., & Paiva, R. P. (2018). Novel audio features for music emotion recognition. IEEE Transactions on Affective Computing, 11(4), 614-626.
  7. Panda, R., Malheiro, R., & Paiva, R. P. (2020). Audio features for music emotion recognition: a survey. IEEE Transactions on Affective Computing, 14(1), 68-88.
  8. Gómez-Cañón, J. S., Cano, E., Eerola, T., Herrera, P., Hu, X., Yang, Y. H., & Gómez, E. (2021). Music emotion recognition: Toward new, robust standards in personalized and context-sensitive applications. IEEE Signal Processing Magazine, 38(6), 106-114.
  9. Lin, Y. C., Yang, Y. H., & Chen, H. H. (2011). Exploiting online music tags for music emotion classification. ACM Transactions on Multimedia Computing, Communications, and Applications (TOMM), 7(1), 1-16.
  10. Xia, Y., & Xu, F. (2022). Study on music emotion recognition based on the machine learning model clustering algorithm. Mathematical Problems in Engineering, 2022.
  11. Zhang, K., & Sun, S. (2013). Web music emotion recognition based on higher effective gene expression programming. Neurocomputing, 105, 100-106.
  12. Agarwal, G., & Om, H. (2021). An efficient supervised framework for music mood recognition using autoencoder‐based optimised support vector regression model. IET Signal Processing, 15(2), 98-121.
  13. Han, B. J., Rho, S., Dannenberg, R. B., & Hwang, E. (2009, October). SMERS: Music Emotion Recognition Using Support Vector Regression. In ISMIR (pp. 651-656).
  14. Agarwal, G., & Om, H. (2021). An efficient supervised framework for music mood recognition using autoencoder‐based optimised support vector regression model. IET Signal Processing, 15(2), 98-121.
  15. Torres, D. A., Turnbull, D., Barrington, L., & Lanckriet, G. R. (2007, September). Identifying Words that are Musically Meaningful. In ISMIR (Vol. 7, pp. 405-410).
  16. Panwar, S., Rad, P., Choo, K. K. R., & Roopaei, M. (2019). Are you emotional or depressed? Learning about your emotional state from your music using machine learning. The Journal of Supercomputing, 75, 2986-3009.
  17. ER, M. B., & ESİN, E. M. (2021). Music emotion recognition with machine learning based on audio features. Computer Science, 6(3), 133-144.
  18. Song, Y., Dixon, S., & Pearce, M. (2012, June). A survey of music recommendation systems and future perspectives. In 9th international symposium on computer music modeling and retrieval (Vol. 4, pp. 395-410).
  19. Panda, R., Rocha, B., & Paiva, R. P. (2015). Music emotion recognition with standard and melodic audio features. Applied Artificial Intelligence, 29(4), 313-334.
  20. Yang, X., Dong, Y., & Li, J. (2018). Review of data features-based music emotion recognition methods. Multimedia systems, 24, 365-389.
  21. Soleymani, M., Aljanaki, A., & Yang, Y. H. (2016). DEAM: Mediaeval database for emotional analysis in music. Geneva, Switzerland.
  22. https://cvml.unige.ch/databases/DEAM/.
  23. Takashima, N., Li, F., Grzegorzek, M., & Shirahama, K. (2023). Embedding-based music emotion recognition using composite loss. IEEE Access.
  24. Morris, J. D. (1995). Observations: SAM: the Self-Assessment Manikin; an efficient cross-cultural measurement of emotional response. Journal of advertising research, 35(6), 63-68.
  25. Stevens, F., Murphy, D. T., & Smith, S. L. (2017, September). Soundscape categorisation and the self-assessment manikin. In Proceedings of the 20th International Conference on Digital Audio Effects.
  26. Ellis, D. P., & Poliner, G. E. (2007, April). Identifyingcover songs' with chroma features and dynamic programming beat tracking. In 2007 IEEE International Conference on Acoustics, Speech and Signal Processing-ICASSP'07 (Vol. 4, pp. IV-1429). IEEE.
  27. http://labrosa.ee.columbia.edu/projects/coversongs
  28. Dubnov, S. (2004). Generalization of spectral flatness measure for non-gaussian linear processes. IEEE Signal Processing Letters, 11(8), 698-701.
  29. https://www.mathworks.com/help/audio/ug/spectral-descriptors.html.
  30. Birajdar, G. K., & Patil, M. D. (2020). Speech/music classification using visual and spectral chromagram features. Journal of Ambient Intelligence and Humanized Computing, 11(1), 329-347.
  31. Weineck, K., Wen, O. X., & Henry, M. J. (2022). Neural synchronization is strongest to the spectral flux of slow music and depends on familiarity and beat salience. Elife, 11, e75515.
  32. Khare, S. K., Blanes-Vidal, V., Nadimi, E. S., & Acharya, U. R. (2023). Emotion recognition and artificial intelligence: A systematic review (2014–2023) and research recommendations. Information Fusion, 102019.
  33. Wilcoxon, F. (1992). Individual comparisons by ranking methods. In Breakthroughs in statistics: Methodology and distribution (pp. 196-202). New York, NY: Springer New York.
Volume 17, Issue 3
Autumn 2023
Pages 249-262

  • Receive Date 20 March 2024
  • Revise Date 24 May 2024
  • Accept Date 01 July 2024