A Comparison Between Convolutional and Transformer Architectures for Speech Emotion Recognition

Shreyah Iyer, Cornelius Glackin, Nigel Cannings, Vito Veneziano, Yi Sun

Research output: Chapter in Book/Report/Conference proceedingConference contribution

19 Downloads (Pure)


Creating speech emotion recognition models com-parable to the capability of how humans recognise emotions is a long-standing challenge in the field of speech technology with many potential commercial applications. As transformer-based architectures have recently become the state-of-the-art for many natural language processing related applications, this paper investigates their suitability for acoustic emotion recognition and compares them to the well-known AlexNet convolutional approach. This comparison is made using several publicly available speech emotion corpora. Experimental results demonstrate the efficacy of the different architectural approaches for particular emotions. The results show that the transformer-based models outperform their convolutional counterparts yielding F1-scores in the range [70.33%, 75.76%]. This paper further provides insights via dimensionality reduction analysis of output layer activations in both architectures and reveals significantly improved clustering in transformer-based models whilst highlighting the nuances with regard to the separability of different emotion classes.
Original languageEnglish
Title of host publication2022 International Joint Conference on Neural Networks (IJCNN)
Place of PublicationPadua, Italy
PublisherInstitute of Electrical and Electronics Engineers (IEEE)
Number of pages8
ISBN (Electronic)978-1-7281-8671-9
ISBN (Print)978-1-6654-9526-4
Publication statusPublished - 30 Sept 2022
Event2022 International Joint Conference on Neural Networks (IJCNN) - Padua, Italy
Duration: 18 Jul 202223 Jul 2022


Conference2022 International Joint Conference on Neural Networks (IJCNN)
Internet address


  • alexnet
  • convolutional neural networks
  • mel spectrograms
  • speech emotion recognition
  • transfer learning
  • transformers
  • wav2vec2


Dive into the research topics of 'A Comparison Between Convolutional and Transformer Architectures for Speech Emotion Recognition'. Together they form a unique fingerprint.

Cite this