University of Hertfordshire

From the same journal

From the same journal

By the same authors

CNN and LSTM-Based Emotion Charting Using Physiological Signals

Research output: Contribution to journalArticlepeer-review

Standard

CNN and LSTM-Based Emotion Charting Using Physiological Signals. / Dar, Muhammad Najam; Akram, Muhammad Usman; Khawaja, Sajid Gul; Pujari, Amit N.

In: Sensors (Switzerland), Vol. 20, No. 16, 4551, 14.08.2020, p. 1-26.

Research output: Contribution to journalArticlepeer-review

Harvard

APA

Vancouver

Author

Dar, Muhammad Najam ; Akram, Muhammad Usman ; Khawaja, Sajid Gul ; Pujari, Amit N. / CNN and LSTM-Based Emotion Charting Using Physiological Signals. In: Sensors (Switzerland). 2020 ; Vol. 20, No. 16. pp. 1-26.

Bibtex

@article{db9d093cb0a24938910393bb4d799ae0,
title = "CNN and LSTM-Based Emotion Charting Using Physiological Signals",
abstract = "Novel trends in affective computing are based on reliable sources of physiological signals such as Electroencephalogram (EEG), Electrocardiogram (ECG), and Galvanic Skin Response (GSR). The use of these signals provides challenges of performance improvement within a broader set of emotion classes in a less constrained real-world environment. To overcome these challenges, we propose a computational framework of 2D Convolutional Neural Network (CNN) architecture for the arrangement of 14 channels of EEG, and a combination of Long Short-Term Memory (LSTM) and 1D-CNN architecture for ECG and GSR. Our approach is subject-independent and incorporates two publicly available datasets of DREAMER and AMIGOS with low-cost, wearable sensors to extract physiological signals suitable for real-world environments. The results outperform state-of-the-art approaches for classification into four classes, namely High Valence—High Arousal, High Valence—Low Arousal, Low Valence—High Arousal, and Low Valence—Low Arousal. Emotion elicitation average accuracy of 98.73% is achieved with ECG right-channel modality, 76.65% with EEG modality, and 63.67% with GSR modality for AMIGOS. The overall highest accuracy of 99.0% for the AMIGOS dataset and 90.8% for the DREAMER dataset is achieved with multi-modal fusion. A strong correlation between spectral-and hidden-layer feature analysis with classification performance suggests the efficacy of the proposed method for significant feature extraction and higher emotion elicitation performance to a broader context for less constrained environments.",
keywords = "Convolutional neural network (CNN), Deep neural network, ECG, EEG, Emotion recognition, GSR, Long short-term memory (LSTM), Physiological signals",
author = "Dar, {Muhammad Najam} and Akram, {Muhammad Usman} and Khawaja, {Sajid Gul} and Pujari, {Amit N.}",
year = "2020",
month = aug,
day = "14",
doi = "10.3390/s20164551",
language = "English",
volume = "20",
pages = "1--26",
journal = "Sensors",
issn = "1424-8220",
publisher = "Multidisciplinary Digital Publishing Institute",
number = "16",

}

RIS

TY - JOUR

T1 - CNN and LSTM-Based Emotion Charting Using Physiological Signals

AU - Dar, Muhammad Najam

AU - Akram, Muhammad Usman

AU - Khawaja, Sajid Gul

AU - Pujari, Amit N.

PY - 2020/8/14

Y1 - 2020/8/14

N2 - Novel trends in affective computing are based on reliable sources of physiological signals such as Electroencephalogram (EEG), Electrocardiogram (ECG), and Galvanic Skin Response (GSR). The use of these signals provides challenges of performance improvement within a broader set of emotion classes in a less constrained real-world environment. To overcome these challenges, we propose a computational framework of 2D Convolutional Neural Network (CNN) architecture for the arrangement of 14 channels of EEG, and a combination of Long Short-Term Memory (LSTM) and 1D-CNN architecture for ECG and GSR. Our approach is subject-independent and incorporates two publicly available datasets of DREAMER and AMIGOS with low-cost, wearable sensors to extract physiological signals suitable for real-world environments. The results outperform state-of-the-art approaches for classification into four classes, namely High Valence—High Arousal, High Valence—Low Arousal, Low Valence—High Arousal, and Low Valence—Low Arousal. Emotion elicitation average accuracy of 98.73% is achieved with ECG right-channel modality, 76.65% with EEG modality, and 63.67% with GSR modality for AMIGOS. The overall highest accuracy of 99.0% for the AMIGOS dataset and 90.8% for the DREAMER dataset is achieved with multi-modal fusion. A strong correlation between spectral-and hidden-layer feature analysis with classification performance suggests the efficacy of the proposed method for significant feature extraction and higher emotion elicitation performance to a broader context for less constrained environments.

AB - Novel trends in affective computing are based on reliable sources of physiological signals such as Electroencephalogram (EEG), Electrocardiogram (ECG), and Galvanic Skin Response (GSR). The use of these signals provides challenges of performance improvement within a broader set of emotion classes in a less constrained real-world environment. To overcome these challenges, we propose a computational framework of 2D Convolutional Neural Network (CNN) architecture for the arrangement of 14 channels of EEG, and a combination of Long Short-Term Memory (LSTM) and 1D-CNN architecture for ECG and GSR. Our approach is subject-independent and incorporates two publicly available datasets of DREAMER and AMIGOS with low-cost, wearable sensors to extract physiological signals suitable for real-world environments. The results outperform state-of-the-art approaches for classification into four classes, namely High Valence—High Arousal, High Valence—Low Arousal, Low Valence—High Arousal, and Low Valence—Low Arousal. Emotion elicitation average accuracy of 98.73% is achieved with ECG right-channel modality, 76.65% with EEG modality, and 63.67% with GSR modality for AMIGOS. The overall highest accuracy of 99.0% for the AMIGOS dataset and 90.8% for the DREAMER dataset is achieved with multi-modal fusion. A strong correlation between spectral-and hidden-layer feature analysis with classification performance suggests the efficacy of the proposed method for significant feature extraction and higher emotion elicitation performance to a broader context for less constrained environments.

KW - Convolutional neural network (CNN)

KW - Deep neural network

KW - ECG

KW - EEG

KW - Emotion recognition

KW - GSR

KW - Long short-term memory (LSTM)

KW - Physiological signals

UR - http://www.scopus.com/inward/record.url?scp=85089387052&partnerID=8YFLogxK

U2 - 10.3390/s20164551

DO - 10.3390/s20164551

M3 - Article

AN - SCOPUS:85089387052

VL - 20

SP - 1

EP - 26

JO - Sensors

JF - Sensors

SN - 1424-8220

IS - 16

M1 - 4551

ER -