Data-Segmentation Verification and a Target Generative Adversarial Network: EEG-Based Emotion Recognition

Abstract

Emotion recognition is a crucial component of artificial intelligence. As one of the main factors in emotion recognition, data-driven affective computing heavily relies on high-quality training data, which may not always be readily available due to various reasons. Addressing the challenge of data augmentation with inter-class and intra-class imbalances in emotion-evoking data is a critical issue in affective computing. Currently, many researches have addressed the problem of inter-class imbalance, in which data segmentation processing methods are widely used. However, the rationality of data segmentation methods needs to be verified; meanwhile, the solution to the intra-class imbalance problem remains to be solved. In this paper, we validate the rationality of data segmentation methods through experiments and propose a targeted data generation mechanism. This mechanism intentionally generates pseudo samples in proximity to the oftenoverlooked samples, aimed at mitigating intra-class imbalance. Combined with Wasserstein generative adversarial network-gradient penalty and generative adversarial network-based self-supervised, we have got T-WGAN-GP and T-GANSER. We then apply these approaches to the DEAP dataset for emotion recognition with different data segmentation methods, including segmenting the data after (MI) or before (MII) the division of the training and testing sets, as well as no data segmentation at all (MIII). Results show that MII and MIII in the segmentation method are theoretically sound, and T-WGAN-GP obtains the best accuracy in the reasonable segmentation method due to its targeted data generation mechanism. This mechanism effectively mitigates the intra-class imbalance to some extent.

Received: 2 February 2024 | Revised: 16 April 2024 | Accepted: 14 May 2024

Conflicts of Interest

The authors declare that they have no conflicts of interest to this work.

Data Availability Statement

The data that support the findings of this study are openly available at https://doi.org/10.1109/T-AFFC.2011.15, reference number [44]. The DEAP datasets that support the findings of this study are openly available at http://www.eecs.qmul.ac.uk/mmv/da tasets/deap/download.html.

Author Contribution Statement

Lufeng Yin: Software, Formal analysis, Investigation, Resources, Data curation, Writing – original draft, Visualization. Yong Li: Conceptualization, Methodology, Validation, Writing – review & editing, Supervision, Project administration.