Print Email Facebook Twitter Corrnet Title Corrnet: Fine-grained emotion recognition for video watching using wearable physiological sensors Author Zhang, T. (TU Delft Multimedia Computing; Centrum Wiskunde & Informatica (CWI)) Ali, Abdallah El (Centrum Wiskunde & Informatica (CWI)) Chen, C. (Xinhua News Agency, Beijing) Hanjalic, A. (TU Delft Intelligent Systems) Cesar, Pablo (TU Delft Multimedia Computing; Centrum Wiskunde & Informatica (CWI)) Department Intelligent Systems Date 2020 Abstract Recognizing user emotions while they watch short-form videos anytime and anywhere is essential for facilitating video content customization and personalization. However, most works either classify a single emotion per video stimuli, or are restricted to static, desktop environments. To address this, we propose a correlation-based emotion recognition algorithm (CorrNet) to recognize the valence and arousal (V-A) of each instance (fine-grained segment of signals) using only wearable, physiological signals (e.g., electrodermal activity, heart rate). CorrNet takes advantage of features both inside each instance (intra-modality features) and between different instances for the same video stimuli (correlation-based features). We first test our approach on an indoor-desktop affect dataset (CASE), and thereafter on an outdoor-mobile affect dataset (MERCA) which we collected using a smart wristband and wearable eyetracker. Results show that for subject-independent binary classification (high-low), CorrNet yields promising recognition accuracies: 76.37% and 74.03% for V-A on CASE, and 70.29% and 68.15% for V-A on MERCA. Our findings show: (1) instance segment lengths between 1–4 s result in highest recognition accuracies (2) accuracies between laboratory-grade and wearable sensors are comparable, even under low sampling rates (≤64 Hz) (3) large amounts of neu-tral V-A labels, an artifact of continuous affect annotation, result in varied recognition performance. Subject Emotion recognitionMachine learningPhysiological signalsVideo To reference this document use: http://resolver.tudelft.nl/uuid:34854ebd-489c-480c-aa74-8712f6ef6ec5 DOI https://doi.org/10.3390/s21010052 ISSN 1424-8220 Source Sensors, 21 (1), 1-25 Part of collection Institutional Repository Document type journal article Rights © 2020 T. Zhang, Abdallah El Ali, C. Chen, A. Hanjalic, Pablo Cesar Files PDF sensors_21_00052_v2.pdf 11.36 MB Close viewer /islandora/object/uuid:34854ebd-489c-480c-aa74-8712f6ef6ec5/datastream/OBJ/view