Cognitive activity recognition by analyzing eye movement with convolutional neural networks

Cognitive activity recognition by analyzing eye movement with convolutional neural networks

Brockbernd, Bob (TU Delft Electrical Engineering, Mathematics and Computer Science)

Lan, G. (mentor)
Du, L. (mentor)
Spaan, M.T.J. (graduation committee)

Delft University of Technology

Computer Science and Engineering

CSE3000 Research Project

2022-06-23

This research proposes a novel method to classify cognitive behavior based on eye-movement data. Most state-of-the-art approaches use conventional machine learning techniques needing manual feature extraction. This experiment explores the possibility of applying deep learning algorithms to cognitive activity recognition for feature extraction and classification of eye-movement data. Convolutional neural networks will be explored in particular. Two neural networks are proposed and optimized using hyperparameter tuning. This research shows that convolutional neural networks can indeed perform cognitive activity recognition. Some neural networks significantly outperform the state-of-the-art methods for known subjects. However, further research is needed to improve performance in classifying activities for unknown subjects.

Human Activity Recognition
Gaze bazed activity recognition
CNN

http://resolver.tudelft.nl/uuid:5ff6be37-5ba5-4d7c-8070-417a660aac14

Student theses

bachelor thesis

© 2022 Bob Brockbernd