Recognising Gestures Using Ambient Light and Convolutional Neural Networks

Recognising Gestures Using Ambient Light and Convolutional Neural Networks: Adapting Convolutional Neural Networks for Gesture Recognition on Resource-constrained Microcontrollers

Narchi, William (TU Delft Electrical Engineering, Mathematics and Computer Science)

Wang, Q. (mentor)
Yang, M. (mentor)
Zhu, R. (mentor)
Lofi, C. (graduation committee)

Delft University of Technology

Computer Science and Engineering

CSE3000 Research Project

2022-07-22

This paper presents how a convolutional neural network can be constructed in order to recognise gestures using photodiodes and ambient light. A number of candidates are presented and evaluated, with the most performant being adopted for in-depth analysis. This network is then compressed in order to be ran on an Arduino Nano 33 BLE microcontroller to present its feasibility in embedded operation. The final utilised network was observed to have accuracies between 75.4% and 86.8% depending on the testing conditions. Further, all candidates were found to be sufficiently compact and low-latency for real-time operation.

neural network
convolutional neural network
embedded systems
deep learning
Arduino
Arduino Nano 33 BLE
gesture recognition
ambient light
Tensorflow
machine learning
microcontroller

http://resolver.tudelft.nl/uuid:c4a0207e-8ebe-4f5a-ab4c-5675b6d92551

Student theses

bachelor thesis

© 2022 William Narchi