Touchless Hand Gesture-Based Digit Recognition

Touchless Hand Gesture-Based Digit Recognition: using Light-Sensors, Convolutional Neural Networks and a Microcontroller

Smit, Winstijn (TU Delft Electrical Engineering, Mathematics and Computer Science)

Wang, Q. (mentor)
Yang, M. (mentor)
Zhu, R. (mentor)
Venkatesha Prasad, Ranga Rao (graduation committee)

Delft University of Technology

Computer Science and Engineering

CSE3000 Research Project

2023-07-03

Touchless interaction with computers has become more important in recent years, especially in the context of the COVID-19 pandemic.
Applications include situations where touch input is not possible or not desirable, e.g. for hygienic purposes in a public setting or a medical setting.
Practical examples for touchless interaction include elevators, vending machines, and other public devices that are used by many people.
However, most touchless interaction systems are expensive and require significant computational power.
This paper proposes a bare-bones low-power and low-cost system for recognizing air-written digits using a microcontroller and light sensitivity sensors.
A proof of concept has been created and tested in a fixed lighting scenario with a fixed set of gestures for the digits 0 to 9 to show the feasibility of such a system.
The system uses a convolutional neural network to recognize digits and achieves an average accuracy of 58,8% on a validation set of unseen participants.
It performs significantly better on new samples from users already seen during training, achieving an average accuracy of 93,5%.

Convolutional Neural Networks (CNNs)
Air-Writing
Digit Recognition
Arduino Nano 33 BLE
OPT101
Photodiodes
Touchless
Hand-Gestures

http://resolver.tudelft.nl/uuid:9e749a25-5957-419c-9629-20f354ef1b7d

Student theses

bachelor thesis

© 2023 Winstijn Smit