Print Email Facebook Twitter LightDigit: Embedded Deep Learning Empowered Fingertip Air-Writing with Ambient Light Title LightDigit: Embedded Deep Learning Empowered Fingertip Air-Writing with Ambient Light Author Liu, Hao (TU Delft Electrical Engineering, Mathematics and Computer Science) Contributor Zuniga, Marco (mentor) Wang, Q. (mentor) Yang, J. (mentor) Degree granting institution Delft University of Technology Programme Electrical Engineering | Embedded Systems Date 2021-08-21 Abstract This thesis introduces LightDigit, a recognition system for fingertip air-writing digits that enables low-cost, privacy-preserving, and in air interaction with pub- lic devices such as touchscreens. The current LightDigit prototype can input digits to public devices with a simple array of photodiodes and with ubiquitous ambient light in our surroundings. The key enabler is an approach detecting and interpreting the dynamic shadow introduced by the movement of the fingertip when a person writes digits in air using a finger. We design two embedded deep learning models, a visible light Transformer (ViLiT) and a customized model ConvRNN with attention pooling, to extract and analyze the spatial and temporal patterns in the dynamic shadow. For training these two models, we build a dataset with 20880 instances of the air-writing for 174 different types of ten digits 0–9. We evaluate LightDigit through extensive experiments in three different scenarios and in two different environments. The evaluation results show that both models are effective, with ConvRNN achieving 98% accuracy in the Within-Subjects scenario. Through model compression, the size of ConvRNN is reduced by 99% with less than a 5% drop in performance. Results further show that LightDigit is robust when ambient light is tilted by 60◦, or when only 33% of photodiodes are working. To reference this document use: http://resolver.tudelft.nl/uuid:afc5471f-c2a3-4ecf-b15e-6edf43a9d22b Embargo date 2022-09-01 Part of collection Student theses Document type master thesis Rights © 2021 Hao Liu Files PDF TUD_ENS_MSc_Thesis_HaoLiu ... 807456.pdf 3.5 MB Close viewer /islandora/object/uuid:afc5471f-c2a3-4ecf-b15e-6edf43a9d22b/datastream/OBJ/view