Print Email Facebook Twitter Optimizing Event-Based Vision by Realizing Super-Resolution in Event-Space: an Experimental Approach Title Optimizing Event-Based Vision by Realizing Super-Resolution in Event-Space: an Experimental Approach Author Şabanoğlu, Mahir (TU Delft Mechanical, Maritime and Materials Engineering) Contributor Tömen, N. (mentor) de Winter, J.C.F. (mentor) Souman, Martijn (mentor) van Gemert, J.C. (graduation committee) Kooij, J.F.P. (graduation committee) Araghi, M.R. (graduation committee) Degree granting institution Delft University of Technology Programme Mechanical Engineering | Vehicle Engineering | Cognitive Robotics Date 2023-01-25 Abstract An event-based camera enables capturing a video at a high temporal resolution, high dynamical range, reduced power consumption and minimal data bandwidth while the camera has minimal physical dimensions compared to a frame-based camera with the same vision properties. The limiting factor, however, of an event-based camera is the spatial resolution which ranges between 40 × 40 and 1280 × 960. To counter this deficiency, a method is researched to super resolve event-based vision in order to enhance spatial resolution. A selection of different neural network types and configurations are researched in a step-by-step fashion. Subsequent experiments tested the selected networks on their ability to process event-based data and extract features from it. Followed by experiments that exploited the limitations of the networks to super resolve at different ratios, lengths of eventstreams and more complex event-based data. Results of various experiments showed that a network configuration that utilizes a transformer architecture was best able to super resolve event-based vision. This type of network leverages the ability to extract features based on dependencies between events which aligns with the characteristics of event- based vision. Based on the obtained results from the exper- iments, a pipeline is proposed to super resolve event-based vision and consists of a combination of a transformer network, multilayer perceptrons and a k-nearest-neighbor algorithm. Using this pipeline, eventstreams can be super resolved in the spatial resolution at a scaling ratio of 4. Visually, these super resolved eventstreams resemble more detailed and enhanced version to the low-resolution input. This proposed pipeline can be considered as a starting point in further research toward the super-resolution of event-based data and thereby contributes to the extension of application possibilities of event-based vision. Subject Event-based visionSuper-resolutionDeep-learningComputer visionSpatial resolutionNeuromorphic vision To reference this document use: http://resolver.tudelft.nl/uuid:20607b85-ec70-4e39-b5bc-7c4e35b20745 Part of collection Student theses Document type master thesis Rights © 2023 Mahir Şabanoğlu Files PDF thesis_report_mm_sabanoglu_3.pdf 18.68 MB Close viewer /islandora/object/uuid:20607b85-ec70-4e39-b5bc-7c4e35b20745/datastream/OBJ/view