Fusion of stereo and monocular depth estimates in a self-supervised learning context

Fusion of stereo and monocular depth estimates in a self-supervised learning context

Tomás Cardoso Rézio Martins, Diogo (TU Delft Aerospace Engineering)

de Croon, G.C.H.E. (mentor)

Delft University of Technology

Aerospace Engineering | Control & Simulation

2017-10-11

We study how autonomous robots can better evaluate distances by fusing depth estimates from both stereo vision and a convolutional neural network (CNN) that processes a single still image. The main contribution is a novel fusion method that preserves high confidence stereo estimates, while leveraging the CNN estimates in the low-confidence regions. The main concern with such a fusion scheme is that the CNN may work on the training set, but will degrade significantly in the operational environment. Therefore, we also show that the performance of the monocular estimator in the operational environment improves if stereo vision provides supervised targets in a self-supervised learning (SSL) fashion. The merging framework is implemented on-board of a Parrot SLAMDunk
and tested in real world scenarios, providing more reliable depth maps for use in autonomous navigation.

http://resolver.tudelft.nl/uuid:faf5d4fb-5785-4d27-9d52-0b09214f3a6a

Student theses

master thesis

© 2017 Diogo Tomás Cardoso Rézio Martins