Room geometry estimation from stereo recordings using neural networks

Room geometry estimation from stereo recordings using neural networks

Bologni, Giovanni (TU Delft Electrical Engineering, Mathematics and Computer Science; TU Delft Circuits and Systems)

Heusdens, R. (mentor)
Giron, Franck (graduation committee)

Delft University of Technology

Electrical Engineering | Circuits and Systems

2020

Acoustic room geometry estimation is often performed in ad hoc settings, i.e., using multiple microphones and sources distributed around the room, or assuming control over the excitation signals. To facilitate practical applications, we propose a fully convolutional network (FCN) that localizes reflective surfaces under the relaxed assumptions that (i) a compact array of only two microphones is available, (ii) emitter and receivers are not synchronized, and (iii), both the excitation signals and the impulse responses of the enclosures are unknown.
Our FCN is designed to extract spectral and temporal patterns from stereo recordings, aggregate the temporal information over time-frames, and predict the likelihood of virtual sources corresponding to reflective surfaces at specific locations. Whereas most source localization algorithms are limited to direction-of-arrival (DOA) estimation, the proposed method jointly estimates distances and DOAs. Numerical experiments confirm that the network is able to generalize to mismatched microphone array sizes, sensor directivity patterns, or audio signal types, while highlighting front-back ambiguity as a prominent source of uncertainty. When a single reflective surface is present, up to 80% of the sources are detected, while this figure approaches 50% in rectangular rooms.
Further tests on real-world recordings report similar accuracy as with artificially reverberated speech signals, validating the generalization capabilities of the framework.

msc
deep
fully
computers
code
thesis
geometry
Room
estimation
recordings
stereo
Neural Network
supervised

http://resolver.tudelft.nl/uuid:fbe463ac-902b-45e8-adf7-7b530cdd0ef8

2022-01-31

Student theses

master thesis

© 2020 Giovanni Bologni