University of Surrey

Test tubes in the lab Research in the ATI Dance Research

3D Room Geometry Reconstruction Using Audio-Visual Sensors

Kim, Hansung, Remaggi, Luca, Jackson, Philip, Fazi, F and Hilton, Adrian (2017) 3D Room Geometry Reconstruction Using Audio-Visual Sensors In: International Conference on 3D Vision (3DV 2017), 10 - 12 October 2017, Qingdao, China.

[img]
Preview
Text
3DV2017_CameraReady.pdf - Accepted version Manuscript

Download (6MB) | Preview

Abstract

In this paper we propose a cuboid-based air-tight indoor room geometry estimation method using combination of audio-visual sensors. Existing vision-based 3D reconstruction methods are not applicable for scenes with transparent or reflective objects such as windows and mirrors. In this work we fuse multi-modal sensory information to overcome the limitations of purely visual reconstruction for reconstruction of complex scenes including transparent and mirror surfaces. A full scene is captured by 360� cameras and acoustic room impulse responses (RIRs) recorded by a loudspeaker and compact microphone array. Depth information of the scene is recovered by stereo matching from the captured images and estimation of major acoustic reflector locations from the sound. The coordinate systems for audiovisual sensors are aligned into a unified reference frame and plane elements are reconstructed from audio-visual data. Finally cuboid proxies are fitted to the planes to generate a complete room model. Experimental results show that the proposed system generates complete representations of the room structures regardless of transparent windows, featureless walls and shiny surfaces.

Item Type: Conference or Workshop Item (Conference Paper)
Divisions : Faculty of Engineering and Physical Sciences > Electronic Engineering
Authors :
NameEmailORCID
Kim, HansungH.Kim@surrey.ac.ukUNSPECIFIED
Remaggi, Lucalr0021@surrey.ac.ukUNSPECIFIED
Jackson, PhilipP.Jackson@surrey.ac.ukUNSPECIFIED
Fazi, FUNSPECIFIEDUNSPECIFIED
Hilton, AdrianA.Hilton@surrey.ac.ukUNSPECIFIED
Date : 12 September 2017
Copyright Disclaimer : Copyright 2017.Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.
Depositing User : Melanie Hughes
Date Deposited : 08 Sep 2017 14:17
Last Modified : 15 Sep 2017 07:39
URI: http://epubs.surrey.ac.uk/id/eprint/842224

Actions (login required)

View Item View Item

Downloads

Downloads per month over past year


Information about this web site

© The University of Surrey, Guildford, Surrey, GU2 7XH, United Kingdom.
+44 (0)1483 300800