University of Surrey

Test tubes in the lab Research in the ATI Dance Research

Immersive Spatial Audio Reproduction for VR/AR Using Room Acoustic Modelling from 360° Images

Kim, Hansung, Remaggi, Luca, Jackson, Philip and Hilton, Adrian (2019) Immersive Spatial Audio Reproduction for VR/AR Using Room Acoustic Modelling from 360° Images In: IEEE VR 2019, 2019-03-23 - 2019-03-27, Osaka, Japan.

[img]
Preview
Text
IEEE_VR2019-Camera-Ready.pdf - Accepted version Manuscript

Download (1MB) | Preview
Official URL: http://ieeevr.org/2019/

Abstract

Recent progresses in Virtual Reality (VR) and Augmented Reality (AR) allow us to experience various VR/AR applications in our daily life. In order to maximise the immersiveness of user in VR/AR environments, a plausible spatial audio reproduction synchronised with visual information is essential. In this paper, we propose a simple and efficient system to estimate room acoustic for plausible reproducton of spatial audio using 360° cameras for VR/AR applications. A pair of 360° images is used for room geometry and acoustic property estimation. A simplified 3D geometric model of the scene is estimated by depth estimation from captured images and semantic labelling using a convolutional neural network (CNN). The real environment acoustics are characterised by frequency-dependent acoustic predictions of the scene. Spatially synchronised audio is reproduced based on the estimated geometric and acoustic properties in the scene. The reconstructed scenes are rendered with synthesised spatial audio as VR/AR content. The results of estimated room geometry and simulated spatial audio are evaluated against the actual measurements and audio calculated from ground-truth Room Impulse Responses (RIRs) recorded in the rooms.

Item Type: Conference or Workshop Item (Conference Paper)
Divisions : Faculty of Engineering and Physical Sciences > Electronic Engineering > Centre for Vision Speech and Signal Processing
Authors :
NameEmailORCID
Kim, HansungH.Kim@surrey.ac.uk
Remaggi, Lucal.remaggi@surrey.ac.uk
Jackson, PhilipP.Jackson@surrey.ac.uk
Hilton, AdrianA.Hilton@surrey.ac.uk
Date : 15 February 2019
Copyright Disclaimer : © 2019 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.
Uncontrolled Keywords : Human-centered computing; Human computer interaction (HCI); Interaction paradigms; Virtual reality; Computing methodologies; Artificial intelligence; Computer vision; Scene understanding
Depositing User : Diane Maxfield
Date Deposited : 28 Feb 2019 16:06
Last Modified : 28 Feb 2019 16:06
URI: http://epubs.surrey.ac.uk/id/eprint/850589

Actions (login required)

View Item View Item

Downloads

Downloads per month over past year


Information about this web site

© The University of Surrey, Guildford, Surrey, GU2 7XH, United Kingdom.
+44 (0)1483 300800