University of Surrey

Test tubes in the lab Research in the ATI Dance Research

Reverberant Speech Separation Based on Audio-visual Dictionary Learning and Binaural Cues

Liu, Q, Wang, W, Jackson, PJB and Barnard, M (2012) Reverberant Speech Separation Based on Audio-visual Dictionary Learning and Binaural Cues In: IEEE Statistical Signal Processing Workshop (SSP), 2012-08-05 - 2012-08-08, Ann Abor, USA.

[img]
Preview
PDF
LiuWangJacksonBarnard_SSP12_preprint.pdf
Available under License : See the attached licence file.

Download (320kB)
[img]
Preview
PDF (licence)
SRI_deposit_agreement.pdf

Download (33kB)

Abstract

Probabilistic models of binaural cues, such as the interaural phase difference (IPD) and the interaural level difference (ILD), can be used to obtain the audio mask in the time-frequency (TF) domain, for source separation of binaural mixtures. Those models are, however, often degraded by acoustic noise. In contrast, the video stream contains relevant information about the synchronous audio stream that is not affected by acoustic noise. In this paper, we present a novel method for modeling the audio-visual (AV) coherence based on dictionary learning. A visual mask is constructed from the video signal based on the learnt AV dictionary, and incorporated with the audio mask to obtain a noise-robust audio-visual mask, which is then applied to the binaural signal for source separation. We tested our algorithm on the XM2VTS database, and observed considerable performance improvement for noise corrupted signals.

Item Type: Conference or Workshop Item (Conference Paper)
Divisions : Faculty of Engineering and Physical Sciences > Electronic Engineering > Centre for Vision Speech and Signal Processing
Authors :
AuthorsEmailORCID
Liu, QUNSPECIFIEDUNSPECIFIED
Wang, WUNSPECIFIEDUNSPECIFIED
Jackson, PJBUNSPECIFIEDUNSPECIFIED
Barnard, MUNSPECIFIEDUNSPECIFIED
Date : 5 August 2012
Identification Number : 10.1109/SSP.2012.6319789
Contributors :
ContributionNameEmailORCID
PublisherIEEE, UNSPECIFIEDUNSPECIFIED
Additional Information : © 2012IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.
Depositing User : Symplectic Elements
Date Deposited : 17 May 2013 17:15
Last Modified : 23 Sep 2013 20:07
URI: http://epubs.surrey.ac.uk/id/eprint/771638

Actions (login required)

View Item View Item

Downloads

Downloads per month over past year


Information about this web site

© The University of Surrey, Guildford, Surrey, GU2 7XH, United Kingdom.
+44 (0)1483 300800