University of Surrey

Test tubes in the lab Research in the ATI Dance Research

Interference reduction in reverberant speech separation with visual voice activity detection

Liu, Q, Aubrey, AJ and Wang, W (2014) Interference reduction in reverberant speech separation with visual voice activity detection IEEE Transactions on Multimedia, 16 (6). pp. 1610-1623.

Full text not available from this repository.


© 2014 IEEE.The visual modality, deemed to be complementary to the audio modality, has recently been exploited to improve the performance of blind source separation (BSS) of speech mixtures, especially in adverse environments where the performance of audio-domain methods deteriorates steadily. In this paper, we present an enhancement method to audio-domain BSS with the integration of voice activity information, obtained via a visual voice activity detection (VAD) algorithm. Mimicking aspects of human hearing, binaural speech mixtures are considered in our two-stage system. Firstly, in the off-line training stage, a speaker-independent voice activity detector is formed using the visual stimuli via the adaboosting algorithm. In the on-line separation stage, interaural phase difference (IPD) and interaural level difference (ILD) cues are statistically analyzed to assign probabilistically each time-frequency (TF) point of the audio mixtures to the source signals. Next, the detected voice activity cues (found via the visual VAD) are integrated to reduce the interference residual. Detection of the interference residual takes place gradually, with two layers of boundaries in the correlation and energy ratio map. We have tested our algorithm on speech mixtures generated using room impulse responses at different reverberation times and noise levels. Simulation results show performance improvement of the proposed method for target speech extraction in noisy and reverberant environments, in terms of signal-to-interference ratio (SIR) and perceptual evaluation of speech quality (PESQ).

Item Type: Article
Divisions : Surrey research (other units)
Authors :
Aubrey, AJ
Date : 1 October 2014
DOI : 10.1109/TMM.2014.2322824
Depositing User : Symplectic Elements
Date Deposited : 17 May 2017 13:29
Last Modified : 25 Jan 2020 00:01

Actions (login required)

View Item View Item


Downloads per month over past year

Information about this web site

© The University of Surrey, Guildford, Surrey, GU2 7XH, United Kingdom.
+44 (0)1483 300800