Coupled Dictionaries for Exemplar-Based Speech Enhancement and Automatic Speech Recognition

Abstract

Exemplar-based speech enhancement systems work by decomposing the noisy speech as a weighted sum of speech and noise exemplars stored in a dictionary and use the resulting speech and noise estimates to obtain a time-varying filter in the full-resolution frequency domain to enhance the noisy speech. To obtain the decomposition, exemplars sampled in lower dimensional spaces are preferred over the full-resolution frequency domain for their reduced computational complexity and the ability to better generalize to unseen cases. But the resulting filter may be sub-optimal as the mapping of the obtained speech and noise estimates to the full-resolution frequency domain yields a low-rank approximation. This paper proposes an efficient way to directly compute the full-resolution frequency estimates of speech and noise using coupled dictionaries: an input dictionary containing atoms from the desired exemplar space to obtain the decomposition and a coupled output dictionary containing exemplars from the full-resolution frequency domain. We also introduce modulation spectrogram features for the exemplar-based tasks using this approach. The proposed system was evaluated for various choices of input exemplars and yielded improved speech enhancement performances on the AURORA-2 and AURORA-4 databases. We further show that the proposed approach also results in improved word error rates (WERs) for the speech recognition tasks using HMM-GMM and deep-neural network (DNN) based systems.

Publication
IEEE/ACM Transactions on Audio, Speech and Language Processing
Deepak Baby
Deepak Baby
Applied Scientist

My research interests include speech recognition, enhancement and deep learning.

Tuomas Virtanen
Tuomas Virtanen
Professor

Professor at Tampere University of Technolog

Tech Lead, Autonomous Systems at Apple Inc., Cupertino, USA

Tech Lead, Autonomous Systems at Apple Inc., Cupertino, USA

Hugo Van hamme
Hugo Van hamme
Professor

Professor at KU Leuven, Belgium

comments powered by Disqus

Related