Speaker Diarization with LSTM


[Link] to arXiv paper


Quan Wang, Carlton Downey, Li Wan, Philip Andrew Mansfield, Ignacio Lopez Moreno


For many years, i-vector based audio embedding techniques were the dominant approach for speaker verification and speaker diarization applications. However, mirroring the rise of deep learning in various domains, neural network based audio embeddings, also known as d-vectors, have consistently demonstrated superior speaker verification performance. In this paper, we build on the success of d-vector based speaker verification systems to develop a new d-vector based approach to speaker diarization. Specifically, we combine LSTM-based d-vector audio embeddings with recent work in non-parametric clustering to obtain a state-of-the-art speaker diarization system. Our system is evaluated on three standard public datasets, suggesting that d-vector based diarization systems offer significant advantages over traditional i-vector based systems. We achieved a 12.0% diarization error rate on NIST SRE 2000 CALLHOME, while our model is trained with out-of-domain data from voice search logs.


A Python re-implementation of the spectral clustering algorithm described in the paper is available on GitHub: https://github.com/wq2012/SpectralCluster

More resources on speaker diarization: awesome-diarization

Supplementary Material

[Link] to arXiv paper for the Links online clustering algorithm


[PDF] version of the poster presented at ICASSP 2018

Evaluation Protocol

If you want to evaluate your own speaker diarization system and compare with ours, we provide these useful files for you:

  • Dev-vs-Eval division for NIST RT03 English CTS
  • Dev-vs-Eval division for CALLHOME American English
  • Ground truth information for NIST SRE 2000 Disk8 (CALLHOME)
  • Update

    We have new papers including updated results:

  • Fully Supervised Speaker Diarization [code]
  • Turn-to-Diarize: Online Speaker Diarization Constrained by Transformer Transducer Speaker Turn Detection
  • Lecture