Audio samples from "Effective Use of Variational Embedding Capacity in Expressive End-to-End Speech Synthesis"

This page contains a set of audio samples in support of the paper: it is suggested that the reader listen to the samples in conjunction with reading the paper. All reference and target utterances were unseen during training.

Click here for more from the Tacotron team.

Contents

Here we show that models with and without text dependencies in the variational posterior (Var+Txt and Var, respectively) exhibit increasing similarity to the reference as embedding capacity, C, is increased. The baseline model does not use a reference embedding.

In this section, we compare models with and without text dependencies in the variational posterior (Var+Txt and Var, respectively) when transferring between utterances with different text. The examples show how the quality of the samples from Var deteriorate as the capacity, C, is increased. This is especially noticeable when there is a significant mismatch in the length of the two utterances.

Here we sample the latent embedding from the prior distribution for models with different capacities. The quality of the samples from the Var model deteriorates as capacity, C, is increased, while Var+Txt remains stable. This is especially noticeable for very short and very long utterances.

In this experiment, we demonstrate prosody transfer between speakers. When conditioned on the reference speaker (Var+Txt+Spk), the output preserves the pitch range of the target speaker, whereas without speaker conditioning (Var+Txt), the output tends to match the pitch of the reference speaker. As the capacity, C, is increased, the difference becomes more apparent.

Here we sythesize from multi-speaker models by sampling the latent embeddings from the prior. The model that includes speaker dependencies in the posterior (Var+Txt+Spk) does a better job of preserving target speaker identity and pitch range, compared to the model without posterior speaker dependencies (Var+Txt). Notice also how the samples become more expressive (and erratic) as embedding capacity is increased.

In this experiment, we use models with hierarchical latents (high-level latents, Z_H, and low-level latents, Z_L) and demonstrate the effect of varying C_H and C_L when transferring via Z_H. When increasing C_H, similarity to the reference increases. When increasing C_L, inter-sample variation increases (between samples generated using the same Z_H). We include groups of 3 samples generated using the same Z_H inferred from the reference.

In this experiment, we use models with hierarchical latents (Z_H and Z_L) and demonstrate the effect of varying C_H and C_L when transferring via Z_L (instead of Z_H). We can see that as the sum of C_H and C_L increases, similarity to the reference increases. The variation across samples generated using the same reference is low. We include groups of 3 samples generated using different Z_L's that are inferred from the same reference.