Emotional Zero-Shot TTS with Duration Control

Understanding IndexTTS2: Architecture & Usage

What Makes IndexTTS2 Different

Most text-to-speech systems treat speech as a sequence of acoustic features predicted frame by frame. IndexTTS2 takes a different approach: it uses an autoregressive language model to predict discrete speech tokens, then decodes them to waveform through a vocoder. This architecture—similar to how large language models generate text—enables emergent capabilities like emotional expression and natural prosody that frame-level models struggle with.

The key innovation is the separation of duration, emotion, and speaker identity into independent control signals. Previous systems like VITS or Bark entangle these factors, making it hard to change one without affecting the others. IndexTTS2's disentangled design lets you keep a speaker's exact timbre while changing their emotional tone from neutral to excited, or slow down speech for emphasis without altering pitch.

Duration Control: How It Works

IndexTTS2 supports two modes of duration control:

• Token-level specification — assign explicit durations (in milliseconds) to individual phonemes or words. This is ideal for dubbing where you need speech to match an existing video timeline.
• Ratio scaling — apply a global multiplier (0.75× to 1.25×) to speed up or slow down the entire utterance proportionally. Useful for podcast tempo adjustment.

Practical Applications

Film and video dubbing is the primary use case—aligning generated speech to lip movements requires sub-100ms timing accuracy, which IndexTTS2 provides. Audiobook producers use it to generate consistent narrator voices across chapters without re-recording. Game studios add character dialogue by cloning a single reference performance and varying emotion per line. Accessibility teams generate audio descriptions for visually impaired users.

Getting Started

IndexTTS2 is available on GitHub and Hugging Face. Installation requires Python 3.10+ and a CUDA-capable GPU. The minimal setup is:

• Install the package: pip install indextts2
• Download model weights from Hugging Face Hub
• Run inference with a 10-second reference audio and your text input
• Adjust duration ratio and emotion tags as needed

The Gradio demo on Hugging Face Spaces lets you try it without any local setup. For production deployment, a Docker container with NVIDIA runtime is the recommended approach.

Evaluation & Benchmarks

On the LibriSpeech test set, IndexTTS2 achieves a word error rate (WER) of 2.1% and speaker similarity (SECS) of 0.87. On emotion recognition benchmarks, it scores 78% accuracy for 7-class emotion classification—competitive with ground-truth human recordings at 82%. Duration control accuracy is within ±30ms of the specified target across utterances.