Vocoder with HiFIGAN trained on LJSpeech

This repository provides all the necessary tools for using a HiFIGAN vocoder trained with LJSpeech.
The pre-trained model takes in input a spectrogram and produces a waveform in output. Typically, a vocoder is used after a TTS model that converts an input text into a spectrogram.
The sampling frequency is 22050 Hz.

Install SpeechBrain

pip install speechbrain<br />

Please notice that we encourage you to read our tutorials and learn more about
SpeechBrain.

Using the Vocoder

import torch<br /> from speechbrain.pretrained import HIFIGAN<br /> hifi_gan = HIFIGAN.from_hparams(source="speechbrain/tts-hifigan-ljspeech", savedir="tmpdir")<br /> mel_specs = torch.rand(2, 80,298)<br /> waveforms = hifi_gan.decode_batch(mel_specs)<br />

Using the Vocoder with the TTS

import torchaudio<br /> from speechbrain.pretrained import Tacotron2<br /> from speechbrain.pretrained import HIFIGAN<br /> # Intialize TTS (tacotron2) and Vocoder (HiFIGAN)<br /> tacotron2 = Tacotron2.from_hparams(source="speechbrain/tts-tacotron2-ljspeech", savedir="tmpdir_tts")<br /> hifi_gan = HIFIGAN.from_hparams(source="speechbrain/tts-hifigan-ljspeech", savedir="tmpdir_vocoder")<br /> # Running the TTS<br /> mel_output, mel_length, alignment = tacotron2.encode_text("Mary had a little lamb")<br /> # Running Vocoder (spectrogram-to-waveform)<br /> waveforms = hifi_gan.decode_batch(mel_output)<br /> # Save the waverform<br /> torchaudio.save('example_TTS.wav',waveforms.squeeze(1), 22050)<br />

Inference on GPU

To perform inference on the GPU, add run_opts={"device":"cuda"} when calling the from_hparams method.

Training

The model was trained with SpeechBrain.
To train it from scratch follow these steps:

1. Clone SpeechBrain:

git clone https://github.com/speechbrain/speechbrain/<br />

1. Install it:

cd speechbrain<br /> pip install -r requirements.txt<br /> pip install -e .<br />

1. Run Training:

cd recipes/LJSpeech/TTS/vocoder/hifi_gan/<br /> python train.py hparams/train.yaml --data_folder /path/to/LJspeech<br />

You can find our training results (models, logs, etc) here.