ESPnet2 TTS model
mio/Artoria
This model was trained by mio using fate recipe in espnet.
Demo: How to use in ESPnet2
Follow the ESPnet installation instructions
if you haven’t done that already.
cd espnet<br /> git checkout 49d18064f22b7508ff24a7fa70c470a65f08f1be<br /> pip install -e .<br /> cd egs2/fate/tts1<br /> ./run.sh --skip_data_prep false --skip_train true --download_model mio/Artoria<br />
TTS config
expand
config: conf/tuning/finetune_vits.yaml<br /> print_config: false<br /> log_level: INFO<br /> dry_run: false<br /> iterator_type: sequence<br /> output_dir: exp/22k/tts_fate_saber_vits_finetune_from_jsut<br /> ngpu: 1<br /> seed: 777<br /> num_workers: 4<br /> num_att_plot: 0<br /> dist_backend: nccl<br /> dist_init_method: env://<br /> dist_world_size: 4<br /> dist_rank: 0<br /> local_rank: 0<br /> dist_master_addr: localhost<br /> dist_master_port: 46762<br /> dist_launcher: null<br /> multiprocessing_distributed: true<br /> unused_parameters: true<br /> sharded_ddp: false<br /> cudnn_enabled: true<br /> cudnn_benchmark: false<br /> cudnn_deterministic: false<br /> collect_stats: false<br /> write_collected_feats: false<br /> max_epoch: 10<br /> patience: null<br /> val_scheduler_criterion:<br /> - valid<br /> - loss<br /> early_stopping_criterion:<br /> - valid<br /> - loss<br /> - min<br /> best_model_criterion:<br /> - - train<br /> - total_count<br /> - max<br /> keep_nbest_models: 10<br /> nbest_averaging_interval: 0<br /> grad_clip: -1<br /> grad_clip_type: 2.0<br /> grad_noise: false<br /> accum_grad: 1<br /> no_forward_run: false<br /> resume: true<br /> train_dtype: float32<br /> use_amp: false<br /> log_interval: 50<br /> use_matplotlib: true<br /> use_tensorboard: false<br /> create_graph_in_tensorboard: false<br /> use_wandb: true<br /> wandb_project: fate<br /> wandb_id: null<br /> wandb_entity: null<br /> wandb_name: vits_train_saber<br /> wandb_model_log_interval: -1<br /> detect_anomaly: false<br /> pretrain_path: null<br /> init_param:<br /> - downloads/f3698edf589206588f58f5ec837fa516/exp/tts_train_vits_raw_phn_jaconv_pyopenjtalk_accent_with_pause/train.total_count.ave_10best.pth:tts:tts<br /> ignore_init_mismatch: false<br /> freeze_param: []<br /> num_iters_per_epoch: 1000<br /> batch_size: 20<br /> valid_batch_size: null<br /> batch_bins: 5000000<br /> valid_batch_bins: null<br /> train_shape_file:<br /> - exp/22k/tts_stats_raw_linear_spectrogram_phn_jaconv_pyopenjtalk_accent_with_pause/train/text_shape.phn<br /> - exp/22k/tts_stats_raw_linear_spectrogram_phn_jaconv_pyopenjtalk_accent_with_pause/train/speech_shape<br /> valid_shape_file:<br /> - exp/22k/tts_stats_raw_linear_spectrogram_phn_jaconv_pyopenjtalk_accent_with_pause/valid/text_shape.phn<br /> - exp/22k/tts_stats_raw_linear_spectrogram_phn_jaconv_pyopenjtalk_accent_with_pause/valid/speech_shape<br /> batch_type: numel<br /> valid_batch_type: null<br /> fold_length:<br /> - 150<br /> - 204800<br /> sort_in_batch: descending<br /> sort_batch: descending<br /> multiple_iterator: false<br /> chunk_length: 500<br /> chunk_shift_ratio: 0.5<br /> num_cache_chunks: 1024<br /> train_data_path_and_name_and_type:<br /> - - dump/22k/raw/train/text<br /> - text<br /> - text<br /> - - dump/22k/raw/train/wav.scp<br /> - speech<br /> - sound<br /> valid_data_path_and_name_and_type:<br /> - - dump/22k/raw/dev/text<br /> - text<br /> - text<br /> - - dump/22k/raw/dev/wav.scp<br /> - speech<br /> - sound<br /> allow_variable_data_keys: false<br /> max_cache_size: 0.0<br /> max_cache_fd: 32<br /> valid_max_cache_size: null<br /> optim: adamw<br /> optim_conf:<br /> lr: 0.0001<br /> betas:<br /> - 0.8<br /> - 0.99<br /> eps: 1.0e-09<br /> weight_decay: 0.0<br /> scheduler: exponentiallr<br /> scheduler_conf:<br /> gamma: 0.999875<br /> optim2: adamw<br /> optim2_conf:<br /> lr: 0.0001<br /> betas:<br /> - 0.8<br /> - 0.99<br /> eps: 1.0e-09<br /> weight_decay: 0.0<br /> scheduler2: exponentiallr<br /> scheduler2_conf:<br /> gamma: 0.999875<br /> generator_first: false<br /> token_list:<br /> - <blank><br /> - <unk><br /> - '1'<br /> - '2'<br /> - '0'<br /> - '3'<br /> - '4'<br /> - '-1'<br /> - '5'<br /> - a<br /> - o<br /> - '-2'<br /> - i<br /> - '-3'<br /> - u<br /> - e<br /> - k<br /> - n<br /> - t<br /> - '6'<br /> - r<br /> - '-4'<br /> - s<br /> - N<br /> - m<br /> - pau<br /> - '7'<br /> - sh<br /> - d<br /> - g<br /> - w<br /> - '8'<br /> - U<br /> - '-5'<br /> - I<br /> - cl<br /> - h<br /> - y<br /> - b<br /> - '9'<br /> - j<br /> - ts<br /> - ch<br /> - '-6'<br /> - z<br /> - p<br /> - '-7'<br /> - f<br /> - ky<br /> - ry<br /> - '-8'<br /> - gy<br /> - '-9'<br /> - hy<br /> - ny<br /> - '-10'<br /> - by<br /> - my<br /> - '-11'<br /> - '-12'<br /> - '-13'<br /> - py<br /> - '-14'<br /> - '-15'<br /> - v<br /> - '10'<br /> - '-16'<br /> - '-17'<br /> - '11'<br /> - '-21'<br /> - '-20'<br /> - '12'<br /> - '-19'<br /> - '13'<br /> - '-18'<br /> - '14'<br /> - dy<br /> - '15'<br /> - ty<br /> - '-22'<br /> - '16'<br /> - '18'<br /> - '19'<br /> - '17'<br /> - <sos/eos><br /> odim: null<br /> model_conf: {}<br /> use_preprocessor: true<br /> token_type: phn<br /> bpemodel: null<br /> non_linguistic_symbols: null<br /> cleaner: jaconv<br /> g2p: pyopenjtalk_accent_with_pause<br /> feats_extract: linear_spectrogram<br /> feats_extract_conf:<br /> n_fft: 1024<br /> hop_length: 256<br /> win_length: null<br /> normalize: null<br /> normalize_conf: {}<br /> tts: vits<br /> tts_conf:<br /> generator_type: vits_generator<br /> generator_params:<br /> hidden_channels: 192<br /> spks: -1<br /> global_channels: -1<br /> segment_size: 32<br /> text_encoder_attention_heads: 2<br /> text_encoder_ffn_expand: 4<br /> text_encoder_blocks: 6<br /> text_encoder_positionwise_layer_type: conv1d<br /> text_encoder_positionwise_conv_kernel_size: 3<br /> text_encoder_positional_encoding_layer_type: rel_pos<br /> text_encoder_self_attention_layer_type: rel_selfattn<br /> text_encoder_activation_type: swish<br /> text_encoder_normalize_before: true<br /> text_encoder_dropout_rate: 0.1<br /> text_encoder_positional_dropout_rate: 0.0<br /> text_encoder_attention_dropout_rate: 0.1<br /> use_macaron_style_in_text_encoder: true<br /> use_conformer_conv_in_text_encoder: false<br /> text_encoder_conformer_kernel_size: -1<br /> decoder_kernel_size: 7<br /> decoder_channels: 512<br /> decoder_upsample_scales:<br /> - 8<br /> - 8<br /> - 2<br /> - 2<br /> decoder_upsample_kernel_sizes:<br /> - 16<br /> - 16<br /> - 4<br /> - 4<br /> decoder_resblock_kernel_sizes:<br /> - 3<br /> - 7<br /> - 11<br /> decoder_resblock_dilations:<br /> - - 1<br /> - 3<br /> - 5<br /> - - 1<br /> - 3<br /> - 5<br /> - - 1<br /> - 3<br /> - 5<br /> use_weight_norm_in_decoder: true<br /> posterior_encoder_kernel_size: 5<br /> posterior_encoder_layers: 16<br /> posterior_encoder_stacks: 1<br /> posterior_encoder_base_dilation: 1<br /> posterior_encoder_dropout_rate: 0.0<br /> use_weight_norm_in_posterior_encoder: true<br /> flow_flows: 4<br /> flow_kernel_size: 5<br /> flow_base_dilation: 1<br /> flow_layers: 4<br /> flow_dropout_rate: 0.0<br /> use_weight_norm_in_flow: true<br /> use_only_mean_in_flow: true<br /> stochastic_duration_predictor_kernel_size: 3<br /> stochastic_duration_predictor_dropout_rate: 0.5<br /> stochastic_duration_predictor_flows: 4<br /> stochastic_duration_predictor_dds_conv_layers: 3<br /> vocabs: 85<br /> aux_channels: 513<br /> discriminator_type: hifigan_multi_scale_multi_period_discriminator<br /> discriminator_params:<br /> scales: 1<br /> scale_downsample_pooling: AvgPool1d<br /> scale_downsample_pooling_params:<br /> kernel_size: 4<br /> stride: 2<br /> padding: 2<br /> scale_discriminator_params:<br /> in_channels: 1<br /> out_channels: 1<br /> kernel_sizes:<br /> - 15<br /> - 41<br /> - 5<br /> - 3<br /> channels: 128<br /> max_downsample_channels: 1024<br /> max_groups: 16<br /> bias: true<br /> downsample_scales:<br /> - 2<br /> - 2<br /> - 4<br /> - 4<br /> - 1<br /> nonlinear_activation: LeakyReLU<br /> nonlinear_activation_params:<br /> negative_slope: 0.1<br /> use_weight_norm: true<br /> use_spectral_norm: false<br /> follow_official_norm: false<br /> periods:<br /> - 2<br /> - 3<br /> - 5<br /> - 7<br /> - 11<br /> period_discriminator_params:<br /> in_channels: 1<br /> out_channels: 1<br /> kernel_sizes:<br /> - 5<br /> - 3<br /> channels: 32<br /> downsample_scales:<br /> - 3<br /> - 3<br /> - 3<br /> - 3<br /> - 1<br /> max_downsample_channels: 1024<br /> bias: true<br /> nonlinear_activation: LeakyReLU<br /> nonlinear_activation_params:<br /> negative_slope: 0.1<br /> use_weight_norm: true<br /> use_spectral_norm: false<br /> generator_adv_loss_params:<br /> average_by_discriminators: false<br /> loss_type: mse<br /> discriminator_adv_loss_params:<br /> average_by_discriminators: false<br /> loss_type: mse<br /> feat_match_loss_params:<br /> average_by_discriminators: false<br /> average_by_layers: false<br /> include_final_outputs: true<br /> mel_loss_params:<br /> fs: 22050<br /> n_fft: 1024<br /> hop_length: 256<br /> win_length: null<br /> window: hann<br /> n_mels: 80<br /> fmin: 0<br /> fmax: null<br /> log_base: null<br /> lambda_adv: 1.0<br /> lambda_mel: 45.0<br /> lambda_feat_match: 2.0<br /> lambda_dur: 1.0<br /> lambda_kl: 1.0<br /> sampling_rate: 22050<br /> cache_generator_outputs: true<br /> pitch_extract: null<br /> pitch_extract_conf: {}<br /> pitch_normalize: null<br /> pitch_normalize_conf: {}<br /> energy_extract: null<br /> energy_extract_conf: {}<br /> energy_normalize: null<br /> energy_normalize_conf: {}<br /> required:<br /> - output_dir<br /> - token_list<br /> version: '202207'<br /> distributed: true<br />
Citing ESPnet
@inproceedings{watanabe2018espnet,<br /> author={Shinji Watanabe and Takaaki Hori and Shigeki Karita and Tomoki Hayashi and Jiro Nishitoba and Yuya Unno and Nelson Yalta and Jahn Heymann and Matthew Wiesner and Nanxin Chen and Adithya Renduchintala and Tsubasa Ochiai},<br /> title={{ESPnet}: End-to-End Speech Processing Toolkit},<br /> year={2018},<br /> booktitle={Proceedings of Interspeech},<br /> pages={2207--2211},<br /> doi={10.21437/Interspeech.2018-1456},<br /> url={http://dx.doi.org/10.21437/Interspeech.2018-1456}<br /> }<br /> @inproceedings{hayashi2020espnet,<br /> title={{Espnet-TTS}: Unified, reproducible, and integratable open source end-to-end text-to-speech toolkit},<br /> author={Hayashi, Tomoki and Yamamoto, Ryuichi and Inoue, Katsuki and Yoshimura, Takenori and Watanabe, Shinji and Toda, Tomoki and Takeda, Kazuya and Zhang, Yu and Tan, Xu},<br /> booktitle={Proceedings of IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)},<br /> pages={7654--7658},<br /> year={2020},<br /> organization={IEEE}<br /> }<br />
or arXiv:
@misc{watanabe2018espnet,<br /> title={ESPnet: End-to-End Speech Processing Toolkit},<br /> author={Shinji Watanabe and Takaaki Hori and Shigeki Karita and Tomoki Hayashi and Jiro Nishitoba and Yuya Unno and Nelson Yalta and Jahn Heymann and Matthew Wiesner and Nanxin Chen and Adithya Renduchintala and Tsubasa Ochiai},<br /> year={2018},<br /> eprint={1804.00015},<br /> archivePrefix={arXiv},<br /> primaryClass={cs.CL}<br /> }<br />
收录说明:
1、本网页并非 mio/Artoria 官网网址页面,此页面内容编录于互联网,只作展示之用;2、如果有与 mio/Artoria 相关业务事宜,请访问其网站并获取联系方式;3、本站与 mio/Artoria 无任何关系,对于 mio/Artoria 网站中的信息,请用户谨慎辨识其真伪。4、本站收录 mio/Artoria 时,此站内容访问正常,如遇跳转非法网站,有可能此网站被非法入侵或者已更换新网址,导致旧网址被非法使用,5、如果你是网站站长或者负责人,不想被收录请邮件删除:i-hu#Foxmail.com (#换@)
前往AI网址导航
2、本站所有文章、图片、资源等如果未标明原创,均为收集自互联网公开资源;分享的图片、资源、视频等,出镜模特均为成年女性正常写真内容,版权归原作者所有,仅作为个人学习、研究以及欣赏!如有涉及下载请24小时内删除;
3、如果您发现本站上有侵犯您的权益的作品,请与我们取得联系,我们会及时修改、删除并致以最深的歉意。邮箱: i-hu#(#换@)foxmail.com