mirror of
https://github.com/pstrueb/piper.git
synced 2026-04-19 06:44:50 +00:00
420 lines
18 KiB
Plaintext
420 lines
18 KiB
Plaintext
{
|
|
"nbformat": 4,
|
|
"nbformat_minor": 0,
|
|
"metadata": {
|
|
"colab": {
|
|
"provenance": [],
|
|
"gpuType": "T4",
|
|
"include_colab_link": true
|
|
},
|
|
"kernelspec": {
|
|
"name": "python3",
|
|
"display_name": "Python 3"
|
|
},
|
|
"language_info": {
|
|
"name": "python"
|
|
},
|
|
"accelerator": "GPU",
|
|
"gpuClass": "standard"
|
|
},
|
|
"cells": [
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"id": "view-in-github",
|
|
"colab_type": "text"
|
|
},
|
|
"source": [
|
|
"<a href=\"https://colab.research.google.com/github/rmcpantoja/piper/blob/master/notebooks/piper_multilingual_training_notebook.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"source": [
|
|
"# <font color=\"pink\"> **[Piper](https://github.com/rhasspy/piper) training notebook.**\n",
|
|
"## \n",
|
|
"\n",
|
|
"---\n",
|
|
"\n",
|
|
"- Notebook made by [rmcpantoja](http://github.com/rmcpantoja)\n",
|
|
"- Collaborator: [Xx_Nessu_xX](https://fakeyou.com/profile/Xx_Nessu_xX)"
|
|
],
|
|
"metadata": {
|
|
"id": "eK3nmYDB6C1a"
|
|
}
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"source": [
|
|
"# <font color=\"pink\">🔧 ***First steps.*** 🔧"
|
|
],
|
|
"metadata": {
|
|
"id": "AICh6p5OJybj"
|
|
}
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"source": [
|
|
"#@markdown ## <font color=\"pink\"> **Google Colab Anti-Disconnect.** 🔌\n",
|
|
"#@markdown ---\n",
|
|
"#@markdown #### Avoid automatic disconnection. Still, it will disconnect after <font color=\"orange\">**6 to 12 hours**</font>.\n",
|
|
"\n",
|
|
"import IPython\n",
|
|
"js_code = '''\n",
|
|
"function ClickConnect(){\n",
|
|
"console.log(\"Working\");\n",
|
|
"document.querySelector(\"colab-toolbar-button#connect\").click()\n",
|
|
"}\n",
|
|
"setInterval(ClickConnect,60000)\n",
|
|
"'''\n",
|
|
"display(IPython.display.Javascript(js_code))"
|
|
],
|
|
"metadata": {
|
|
"cellView": "form",
|
|
"id": "qyxSMuzjfQrz"
|
|
},
|
|
"execution_count": null,
|
|
"outputs": []
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"source": [
|
|
"#@markdown ## <font color=\"pink\"> **Check GPU type.** 👁️\n",
|
|
"#@markdown ---\n",
|
|
"#@markdown #### A higher capable GPU can lead to faster training speeds. By default, you will have a <font color=\"orange\">**Tesla T4**</font>.\n",
|
|
"!nvidia-smi"
|
|
],
|
|
"metadata": {
|
|
"cellView": "form",
|
|
"id": "ygxzp-xHTC7T"
|
|
},
|
|
"execution_count": null,
|
|
"outputs": []
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {
|
|
"cellView": "form",
|
|
"id": "sUNjId07JfAK"
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"#@markdown # <font color=\"pink\"> **Mount Google Drive.** 📂\n",
|
|
"from google.colab import drive\n",
|
|
"drive.mount('/content/drive', force_remount=True)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"source": [
|
|
"#@markdown # <font color=\"pink\"> **Install software.** 📦\n",
|
|
"\n",
|
|
"#@markdown ####In this cell the synthesizer and its necessary dependencies to execute the training will be installed. (this may take a while)\n",
|
|
"\n",
|
|
"# clone:\n",
|
|
"!git clone -q https://github.com/rmcpantoja/piper\n",
|
|
"%cd /content/piper/src/python\n",
|
|
"!pip install -q -r requirements.txt\n",
|
|
"!pip install -q torchtext==0.12.0\n",
|
|
"!pip install -q torchvision==0.12.0\n",
|
|
"!bash build_monotonic_align.sh\n",
|
|
"!apt-get install -q espeak-ng\n",
|
|
"# download patches:\n",
|
|
"print(\"Downloading patch...\")\n",
|
|
"!gdown -q \"1EWEb7amo1rgFGpBFfRD4BKX3pkjVK1I-\" -O \"/content/piper/src/python/patch.zip\"\n",
|
|
"!unzip -o -q \"patch.zip\"\n",
|
|
"%cd /content"
|
|
],
|
|
"metadata": {
|
|
"cellView": "form",
|
|
"id": "_XwmTVlcUgCh"
|
|
},
|
|
"execution_count": null,
|
|
"outputs": []
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"source": [
|
|
"# <font color=\"pink\"> 🤖 ***Training.*** 🤖"
|
|
],
|
|
"metadata": {
|
|
"id": "A3bMzEE0V5Ma"
|
|
}
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"source": [
|
|
"#@markdown # <font color=\"pink\"> **1. Extract dataset.** 📥\n",
|
|
"#@markdown ####Important: the audios must be in <font color=\"orange\">**wav format, (16000 or 22050hz, 16-bits, mono), and, for convenience, numbered. Example:**\n",
|
|
"\n",
|
|
"#@markdown * <font color=\"orange\">**1.wav**</font>\n",
|
|
"#@markdown * <font color=\"orange\">**2.wav**</font>\n",
|
|
"#@markdown * <font color=\"orange\">**3.wav**</font>\n",
|
|
"#@markdown * <font color=\"orange\">**.....**</font>\n",
|
|
"\n",
|
|
"#@markdown ---\n",
|
|
"\n",
|
|
"%cd /content\n",
|
|
"!mkdir /content/dataset\n",
|
|
"%cd /content/dataset\n",
|
|
"!mkdir /content/dataset/wavs\n",
|
|
"#@markdown ### Audio dataset path to unzip\n",
|
|
"zip_path = \"/content/drive/MyDrive/wavs.zip\" #@param {type:\"string\"}\n",
|
|
"!unzip \"{zip_path}\" -d /content/dataset/wavs\n",
|
|
"#@markdown ---"
|
|
],
|
|
"metadata": {
|
|
"cellView": "form",
|
|
"id": "SvEGjf0aV8eg"
|
|
},
|
|
"execution_count": null,
|
|
"outputs": []
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"source": [
|
|
"#@markdown # <font color=\"pink\"> **2. Upload the transcript file.** 📝\n",
|
|
"#@markdown ---\n",
|
|
"#@markdown ####Important: the transcription means writing what the character says in each of the audios, and it must have the following structure:\n",
|
|
"\n",
|
|
"#@markdown ##### For a single-speaker dataset:\n",
|
|
"#@markdown * wavs/1.wav|This is what my character says in audio 1.\n",
|
|
"#@markdown * wavs/2.wav|This, the text that the character says in audio 2.\n",
|
|
"#@markdown * ...\n",
|
|
"\n",
|
|
"#@markdown ##### For a multi-speaker dataset:\n",
|
|
"\n",
|
|
"#@markdown * wavs/speaker1audio1.wav|speaker1|This is what the first speaker says.\n",
|
|
"#@markdown * wavs/speaker1audio2.wav|speaker1|This is another audio of the first speaker.\n",
|
|
"#@markdown * wavs/speaker2audio1.wav|speaker2|This is what the second speaker says in the first audio.\n",
|
|
"#@markdown * wavs/speaker2audio2.wav|speaker2|This is another audio of the second speaker.\n",
|
|
"#@markdown * ...\n",
|
|
"\n",
|
|
"#@markdown And so on. In addition, the transcript must be in a .csv format. (UTF-8 without BOM)\n",
|
|
"\n",
|
|
"#@markdown ---\n",
|
|
"%cd /content/dataset\n",
|
|
"from google.colab import files\n",
|
|
"!rm /content/dataset/metadata.csv\n",
|
|
"listfn, length = files.upload().popitem()\n",
|
|
"if listfn != \"metadata.csv\":\n",
|
|
" !mv \"$listfn\" metadata.csv\n",
|
|
"%cd .."
|
|
],
|
|
"metadata": {
|
|
"cellView": "form",
|
|
"id": "E0W0OCvXXvue"
|
|
},
|
|
"execution_count": null,
|
|
"outputs": []
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"source": [
|
|
"#@markdown # <font color=\"pink\"> **3. Preprocess dataset.** 🔄\n",
|
|
"\n",
|
|
"import os\n",
|
|
"#@markdown ### First of all, select the language of your dataset.\n",
|
|
"language = \"English (U.S.)\" #@param [\"Català\", \"Dansk\", \"Deutsch\", \"Ελληνικά\", \"English (British)\", \"English (U.S.)\", \"Español\", \"Suomi\", \"Français\", \"ქართული\", \"Icelandic\", \"Italiano\", \"қазақша\", \"नेपाली\", \"Nederlands\", \"Norsk\", \"Polski\", \"Português (Brasil)\", \"Русский\", \"Svenska\", \"украї́нська\", \"Tiếng Việt\", \"简体中文\"]\n",
|
|
"#@markdown ---\n",
|
|
"# language definition:\n",
|
|
"languages = {\n",
|
|
" \"Català\": \"ca\",\n",
|
|
" \"Dansk\": \"da\",\n",
|
|
" \"Deutsch\": \"de\",\n",
|
|
" \"Ελληνικά\": \"grc\",\n",
|
|
" \"English (British)\": \"en\",\n",
|
|
" \"English (U.S.)\": \"en-us\",\n",
|
|
" \"Español\": \"es\",\n",
|
|
" \"Suomi\": \"fi\",\n",
|
|
" \"Français\": \"fr\",\n",
|
|
" \"Icelandic\": \"is\",\n",
|
|
" \"Italiano\": \"it\",\n",
|
|
" \"ქართული\": \"ka\",\n",
|
|
" \"қазақша\": \"kk\",\n",
|
|
" \"नेपाली\": \"ne\",\n",
|
|
" \"Nederlands\": \"nl\",\n",
|
|
" \"Norsk\": \"nb\",\n",
|
|
" \"Polski\": \"pl\",\n",
|
|
" \"Português (Brasil)\": \"pt-br\",\n",
|
|
" \"Русский\": \"ru\",\n",
|
|
" \"Svenska\": \"sv\",\n",
|
|
" \"украї́нська\": \"uk\",\n",
|
|
" \"Tiếng Việt\": \"vi-vn-x-central\",\n",
|
|
" \"简体中文\": \"yue\"\n",
|
|
"}\n",
|
|
"\n",
|
|
"def _get_language(code):\n",
|
|
" return languages[code]\n",
|
|
"\n",
|
|
"final_language = _get_language(language)\n",
|
|
"#@markdown ### Choose a name for your model:\n",
|
|
"model_name = \"Test\" #@param {type:\"string\"}\n",
|
|
"#@markdown ---\n",
|
|
"# output:\n",
|
|
"#@markdown ### Choose the working folder: (recommended to save to Drive)\n",
|
|
"\n",
|
|
"#@markdown The working folder will be used in preprocessing, but also in training the model.\n",
|
|
"output_path = \"/content/drive/MyDrive/colab/piper\" #@param {type:\"string\"}\n",
|
|
"output_dir = output_path+\"/\"+model_name\n",
|
|
"if not os.path.exists(output_dir):\n",
|
|
" os.makedirs(output_dir)\n",
|
|
"#@markdown ---\n",
|
|
"#@markdown ### Choose dataset format:\n",
|
|
"dataset_format = \"ljspeech\" #@param [\"ljspeech\", \"mycroft\"]\n",
|
|
"#@markdown ---\n",
|
|
"#@markdown ### Is this a single speaker dataset? Otherwise, uncheck:\n",
|
|
"single_speaker = True #@param {type:\"boolean\"}\n",
|
|
"if single_speaker:\n",
|
|
" force_sp = \" --single-speaker\"\n",
|
|
"else:\n",
|
|
" force_sp = \"\"\n",
|
|
"#@markdown ---\n",
|
|
"#@markdown ### Select the sample rate of the dataset:\n",
|
|
"sample_rate = \"22050\" #@param [\"16000\", \"22050\"]\n",
|
|
"#@markdown ---\n",
|
|
"%cd /content/piper/src/python\n",
|
|
"!python -m piper_train.preprocess \\\n",
|
|
" --language {final_language} \\\n",
|
|
" --input-dir /content/dataset \\\n",
|
|
" --output-dir {output_dir} \\\n",
|
|
" --dataset-format {dataset_format} \\\n",
|
|
" --sample-rate {sample_rate} \\\n",
|
|
" {force_sp}"
|
|
],
|
|
"metadata": {
|
|
"id": "dOyx9Y6JYvRF",
|
|
"cellView": "form"
|
|
},
|
|
"execution_count": null,
|
|
"outputs": []
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"source": [
|
|
"#@markdown # <font color=\"pink\"> **4. Settings.** 🧰\n",
|
|
"import json\n",
|
|
"import ipywidgets as widgets\n",
|
|
"from IPython.display import display\n",
|
|
"from google.colab import output\n",
|
|
"import os\n",
|
|
"#@markdown ### Select the action to train this dataset:\n",
|
|
"\n",
|
|
"#@markdown * The option to convert a single-speaker model to a multi-speaker model is self-explanatory, and for this it is important that you have processed a dataset that contains text and audio from all possible speakers that you want to train in your model.\n",
|
|
"#@markdown * The finetune option is used to train a dataset using a pretrained model, that is, train on that data. This option is ideal if you want to train a very small dataset (more than five minutes recommended).\n",
|
|
"#@markdown * The train from scratch option builds features such as dictionary and speech form from scratch, and this may take longer to converge. For this, hours of audio (8 at least) are recommended, which have a large collection of phonemes.\n",
|
|
"\n",
|
|
"action = \"finetune\" #@param [\"convert single-speaker to multi-speaker model\", \"finetune\", \"train from scratch\"]\n",
|
|
"#@markdown ---\n",
|
|
"if action == \"finetune\":\n",
|
|
" ft_command = '--resume_from_checkpoint \"/content/pretrained.ckpt\" '\n",
|
|
"elif action == \"convert single-speaker to multi-speaker model\":\n",
|
|
" ft_command = '--resume_from_single_speaker_checkpoint \"/content/pretrained.ckpt\" '\n",
|
|
"else:\n",
|
|
" ft_command = \"\"\n",
|
|
"if action== \"convert single-speaker to multi-speaker model\" or action == \"finetune\":\n",
|
|
" try:\n",
|
|
" with open('/content/piper/notebooks/pretrained_models.json') as f:\n",
|
|
" pretrained_models = json.load(f)\n",
|
|
" if final_language in pretrained_models:\n",
|
|
" models = pretrained_models[final_language]\n",
|
|
" model_options = [(model_name, model_name) for model_name, model_url in models.items()]\n",
|
|
" model_dropdown = widgets.Dropdown(description = \"Choose pretrained model\", options=model_options)\n",
|
|
" download_button = widgets.Button(description=\"Download\")\n",
|
|
" def download_model(btn):\n",
|
|
" model_name = model_dropdown.value\n",
|
|
" model_url = pretrained_models[final_language][model_name]\n",
|
|
" print(\"Downloading pretrained model...\")\n",
|
|
" if model_url.startswith(\"1\"):\n",
|
|
" !gdown -q \"{model_url}\" -O \"/content/pretrained.ckpt\"\n",
|
|
" elif model_url.startswith(\"https://drive.google.com/file/d/\"):\n",
|
|
" !gdown -q \"{model_url}\" -O \"/content/pretrained.ckpt\" --fuzzy\n",
|
|
" else:\n",
|
|
" !wget -q \"{model_url}\" -O \"/content/pretrained.ckpt\"\n",
|
|
" model_dropdown.close()\n",
|
|
" download_button.close()\n",
|
|
" output.clear()\n",
|
|
" if os.path.exists(\"/content/pretrained.ckpt\"):\n",
|
|
" print(\"Model downloaded!\")\n",
|
|
" else:\n",
|
|
" raise Exception(\"Couldn't download the pretrained model!\")\n",
|
|
" download_button.on_click(download_model)\n",
|
|
" display(model_dropdown, download_button)\n",
|
|
" else:\n",
|
|
" raise Exception(f\"There are no pretrained models available for the language {final_language}\")\n",
|
|
" except FileNotFoundError:\n",
|
|
" raise Exception(\"The pretrained_models.json file was not found.\")\n",
|
|
"else:\n",
|
|
" print(\"Warning: this model will be trained from scratch. You need at least 8 hours of data for everything to work decent. Good luck!\")\n",
|
|
"#@markdown ### Choose batch size based on this dataset:\n",
|
|
"batch_size = 8 #@param {type:\"integer\"}\n",
|
|
"#@markdown ---\n",
|
|
"validation_split = 0.01\n",
|
|
"#@markdown ### Choose the quality for this model:\n",
|
|
"\n",
|
|
"#@markdown * x-low - 16Khz audio, 5-7M params\n",
|
|
"#@markdown * medium - 22.05Khz audio, 15-20 params\n",
|
|
"#@markdown * high - 22.05Khz audio, 28-32M params\n",
|
|
"quality = \"medium\" #@param [\"high\", \"x-low\", \"medium\"]\n",
|
|
"#@markdown ---\n",
|
|
"#@markdown ### For how many epochs to save training checkpoints?\n",
|
|
"#@markdown The larger your dataset, you should set this saving interval to a smaller value, as epochs can progress longer time.\n",
|
|
"checkpoint_epochs = 3 #@param {type:\"integer\"}\n",
|
|
"#@markdown ---\n",
|
|
"#@markdown ### Step interval to generate model samples:\n",
|
|
"log_every_n_steps = 250 #@param {type:\"integer\"}\n",
|
|
"#@markdown ---\n",
|
|
"#@markdown ### Training epochs:\n",
|
|
"max_epochs = 10000 #@param {type:\"integer\"}\n",
|
|
"#@markdown ---"
|
|
],
|
|
"metadata": {
|
|
"id": "ickQlOCRjkBL",
|
|
"cellView": "form"
|
|
},
|
|
"execution_count": null,
|
|
"outputs": []
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"source": [
|
|
"#@markdown # <font color=\"pink\"> **5. Train.** 🏋️♂️\n",
|
|
"#@markdown Run this cell to train your final model! If possible, some audio samples will be saved during training in the output folder.\n",
|
|
"\n",
|
|
"get_ipython().system(f'''\n",
|
|
"python -m piper_train \\\n",
|
|
"--dataset-dir \"{output_dir}\" \\\n",
|
|
"--accelerator 'gpu' \\\n",
|
|
"--devices 1 \\\n",
|
|
"--batch-size {batch_size} \\\n",
|
|
"--validation-split {validation_split} \\\n",
|
|
"--num-test-examples 2 \\\n",
|
|
"--quality {quality} \\\n",
|
|
"--checkpoint-epochs {checkpoint_epochs} \\\n",
|
|
"--log_every_n_steps {log_every_n_steps} \\\n",
|
|
"--max_epochs {max_epochs} \\\n",
|
|
"{ft_command}\\\n",
|
|
"--precision 32\n",
|
|
"''')"
|
|
],
|
|
"metadata": {
|
|
"id": "X4zbSjXg2J3N",
|
|
"cellView": "form"
|
|
},
|
|
"execution_count": null,
|
|
"outputs": []
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"source": [
|
|
"# Have you finished training and want to test the model?\n",
|
|
"\n",
|
|
"Export your model using the [model exporter notebook](https://colab.research.google.com/github/rmcpantoja/piper/blob/master/notebooks/piper_model_exporter.ipynb)!"
|
|
],
|
|
"metadata": {
|
|
"id": "6ISG085SYn85"
|
|
}
|
|
}
|
|
]
|
|
} |