kromskii2/alltalk_tts
1
0 Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Merge pull request #85 from erew123/main

Browse Source 
main to dev merge
This commit is contained in:
erew123 authored and GitHub committed 2024-01-23 13:01:56 +00:00
2 parents 5c0f08d49e ab37dd8849
commit 4caecdc5eb
13 files changed
+1578 -1390
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

.github/FUNDING.yml
+1
View File
@@ -0,0 +1 @@
ko_fi: erew123
README.md
+82 -19
View File
@@ -15,7 +15,7 @@ AllTalk is an updated version of the Coqui_tts extension for Text Generation web
- **API Suite and 3rd Party support via JSON calls:** Can be used with 3rd party applications via JSON calls.
- **SillyTavern integration:** Full integration with SillyTavern. [Screenshot](https://github.com/erew123/screenshots/raw/main/sillytavern.jpg)
## Index
### Index
- 🟩 [Installation](https://github.com/erew123/alltalk_tts/#-quick-setup-text-generation-webui--standalone-installation)
- 🟪 [Updating & problems with updating](https://github.com/erew123/alltalk_tts?#-updating)
@@ -27,18 +27,25 @@ AllTalk is an updated version of the Coqui_tts extension for Text Generation web
- 🟠 [API Suite and JSON-CURL](https://github.com/erew123/alltalk_tts?#-api-suite-and-json-curl)
- 🔴 [Future to-do list & Upcoming updates](https://github.com/erew123/alltalk_tts?#-future-to-do-list)
## 🔄 Updates & Bug fixes
### 🔄 Feature requests, Updates & Bug fixes
Please check the below link to find a list of all recent updates and changes.
####     🔄 **Updates list & bug fixes list** can be found [here](https://github.com/erew123/alltalk_tts/issues/25)
####     🔄 **Current Feature request list** can be found [here](https://github.com/erew123/alltalk_tts/discussions/74)
## 🟩 Quick Setup (Text-generation-webui & Standalone Installation)
For Windows 10/11 and Linux machines there is a quick setup script. Click to expand the correct section below:
I welcome your input and ideas for new features, suggestions, and improvements. Feel free to share your thoughts and collaborate in the discussions area. If you find this project valuable and would like to show your appreciation, you can make a donation on my [Ko-fi](https://ko-fi.com/erew123) page. Your support goes a long way in ensuring that I can continue to deliver even better features and experiences.
**ERROR** `ImportError: cannot import name 'SampleOutput' from 'transformers.generation.utils'` please see this issue [here](https://github.com/erew123/alltalk_tts/issues/82)
### 🟩 Quick Setup (Text-generation-webui & Standalone Installation)
For Windows 10/11 and Linux machines there is a quick setup script. Please note, Python on Windows requires you install the C++ development [tools](https://wiki.python.org/moin/WindowsCompilers) to compile packages, further details can be found in the help section.
Click to expand the correct section below:
<details>
<summary>QUICK SETUP - Text-Generation-webui</summary><br>
If you wish to see this as a video, please go [here](https://www.youtube.com/watch?v=icn2XS5rUH8)
1) To download the AllTalk setup you can either:
- A) On this page select **CODE** > **Download ZIP** then extract it to the text-generation-webui extensions folder<br>e.g. `\text-generation-webui\extensions\alltalk_tts\`<br><br>
- A) Go to the [Releases page](https://github.com/erew123/alltalk_tts/releases) and download the latest `alltalk_tts.zip` then extract it to the text-generation-webui extensions folder<br>e.g. `\text-generation-webui\extensions\alltalk_tts\`.<br><br>
- B) Go to a terminal/console, move into the `\text-generation-webui\extensions\` folder<br>and `git clone https://github.com/erew123/alltalk_tts`<br><br>
3) In a terminal/command prompt, in the text-generation-webui folder you will start its Python environment with either `cmd_windows.bat` or `./cmd_linux.sh`
4) Move into the AllTalk folder e.g. `cd extensions` then `cd alltalk_tts`
@@ -56,7 +63,7 @@ For Windows 10/11 and Linux machines there is a quick setup script. Click to exp
If you wish to see this as a video, please go [here](https://www.youtube.com/watch?v=AQYCccDRbaY)
1) To download the AllTalk setup you can either:
- A) On this page select **CODE** > **Download ZIP** then extract it to the folder of your choice<br>e.g. `C:\myfiles\alltalk_tts\`<br><br>
- A) Go to the [Releases page](https://github.com/erew123/alltalk_tts/releases) and download the latest `alltalk_tts.zip` and extract it to the folder of your choice<br>e.g. `C:\myfiles\alltalk_tts\`.<br><br>
- B) Go to a terminal/console, move into the folder of your choice e.g `C:\myfiles\` folder<br>and `git clone https://github.com/erew123/alltalk_tts`<br><br>
4) In a terminal/command prompt, move into the AllTalk folder e.g. `cd alltalk_tts`
5) Start the AllTalk setup script `atsetup.bat` or `./atsetup.sh`
@@ -66,10 +73,12 @@ For Windows 10/11 and Linux machines there is a quick setup script. Click to exp
Please read the `🟩 Other installation notes` (also additional voices are available there).
You cannot have a dash in your folder path e.g. `c:\myfiles\alltalk_tts-main` so please ensure you remove any `-` from your folder path. This is a conda specific requirement and will cause AllTalk not to start at all.
Finetuning has other installation requirements so please read any instructions in the setup utility and refer back here to this page for detailed instructions.<br><br>
</details>
## 🟩 Manual Installation - As part of Text generation web UI
### 🟩 Manual Installation - As part of Text generation web UI
On Mac's or if you wish to perform a manual installation. Click to expand the correct section below:
<details>
<summary>MANUAL INSTALLATION - Text-Generation-webui</summary><br>
@@ -109,7 +118,7 @@ This has been tested on the current Dec 2023 release of Text generation webUI. I
</details>
## 🟩 Manual Installation - As a Standalone Application
### 🟩 Manual Installation - As a Standalone Application
On Mac's or if you wish to perform a manual installation. Click to expand the correct section below:
<details>
<summary>MANUAL INSTALLATION - I want to run AllTalk as a standalone when installed with Text-generation-webui</summary><br>
@@ -160,7 +169,7 @@ Deepspeed and other such things can be installed. Please read the relevant instr
#### 🟩 Other installation notes
On first startup, AllTalk will download the Coqui XTTSv2 2.0.2 model to its **models** folder (1.8GB space required). Check the command prompt/terminal window if you want to know what its doing. After it says "Model Loaded" the Text generation webUI is usually available on its IP address a few seconds later, for you to connect to in your browser.
On first startup, AllTalk will download the Coqui XTTSv2 2.0.2 model to its **models** folder (1.8GB space required). Check the command prompt/terminal window if you want to know what its doing. After it says "Model Loaded" the Text generation webUI is usually available on its IP address a few seconds later, for you to connect to in your browser. If you are running a headless system and need to change the IP, please see the Help with problems section down below.
Once the extension is loaded, please find all documentation and settings on the link provided in the interface (as shown in the screenshot below).
@@ -179,7 +188,7 @@ Most narrator/character systems switch voices upon encountering an asterisk or q
With improvements to the splitter/processor, I'm confident it's functioning well. You can monitor what AllTalk identifies as Narrator lines on the command line and adjust its behavior if needed (Text Not Inside - Function).
## 🟪 Updating
### 🟪 Updating
<details>
<summary>UPDATING - I am using Text-Generation-webui</summary><br>
This is pretty much a repeat of the installation process.
@@ -258,12 +267,12 @@ You can now start text-generation-webui or AllTalk (standalone) and it should st
Assuming its all working fine and you are happy, you can delete the old alltalk_tts.old folder.
</details>
## 🟫 Screenshots
### 🟫 Screenshots
|![image](https://github.com/erew123/screenshots/raw/main/textgensettings.jpg) | ![image](https://github.com/erew123/screenshots/raw/main/setuputilitys.jpg) | ![image](https://github.com/erew123/screenshots/raw/main/deepspeed.jpg) |![image](https://github.com/erew123/screenshots/raw/main/textgen.jpg) |
|:---:|:---:|:---:|:---:|
|![image](https://github.com/erew123/screenshots/raw/main/settingsanddocs.jpg) | ![image](https://github.com/erew123/screenshots/raw/main/finetune1.jpg) | ![image](https://github.com/erew123/screenshots/raw/main/finetune2.jpg) |![image](https://github.com/erew123/screenshots/raw/main/sillytavern.jpg)|
## 🟨 Help with problems
### 🟨 Help with problems
#### &nbsp;&nbsp;&nbsp;&nbsp; 🔄 **Minor updates/bug fixes list** can be found [here](https://github.com/erew123/alltalk_tts/issues/25)
@@ -296,7 +305,7 @@ If you are on a Windows machine or a Linux machine, you should be able to use th
**A)** Its trying to load the voice model into your graphics card VRAM (assuming you have a Nvidia Graphics card, otherwise its your system RAM)<br>
**B)** Its trying to start up the mini-webserver and send the "ready" signal back to the main process.
Before giving other possibilities a go, some people with **old machines** are finding their startup times are **very** slow 2-3 minutes. Ive extended the allowed time within the script from 1 minute to 2 minutes. **If you have an older machine** and wish to try extending this further, you can do so by editing `script.py` and changing line 251 `timeout = 120` changing the timeout to a larger value e.g `timeout = 240` (4 minutes).
Before giving other possibilities a go, some people with **old machines** are finding their startup times are **very** slow 2-3 minutes. Ive extended the allowed time within the script from 1 minute to 2 minutes. **If you have an older machine** and wish to try extending this further, you can do so by editing `script.py` and changing `startup_wait_time = 120` (120 seconds, aka 2 minutes) at the top of the script.py file, to a larger value e.g `startup_wait_time = 240` (240 seconds aka 4 minutes).
**Note:** If you need to create a support ticket, please create a `diagnostics.log` report file to submit with a support request. Details on doing this are above.
@@ -325,6 +334,21 @@ Other possibilities for this issue are:
11) You have built yourself a custom Python environment and something is funky with it. This is very hard to diagnose as its not a standard environment. You may want to updating text-generation-webui and re installing its requirements file (whichever one you use that comes down with text-generation-webui).
</details>
#### 🟨 Windows & Python requirements for compiling packages
`ERROR: Microsoft Visual C++ 14.0 or greater is required` or `ERROR: Could not build wheels for TTS.`
<details>
<summary>Click to expand</summary><br>
Python requires that you install C++ development tools on Windows. This is detailed on the [Python site here](https://wiki.python.org/moin/WindowsCompilers). You would need to install `MSVCv142 - VS 2019 C++ x64/x86 build tools` and `Windows 10/11 SDK` from the C++ Build tools section.
You can get hold of the **Community** edition [here](https://visualstudio.microsoft.com/downloads/) the during installation, selecting `C++ Build tools` and then `MSVCv142 - VS 2019 C++ x64/x86 build tools` and `Windows 10/11 SDK`.
![image](https://github.com/erew123/screenshots/raw/main/pythonrequirementswindows.jpg)
</details>
#### 🟨 I think AllTalks requirements file has installed something another extension doesn't like
<details>
<summary>Click to expand</summary><br>
@@ -347,6 +371,15 @@ You can either follow the [Problems Updating](https://github.com/erew123/alltalk
If you want to use DeepSpeed, you need an Nvidia Graphics card and to install DeepSpeed on your system. Instructions are [here](https://github.com/erew123/alltalk_tts#-deepspeed-installation-options)
</details>
#### 🟨 I am running a Headless system and need to change the IP Address manually as I cannot reach the config page.
<details>
<summary>Click to expand</summary><br>
To do this you can edit the `confignew.json` file within the `alltalk_tts` folder. You would look for `"ip_address": "127.0.0.1",` and change the `127.0.0.1` to your chosen IP address,then save the file and start AllTalk.<br><br>
When doing this, be careful not to impact the formatting of the JSON file. Worst case, you can re-download a fresh copy of `confignew.json` from this website and that will put you back to a factory setting.
</details>
#### 🟨 I am having problems updating/some other issue where it wont start up/Im sure this is a bug
<details>
<summary>Click to expand</summary><br>
@@ -356,6 +389,14 @@ Please see [Problems Updating](https://github.com/erew123/alltalk_tts#-problems-
Also, is your text-generation-webui up to date? [instructions here](https://github.com/oobabooga/text-generation-webui?tab=readme-ov-file#how-to-install)
</details>
#### 🟨 Standalone Install - start_{youros}.xx opens and closes instantly and AllTalk doesnt start
<details>
<summary>Click to expand</summary><br>
This is more than likely caused by having a `-` in your folder path e.g. `c:\myfiles\alltalk_tts-main`. In this circumstance you would be best renaming the folder to remove the `-` from its name e.g. `c:\myfiles\alltalk_tts`, delete the `alltalk_environment` folder and `start_alltalk.bat` or `start_alltalk.sh` and then re-run `atsetup` to re-create the environment and startup files.
</details>
#### 🟨 I am having problems getting AllTalk to start after changing settings or making a custom setup/model setup.
<details>
@@ -391,7 +432,7 @@ Depending on which of your Nvidia GPU's is the more powerful one, you can change
</details>
## ⚫ Finetuning a model
### ⚫ Finetuning a model
If you have a voice that the model doesnt quite reproduce correctly, or indeed you just want to improve the reproduced voice, then finetuning is a way to train your "XTTSv2 local" model **(stored in `/alltalk_tts/models/xxxxx/`)** on a specific voice. For this you will need:
- An Nvidia graphics card. (Please see this [note](https://github.com/erew123/alltalk_tts#-i-have-multiple-gpus-and-i-have-problems-running-finetuning) if you have multiple Nvidia GPU's).
@@ -406,6 +447,14 @@ Everything has been done to make this as simple as possible. At its simplest, yo
#### ⚫ The audio you will use
I would suggest that if its in an interview format, you cut out the interviewer speaking in audacity or your chosen audio editing package. You dont have to worry about being perfect with your cuts, the finetuning Step 1 will go and find spoken audio and cut it out for you. Is there is music over the spoken parts, for best quality you would cut out those parts, though its not 100% necessary. As always, try to avoid bad quality audio with noises in it (humming sounds, hiss etc). You can try something like [Audioenhancer](https://audioenhancer.ai/) to try clean up noisier audio. There is no need to down-sample any of the audio, all of that is handled for you. Just give the finetuning some good quality audio to work with.
#### ⚫ A note about anonymous training Telemetry information & disabling it
Portions of Coqui's TTS trainer scripts gather anonymous training information which you can disable. Their statement on this is listed [here](https://github.com/coqui-ai/Trainer?tab=readme-ov-file#anonymized-telemetry). Although I have tried to pass `TRAINER_TELEMETRY=0` through AllTalk, it appears you will have to set this manually if you wish to finetune on a non-internet enabled computer or disable the anonymous data being sent. You can do this by:
- On Windows by typing `set TRAINER_TELEMETRY=0`
- On Linux & Mac by typing `export TRAINER_TELEMETRY=0`
Before you start `finetune.py`. You will now be able to finetune offline and no anonymous training data will be sent.
#### ⚫ Important requirements CUDA 11.8
As mentioned you must have a small portion of the Nvidia CUDA Toolkit **11.8** installed. Not higher or lower versions. Specifically **11.8**. You do not have to uninstall any other versions, change any graphics drivers, reinstall torch or anything like that. This requirement is for Step1 of Finetuning. To keep the download+install as small as possible, you will need to:
- Download the **xxx (network)** install of the Nvidia CUDA Toolkit 11.8 from [here](https://developer.nvidia.com/cuda-11-8-0-download-archive)
@@ -455,6 +504,19 @@ As mentioned you must have a small portion of the Nvidia CUDA Toolkit **11.8** i
11) Follow the on-screen instructions when the web interface starts up.
12) When you have finished finetuning, the final tab will tell you what to do with your files and how to move your newly trained model to the correct location on disk.
#### ⚫ Evaluation Data Percentage
In the process of finetuning, it's crucial to balance the data used for training the model against the data reserved for evaluating its performance. Typically, a portion of the dataset is set aside as an 'evaluation set' to assess the model's capabilities in dealing with unseen data. On Step 1 of finetuning you have the option to adjust this evaluation data percentage, offering more control over your model training process.<br><br>
**Why Adjust the Evaluation Percentage?**<br><br>
Adjusting the evaluation percentage **can** be beneficial in scenarios with limited voice samples. When dealing with a smaller dataset, allocating a slightly larger portion to training could enhance the model's ability to learn from these scarce samples. Conversely, with abundant data, a higher evaluation percentage might be more appropriate to rigorously test the model's performance. There are currently no absolutely optimal split percentages as it varies by dataset.
- **Default Setting:** The default evaluation percentage is set at 15%, which is a balanced choice for most datasets.
- **Adjustable Range:** Users can now adjust this percentage, but it’s generally recommend keeping it between 5% and 30%.
- **Lower Bound:** A minimum of 5% ensures that there's enough data to evaluate model performance.
- **Upper Bound:** Its suggested not exceeding 30% for evaluation to avoid limiting the amount of data available for training.
- **Understanding the Impact:** Before adjusting this setting, it's important to understand its impact on model training and evaluation. Incorrect adjustments can lead to suboptimal model performance.
- **Gradual Adjustments:** For those unfamiliar with the process, we recommend reading up on training data and training sets, then making small, incremental changes and observing their effects.
- **Data Quality:** Regardless of the split, the quality of the audio data is paramount. Ensure that your datasets are built from good quality audio with enough data within them.
#### ⚫ Using a Finetuned model in Text-generation-webui
At the end of the finetune process, you will have an option to `Compact and move model to /trainedmodel/` this will compact the raw training file and move it to `/model/trainedmodel/`. When AllTalk starts up within Text-generation-webui, if it finds a model in this location a new loader will appear in the interface for `XTTSv2 FT` and you can use this to load your finetuned model. <br><br>**Be careful** not to train a new model from the base model, then overwrite your current `/model/trainedmodel/` **if** you want a seperately trained model. This is why there is an `OPTION B` to move your just trained model to `/models/lastfinetuned/`.
@@ -467,7 +529,7 @@ At the end of the finetune process, you will have an option to `Compact and move
If you've compacted and moved your model, its highly unlikely you would want to keep that data, however the choice is there to keep it if you wish. It will be between 5-10GB in size, so most people will want to delete it.
## 🔵🟢🟡 DeepSpeed Installation Options
### 🔵🟢🟡 DeepSpeed Installation Options
**NOTE**: You **DO NOT** need to set Text-generation-webUI's **--deepspeed** setting for AllTalk to be able to use DeepSpeed. These are two completely separate things and incorrectly setting that on Text-generation-webUI may cause other complications.
#### 🔵 Linux Installation
@@ -645,7 +707,7 @@ def get_shm_size():
</details>
</details>
## ⬜ AllTalk TTS Generator
### ⬜ AllTalk TTS Generator
AllTalk TTS Generator is the solution for converting large volumes of text into speech using the voice of your choice. Whether you're creating audio content or just want to hear text read aloud, the TTS Generator is equipped to handle it all efficiently. Please see here for a quick [demo](https://www.youtube.com/watch?v=hunvXn0mLzc)<br><br>The link to open the TTS generator can be found on the built-in Settings and Documentation page.<br><br>**DeepSpeed** is **highly** recommended to speed up generation. **Low VRAM** would be best turned off and your LLM model unloaded from your GPU VRAM (unload your model). **No Playback** will reduce memory overhead on very large generations (15,000 words or more). Splitting **Export to Wav** into smaller groups will also reduce memory overhead at the point of exporting your wav files (so good for low memory systems).
#### ⬜ Estimated Throughput
@@ -682,7 +744,7 @@ For generating **large amounts of TTS**, it's recommended to select the **No Pla
- Generated audio can be played back from the list, which also highlights the currently playing chunk.
- The TTS Generator remembers your settings, so you can pick up where you left off even after refreshing the page.
## 🟠 API Suite and JSON-CURL
### 🟠 API Suite and JSON-CURL
### 🟠Overview
The Text-to-Speech (TTS) Generation API allows you to generate speech from text input using various configuration options. This API supports both character and narrator voices, providing flexibility for creating dynamic and engaging audio content.
@@ -765,6 +827,7 @@ Generate a preview of a specified voice with hardcoded settings.
JSON return `{"status": "lowvram-success"}`
### 🟠 TTS Generation Endpoint (Standard Generation)
Streaming endpoint details are further down the page.
- URL: `http://127.0.0.1:7851/api/tts-generate`<br> - Method: `POST`<br> - Content-Type: `application/x-www-form-urlencoded`<br>
@@ -872,7 +935,7 @@ Example JSON TTS Generation Response:
`{"status":"generate-success","output_file_path":"C:\\text-generation-webui\\extensions\\alltalk_tts\\outputs\\myoutputfile_1704141936.wav","output_file_url":"http://127.0.0.1:7851/audio/myoutputfile_1704141936.wav","output_cache_url":"http://127.0.0.1:7851/audiocache/myoutputfile_1704141936.wav"}`
### 🟠 Example command lines (Streaming Generation)
### 🟠 TTS Generation Endpoint (Streaming Generation)
Streaming TTS generation does NOT support Narration and will generate an audio stream. Streaming TTS speech JavaScript Example:<br>
- URL: `http://localhost:7851/api/tts-generate-streaming`<br> - Method: `POST`<br> - Content-Type: `application/x-www-form-urlencoded`<br><br>
@@ -898,6 +961,6 @@ audioElement.play(); // Play the audio stream directly
<br>
### 🔴 Future to-do list
- I am maintaining a list of things people request [here](https://github.com/erew123/alltalk_tts/discussions/74)
- Possibly add some additional TTS engines (TBD).
- Tagging support within text.
- Have a break!
atsetup.bat
+3 -2
View File
@@ -341,6 +341,7 @@ if "%UserChoice%" == "1" (
pip install fsspec>=2023.6.0
pip install soundfile==0.12.1
pip install uvicorn==0.24.0.post1
pip install transformers==4.36.2
pip install TTS==0.21.3
pip install fastapi==0.104.1
pip install Jinja2==3.1.2
@@ -351,6 +352,8 @@ if "%UserChoice%" == "1" (
pip install pydantic==1.10.13
pip install sounddevice==0.4.6
pip install python-multipart==0.0.6
pip install cutlet>=0.3.0
pip install unidic-lite>=1.0.8
echo Downloading DeepSpeed...
curl -LO https://github.com/erew123/alltalk_tts/releases/download/DeepSpeed-12.7/deepspeed-0.12.7+d058d4b-cp311-cp311-win_amd64.whl
echo Installing DeepSpeed...
@@ -367,7 +370,6 @@ echo cd /D "%~dp0" >> start_environment.bat
echo set CONDA_ROOT_PREFIX=%cd%\alltalk_environment\conda >> start_environment.bat
echo set INSTALL_ENV_DIR=%cd%\alltalk_environment\env >> start_environment.bat
echo call "%CONDA_ROOT_PREFIX%\condabin\conda.bat" activate "%INSTALL_ENV_DIR%" >> start_environment.bat
echo exit >> start_environment.bat
@rem Create start_alltalk.bat to run AllTalk
echo @echo off > start_alltalk.bat
echo cd /D "%~dp0" >> start_alltalk.bat
@@ -375,7 +377,6 @@ echo set CONDA_ROOT_PREFIX=%cd%\alltalk_environment\conda >> start_alltalk.bat
echo set INSTALL_ENV_DIR=%cd%\alltalk_environment\env >> start_alltalk.bat
echo call "%CONDA_ROOT_PREFIX%\condabin\conda.bat" activate "%INSTALL_ENV_DIR%" >> start_alltalk.bat
echo call python script.py >> start_alltalk.bat
echo exit >> start_alltalk.bat
Echo.
Echo start_alltalk.bat has been created.
Echo You can now start AllTalk with %L_YELLOW%start_alltalk.bat%RESET%
atsetup.sh
+3
View File
@@ -301,6 +301,7 @@ install_custom_standalone() {
pip install fsspec>=2023.6.0
pip install soundfile==0.12.1
pip install uvicorn==0.24.0.post1
pip install transformers==4.36.2
pip install TTS==0.21.3
pip install fastapi==0.104.1
pip install Jinja2==3.1.2
@@ -311,6 +312,8 @@ install_custom_standalone() {
pip install pydantic==1.10.13
pip install sounddevice==0.4.6
pip install python-multipart==0.0.6
pip install cutlet>=0.3.0
pip install unidic-lite>=1.0.8
else
pip install -r requirements_other.txt
fi
finetune.py
+14 -4
View File
@@ -75,10 +75,10 @@ def create_temporary_file(folder, suffix=".wav"):
unique_filename = f"custom_tempfile_{int(time.time())}_{random.randint(1, 1000)}{suffix}"
return os.path.join(folder, unique_filename)
def format_audio_list(target_language, whisper_model, out_path, gradio_progress=progress):
def format_audio_list(target_language, whisper_model, out_path, eval_split_number, gradio_progress=progress):
audio_files = [os.path.join(audio_folder, file) for file in os.listdir(audio_folder) if file.endswith(('.mp3', '.flac', '.wav'))]
buffer=0.2
eval_percentage=0.15
eval_percentage = eval_split_number / 100.0
speaker_name="coqui"
audio_total_size = 0
os.makedirs(out_path, exist_ok=True)
@@ -87,6 +87,7 @@ def format_audio_list(target_language, whisper_model, out_path, gradio_progress=
print("[FINETUNE] \033[94mPart of AllTalk\033[0m https://github.com/erew123/alltalk_tts/")
print("[FINETUNE] \033[94mCoqui Public Model License\033[0m")
print("[FINETUNE] \033[94mhttps://coqui.ai/cpml.txt\033[0m")
print(f"[FINETUNE] \033[94mWhisper model: \033[92m{whisper_model} \033[94mLanguage: \033[92m{target_language} \033[94mEvaluation data percentage: \033[92m{eval_split_number}%\033[0m")
print("[FINETUNE] \033[94mStarting Step 1\033[0m - Preparing Audio/Generating the dataset")
# Write the target language to lang.txt in the output directory
lang_file_path = os.path.join(out_path, "lang.txt")
@@ -968,6 +969,14 @@ if __name__ == "__main__":
"ja"
],
)
eval_split_number = gr.Number(
label="Evaluation data Split (the % to use for Evaluation data)",
value=15, # Default value
minimum=5, # Minimum value
maximum=95, # Maximum value
step=1, # Increment step
)
progress_data = gr.Label(
label="Progress:"
)
@@ -979,14 +988,14 @@ if __name__ == "__main__":
prompt_compute_btn = gr.Button(value="Step 1 - Create dataset")
def preprocess_dataset(language, whisper_model, out_path, progress=gr.Progress(track_tqdm=True)):
def preprocess_dataset(language, whisper_model, out_path, eval_split_number, progress=gr.Progress(track_tqdm=True)):
clear_gpu_cache()
test_for_audio_files = [file for file in os.listdir(audio_folder) if any(file.lower().endswith(ext) for ext in ['.wav', '.mp3', '.flac'])]
if not test_for_audio_files:
return "I cannot find any mp3, wav or flac files in the folder called 'put-voice-samples-in-here'", "", ""
else:
try:
train_meta, eval_meta, audio_total_size = format_audio_list(target_language=language, whisper_model=whisper_model ,out_path=out_path, gradio_progress=progress)
train_meta, eval_meta, audio_total_size = format_audio_list(target_language=language, whisper_model=whisper_model, out_path=out_path, eval_split_number=eval_split_number, gradio_progress=progress)
except:
traceback.print_exc()
error = traceback.format_exc()
@@ -1274,6 +1283,7 @@ if __name__ == "__main__":
lang,
whisper_model,
out_path,
eval_split_number,
],
outputs=[
progress_data,
requirements_nvidia.txt
+3
View File
@@ -1,6 +1,7 @@
numpy>=1.22.0
soundfile>=0.12.1
uvicorn>=0.24.0.post1
transformers==4.36.2
TTS>=0.21.3
torch>=2.1.0+cu118
torchaudio>=2.1.0+cu118
@@ -13,3 +14,5 @@ packaging>=23.2
pydantic>=1.10.13
sounddevice>=0.4.6
python-multipart>=0.0.6
cutlet>=0.3.0
unidic-lite>=1.0.8
requirements_other.txt
+3
View File
@@ -1,6 +1,7 @@
numpy>=1.22.0
soundfile>=0.12.1
uvicorn>=0.24.0.post1
transformers==4.36.2
TTS>=0.21.3
torch>=2.1.0
torchaudio>=2.1.0
@@ -13,3 +14,5 @@ packaging>=23.2
pydantic>=1.10.13
sounddevice>=0.4.6
python-multipart>=0.0.6
cutlet>=0.3.0
unidic-lite>=1.0.8
script.py
+43 -22
View File
@@ -16,6 +16,14 @@ import numpy as np
import soundfile as sf
import uuid
######################################
#### ALLTALK ALLOWED STARTUP TIME ####
######################################
startup_wait_time = 120
# You can change the above setting to a larger number to allow AllTAlk more time to start up. The default setting is 120 seconds (2 minutes).
# On some older systems you may need to allow AllTalk more time. So you could set it to 240 (4 minutes) which will give AllTalk more to load.
#################################################################
#### LOAD PARAMS FROM confignew.json - REQUIRED FOR BRANDING ####
#################################################################
@@ -239,7 +247,7 @@ if process.poll() is None:
print(f"[{params['branding']}Startup] TTS Subprocess starting")
print(f"[{params['branding']}Startup]")
print(
f"[{params['branding']}Startup] \033[94mSettings & Documentation:\033[00m",
f"[{params['branding']}Startup] \033[94m {params['branding']}Settings & Documentation:\033[00m",
f"\033[92mhttp://{params['ip_address']}:{params['port_number']}\033[00m",
)
print(f"[{params['branding']}Startup]")
@@ -266,7 +274,7 @@ else:
# Cleanly kill off this script, but allow text-generation-webui to keep running, albeit without this alltalk_tts
sys.exit(1)
timeout = 120 # Adjust the timeout as needed
timeout = startup_wait_time # Gather timeout setting from startup_wait_time
# Introduce a delay before starting the check loop
time.sleep(26) # Wait 26 secs before checking if the tts_server.py has started up.
@@ -279,13 +287,15 @@ while time.time() - start_time < timeout:
except requests.RequestException as e:
# Print the exception for debugging purposes
print(
f"[{params['branding']}Startup] \033[91mWarning\033[0m TTS Subprocess has NOT started up yet, Will keep trying for 120 seconds maximum. Please wait."
f"[{params['branding']}Startup] \033[91mWarning\033[0m TTS Subprocess has NOT started up yet, Will keep trying for {timeout} seconds maximum. Please wait."
)
time.sleep(4)
time.sleep(5)
else:
print(
f"[{params['branding']}Startup] Startup timed out. Check the server logs for more information."
)
print(f"\n[{params['branding']}Startup] Startup timed out. Full help available here \033[92mhttps://github.com/erew123/alltalk_tts#-help-with-problems\033[0m")
print(f"[{params['branding']}Startup] On older system you may wish to open and edit \033[94mscript.py\033[0m with a text editor and changing the")
print(f"[{params['branding']}Startup] \033[94mstartup_wait_time = 120\033[0m setting to something like \033[94mstartup_wait_time = 240\033[0m as this will allow")
print(f"[{params['branding']}Startup] AllTalk more time to try load the model into your VRAM. Otherise please visit the Github for")
print(f"[{params['branding']}Startup] a list of other possible troubleshooting options.")
# Cleanly kill off this script, but allow text-generation-webui to keep running, albeit without this alltalk_tts
sys.exit(1)
@@ -489,12 +499,14 @@ def voice_preview(string):
output_file = Path(params["output_folder_wav"]) / "voice_preview.wav"
# Generate the audio
language_code = languages.get(params["language"])
temperature = params["local_temperature"]
repetition_penalty = params["local_repetition_penalty"]
# Convert the WindowsPath object to a string before using it in JSON payload
output_file_str = output_file.as_posix()
# Lock before making the generate request
with process_lock:
generate_response = send_generate_request(
string, params["voice"], language_code, output_file_str
string, params["voice"], language_code, temperature, repetition_penalty, output_file_str
)
# Check if lock is already acquired
if process_lock.locked():
@@ -573,6 +585,8 @@ def output_modifier(string, state):
return
string = cleaned_string
language_code = languages.get(params["language"])
temperature = params["local_temperature"]
repetition_penalty = params["local_repetition_penalty"]
# Create a list to store generated audio paths
audio_files = []
if process_lock.acquire(blocking=False):
@@ -609,7 +623,7 @@ def output_modifier(string, state):
# Generate TTS and output to a file
output_filename = get_output_filename(state)
generate_response = send_generate_request(
cleaned_part, voice_to_use, language_code, output_filename
cleaned_part, voice_to_use, language_code, temperature, repetition_penalty, output_filename
)
audio_path = generate_response.get("data", {}).get("audio_path")
audio_files_all_paragraphs.append(audio_path)
@@ -635,7 +649,7 @@ def output_modifier(string, state):
output_file_str = output_file.as_posix()
output_file = get_output_filename(state)
generate_response = send_generate_request(
cleaned_part, params["voice"], language_code, output_file_str
cleaned_part, params["voice"], language_code, temperature, repetition_penalty, output_file_str
)
audio_path = generate_response.get("data", {}).get("audio_path")
final_output_file = audio_path
@@ -684,12 +698,14 @@ def get_output_filename(state):
###############################################
#### SEND GENERATION REQUEST TO TTS ENGINE ####
###############################################
def send_generate_request(text, voice, language, output_file):
def send_generate_request(text, voice, language, temperature, repetition_penalty, output_file):
url = f"{base_url}/api/generate"
payload = {
"text": text,
"voice": voice,
"language": language,
"temperature": temperature,
"repetition_penalty": repetition_penalty,
"output_file": output_file,
}
headers = {"Content-Type": "application/json"}
@@ -737,11 +753,13 @@ def input_modifier(string, state):
def ui():
with gr.Accordion(params["branding"] + " TTS (XTTSv2)"):
# Activate alltalk_tts, Enable autoplay, Show text
with gr.Row():
activate = gr.Checkbox(value=params['activate'], label='Enable TTS')
autoplay = gr.Checkbox(value=params['autoplay'], label='Autoplay TTS')
show_text = gr.Checkbox(value=params['show_text'], label='Show Text')
# Low vram enable, Deepspeed enable, Remove trailing dots
with gr.Row():
low_vram = gr.Checkbox(value=params['low_vram'], label='Enable Low VRAM Mode')
low_vram_play = gr.HTML(visible=False)
@@ -749,6 +767,7 @@ def ui():
deepspeed_checkbox_play = gr.HTML(visible=False)
remove_trailing_dots = gr.Checkbox(value=params['remove_trailing_dots'], label='Remove trailing "."')
# TTS method, Character voice selection
with gr.Row():
model_loader_choices=["API TTS", "API Local", "XTTSv2 Local"]
if finetuned_model:
@@ -769,17 +788,19 @@ def ui():
voice = gr.Dropdown(available_voices, label="Character Voice", value=default_voice, allow_custom_value=True, )
create_refresh_button(voice, lambda: None, lambda: {"choices": get_available_voices(), "value": params["voice"],}, "refresh-button",)
# Language, Narrator voice
with gr.Row():
language = gr.Dropdown(languages.keys(), label="Language", value=params["language"])
with gr.Row():
narrator_voice_gr = gr.Dropdown(get_available_voices(), label="Narrator Voice", allow_custom_value=True, value=params["narrator_voice"])
create_refresh_button(narrator_voice_gr, lambda: None, lambda: {"choices": get_available_voices(), "value": params["narrator_voice"],}, "refresh-button")
# Temperature and Repetition Penalty (Not yet parsed to api/implemented)
# with gr.Row():
# local_temperature_gr = gr.Slider(minimum=0, maximum=1, step=0.05, label="Temperature", value=params["local_temperature"])
# local_repetition_penalty_gr = gr.Slider(minimum=0, maximum=20, step=0.5, label="Repetition Penalty", value=params["local_repetition_penalty"])
# Temperature, Repetition Penalty
with gr.Row():
local_temperature_gr = gr.Slider(minimum=0.05, maximum=1, step=0.05, label="Temperature", value=params["local_temperature"])
local_repetition_penalty_gr = gr.Slider(minimum=0.5, maximum=20, step=0.5, label="Repetition Penalty", value=params["local_repetition_penalty"])
# Narrator enable, Non quoted text, Explanation text
with gr.Row():
with gr.Row():
narrator_enabled_gr = gr.Radio(choices={"Enabled": "true", "Disabled": "false"}, label="Narrator", value="Enabled" if params.get("narrator_enabled") else "Disabled")
@@ -787,6 +808,7 @@ def ui():
explanation_text = gr.HTML(
f"<p>⚙️ <a href='http://{params['ip_address']}:{params['port_number']}'>Settings and Documentation Page</a><a href='http://{params['ip_address']}:{params['port_number']}'></a>⚙️<br>- Low VRAM Mode and Deepspeed take 15 seconds to be enabled or disabled.<br>- The DeepSpeed checkbox is only visible if DeepSpeed is present.</p>")
# Preview speech
with gr.Row():
preview_text = gr.Text(
show_label=False,
@@ -795,7 +817,7 @@ def ui():
)
preview_play = gr.Button("Preview")
preview_audio = gr.HTML(visible=False)
with gr.Row():
convert = gr.Button("Permanently replace audios with the message texts")
convert_cancel = gr.Button("Cancel", visible=False)
@@ -864,10 +886,9 @@ def ui():
voice.change(lambda x: params.update({"voice": x}), voice, None)
language.change(lambda x: params.update({"language": x}), language, None)
# TS Settings (Not yet parsed to api/implemented)
# local_temperature_gr.change(lambda x: params.update({"local_temperature": x}), local_temperature_gr, None)
# local_repetition_penalty_gr.change(lambda x: params.update({"local_repetition_penalty": x}), local_repetition_penalty_gr, None)
# TSS Settings
local_temperature_gr.change(lambda x: params.update({"local_temperature": x}), local_temperature_gr, None)
local_repetition_penalty_gr.change(lambda x: params.update({"local_repetition_penalty": x}), local_repetition_penalty_gr, None)
# Narrator selection actions
narrator_enabled_gr.change(update_narrator_enabled, narrator_enabled_gr, None)
@@ -888,4 +909,4 @@ if running_in_standalone:
try:
time.sleep(1) # Add a small delay to avoid high CPU usage
except KeyboardInterrupt:
break # Allow graceful exit on Ctrl+C
break # Allow graceful exit on Ctrl+C
templates/STfiles/alltalk.js
+105 -210
View File
@@ -3,92 +3,33 @@ import { saveTtsProviderSettings } from './index.js';
export { AllTalkTtsProvider };
const css = `
.settings-row {
display: flex;
justify-content: space-between; /* Distribute space evenly between the children */
align-items: center;
width: 100%; /* Full width to spread out the children */
}
.settings-option {
flex: 1; /* Each child will take equal space */
margin: 0 10px; /* Adjust spacing as needed */
}
.endpoint-option {
flex: 1; /* Each child will take equal space */
margin: 0 10px; /* Adjust spacing as needed */
margin-right: 25px;
width: 38%; /* Full width to spread out the children */
}
.website-row {
display: flex;
justify-content: start; /* Aligns items to the start of the row */
align-items: center;
margin-top: 10px; /* Top margin */
margin-bottom: 10px; /* Bottom margin */
}
.website-option {
flex: 1; /* Allows each option to take equal space */
margin-right: 10px; /* Spacing between the two options, adjust as needed */
margin-left: 10px; /* Adjust this value to align as desired */
/* Remove the individual margin top and bottom declarations if they are set globally on the .settings-row */
}
.settings-separator {
margin-top: 10px;
margin-bottom: 10px;
padding: 18x;
font-weight: bold;
border-top: 1px solid #e1e1e1; /* Grey line */
border-bottom: 1px solid #e1e1e1; /* Grey line */
text-align: center;
}
.status-message {
flex: 1; /* Adjust as needed */
margin: 0 10px;
/* Additional styling */
}
.model-endpoint-row {
display: flex;
justify-content: space-between;
align-items: center;
width: 100%;
}
.model-option, .endpoint-option {
flex: 1;
margin: 0 10px;
margin-left: 10px; /* Adjust this value to align as desired */
/* Adjust width as needed to fit in one line */
}
.endpoint-option {
/* Adjust width to align with model dropdown */
width: 38%;
}
#status_info {
color: lightgreen; /* Or any specific shade of green you prefer */
}
`;
const style = document.createElement('style');
style.type = 'text/css';
style.appendChild(document.createTextNode(css));
document.head.appendChild(style);
class AllTalkTtsProvider {
//########//
// Config //
//########//
settings;
settings = {};
constructor() {
// Initialize with default settings if they are not already set
this.settings = {
provider_endpoint: this.settings.provider_endpoint || 'http://localhost:7851',
language: this.settings.language || 'en',
voiceMap: this.settings.voiceMap || {},
at_generation_method: this.settings.at_generation_method || 'standard_generation',
narrator_enabled: this.settings.narrator_enabled || 'false',
at_narrator_text_not_inside: this.settings.at_narrator_text_not_inside || 'narrator',
narrator_voice_gen: this.settings.narrator_voice_gen || 'female_01.wav',
finetuned_model: this.settings.finetuned_model || 'false'
};
// Separate property for dynamically updated settings from the server
this.dynamicSettings = {
modelsAvailable: [],
currentModel: '',
deepspeed_available: false,
deepSpeedEnabled: false,
lowVramEnabled: false,
};
}
ready = false;
voices = [];
separator = '. ';
@@ -114,23 +55,12 @@ class AllTalkTtsProvider {
'Hindi': 'hi',
};
Settings = {
provider_endpoint: 'http://localhost:7851',
language: 'en',
voiceMap: {},
at_generation_method: 'standard_generation',
narrator_enabled: 'false',
at_narrator_text_not_inside: 'narrator',
narrator_voice_gen: 'female_01.wav',
finetuned_model: 'false'
};
get settingsHtml() {
let html = `<div class="settings-separator">AllTalk Settings</div>`;
let html = `<div class="at-settings-separator">AllTalk Settings</div>`;
html += `<div class="settings-row">
html += `<div class="at-settings-row">
<div class="settings-option">
<div class="at-settings-option">
<label for="at_generation_method">AllTalk TTS Generation Method</label>
<select id="at_generation_method">
<option value="standard_generation">Standard Audio Generation (AT Narrator - Optional)</option>
@@ -139,9 +69,9 @@ class AllTalkTtsProvider {
</div>
</div>`;
html += `<div class="settings-row">
html += `<div class="at-settings-row">
<div class="settings-option">
<div class="at-settings-option">
<label for="at_narrator_enabled">AT Narrator</label>
<select id="at_narrator_enabled">
<option value="true">Enabled</option>
@@ -149,7 +79,7 @@ class AllTalkTtsProvider {
</select>
</div>
<div class="settings-option">
<div class="at-settings-option">
<label for="at_narrator_text_not_inside">Text Not Inside * or " is</label>
<select id="at_narrator_text_not_inside">
<option value="narrator">Narrator</option>
@@ -159,8 +89,8 @@ class AllTalkTtsProvider {
</div>`;
html += `<div class="settings-row">
<div class="settings-option">
html += `<div class="at-settings-row">
<div class="at-settings-option">
<label for="narrator_voice">Narrator Voice</label>
<select id="narrator_voice">`;
if (this.voices) {
@@ -170,7 +100,7 @@ class AllTalkTtsProvider {
}
html += `</select>
</div>
<div class="settings-option">
<div class="at-settings-option">
<label for="language_options">Language</label>
<select id="language_options">`;
for (let language in this.languageLabels) {
@@ -181,8 +111,8 @@ class AllTalkTtsProvider {
</div>`;
html += `<div class="model-endpoint-row">
<div class="model-option">
html += `<div class="at-model-endpoint-row">
<div class="at-model-option">
<label for="switch_model">Switch Model</label>
<select id="switch_model">
<option value="api_tts">API TTS</option>
@@ -191,49 +121,47 @@ class AllTalkTtsProvider {
</select>
</div>
<div class="endpoint-option">
<div class="at-endpoint-option">
<label for="at_server">AllTalk Endpoint:</label>
<input id="at_server" type="text" class="text_pole" maxlength="80" value="${this.Settings.provider_endpoint}"/>
<input id="at_server" type="text" class="text_pole" maxlength="80" value="${this.settings.provider_endpoint}"/>
</div>
</div>`;
html += `<div class="model-endpoint-row">
<div class="settings-option">
html += `<div class="at-model-endpoint-row">
<div class="at-settings-option">
<label for="low_vram">Low VRAM</label>
<input id="low_vram" type="checkbox"/>
</div>
<div class="settings-option">
<div class="at-settings-option">
<label for="deepspeed">DeepSpeed</label>
<input id="deepspeed" type="checkbox"/>
</div>
<div class="settings-option status-option">
<div class="at-settings-option status-option">
<span>Status: <span id="status_info">Ready</span></span>
</div>
<div class="settings-option empty-option">
<div class="at-settings-option empty-option">
<!-- This div remains empty for spacing -->
</div>
</div>`;
html += `<div class="website-row">
<div class="website-option">
<span>AllTalk <a target="_blank" href="${this.Settings.provider_endpoint}">Config & Docs</a>.</span>
html += `<div class="at-website-row">
<div class="at-website-option">
<span>AllTalk <a target="_blank" href="${this.settings.provider_endpoint}">Config & Docs</a>.</span>
</div>
<div class="website-option">
<div class="at-website-option">
<span>AllTalk <a target="_blank" href="https://github.com/erew123/alltalk_tts/">Website</a>.</span>
</div>
</div>`;
html += `<div class="website-row">
<div class="website-option">
html += `<div class="at-website-row">
<div class="at-website-option">
<span><strong>Text-generation-webui</strong> users - Uncheck <strong>Enable TTS</strong> in Text-generation-webui.</span>
</div>
</div>`;
return html;
}
@@ -244,14 +172,35 @@ html += `<div class="website-row">
async loadSettings(settings) {
updateStatus('Offline');
// Use default settings if no settings are provided
this.settings = Object.keys(settings).length === 0 ? this.Settings : settings;
if (Object.keys(settings).length === 0) {
console.info('Using default AllTalk TTS Provider settings');
} else {
// Populate settings with provided values, ignoring server-provided settings
for (const key in settings) {
if (key in this.settings) {
this.settings[key] = settings[key];
} else {
console.debug(`Ignoring non-user-configurable setting: ${key}`);
}
}
}
// Update UI elements to reflect the loaded settings
$('#at_server').val(this.settings.provider_endpoint);
$('#language_options').val(this.settings.language);
//$('#voicemap').val(this.settings.voiceMap);
$('#at_generation_method').val(this.settings.at_generation_method);
$('#at_narrator_enabled').val(this.settings.narrator_enabled);
$('#at_narrator_text_not_inside').val(this.settings.at_narrator_text_not_inside);
$('#narrator_voice').val(this.settings.narrator_voice_gen);
console.debug('AllTalkTTS: Settings loaded');
try {
// Check if TTS provider is ready
await this.checkReady();
await this.updateSettingsFromServer(); // Fetch dynamic settings from the TTS server
await this.fetchTtsVoiceObjects(); // Fetch voices only if service is ready
await this.updateSettingsFromServer();
this.updateNarratorVoicesDropdown();
this.updateLanguageDropdown();
this.setupEventListeners();
@@ -259,29 +208,23 @@ html += `<div class="website-row">
updateStatus('Ready');
} catch (error) {
console.error("Error loading settings:", error);
// Set status to Error if there is an issue in loading settings
updateStatus('Offline');
}
}
applySettingsToHTML() {
// Load AllTalk specific settings first
const loadedSettings = loadAllTalkSettings();
// Apply loaded settings or use defaults
this.settings = loadedSettings ? { ...this.Settings, ...loadedSettings } : this.Settings;
const narratorVoiceSelect = document.getElementById('narrator_voice');
const atNarratorSelect = document.getElementById('at_narrator_enabled');
const textNotInsideSelect = document.getElementById('at_narrator_text_not_inside');
const generationMethodSelect = document.getElementById('at_generation_method');
this.settings.narrator_voice = this.settings.narrator_voice_gen;
// Apply settings to Narrator Voice dropdown
if (narratorVoiceSelect && this.settings.narrator_voice) {
narratorVoiceSelect.value = this.settings.narrator_voice.replace('.wav', '');
this.settings.narrator_voice_gen = this.settings.narrator_voice;
//console.log(this.settings.narrator_voice_gen)
}
// Apply settings to AT Narrator Enabled dropdown
if (atNarratorSelect) {
//console.log(this.settings.narrator_enabled)
// Sync the state with the checkbox in index.js
const ttsPassAsterisksCheckbox = document.getElementById('tts_pass_asterisks'); // Access the checkbox from index.js
const ttsNarrateQuotedCheckbox = document.getElementById('tts_narrate_quoted'); // Access the checkbox from index.js
@@ -314,18 +257,15 @@ html += `<div class="website-row">
const languageSelect = document.getElementById('language_options');
if (languageSelect && this.settings.language) {
languageSelect.value = this.settings.language;
//console.log(this.settings.language)
}
// Apply settings to Text Not Inside dropdown
if (textNotInsideSelect && this.settings.text_not_inside) {
textNotInsideSelect.value = this.settings.text_not_inside;
this.settings.at_narrator_text_not_inside = this.settings.text_not_inside;
//console.log(this.settings.at_narrator_text_not_inside)
}
// Apply settings to Generation Method dropdown
if (generationMethodSelect && this.settings.at_generation_method) {
generationMethodSelect.value = this.settings.at_generation_method;
//console.log(this.settings.at_generation_method)
}
// Additional logic to disable/enable dropdowns based on the selected generation method
const isStreamingEnabled = this.settings.at_generation_method === 'streaming_enabled';
@@ -347,7 +287,6 @@ html += `<div class="website-row">
ftOption.textContent = 'XTTSv2 FT';
modelSelect.appendChild(ftOption);
}
//console.log("Settings applied to HTML.");
}
//##############################//
@@ -486,7 +425,7 @@ html += `<div class="website-row">
const languageSelect = document.getElementById('language_options');
if (languageSelect) {
// Ensure default language is set
this.settings.language = this.Settings.language;
this.settings.language = this.settings.language;
languageSelect.innerHTML = '';
for (let language in this.languageLabels) {
@@ -610,7 +549,7 @@ html += `<div class="website-row">
if (narratorVoiceSelect) {
narratorVoiceSelect.addEventListener('change', (event) => {
this.settings.narrator_voice_gen = `${event.target.value}.wav`;
this.onSettingsChangeAllTalk(); // Save the settings after change
this.onSettingsChange(); // Save the settings after change
});
}
@@ -618,7 +557,7 @@ html += `<div class="website-row">
if (textNotInsideSelect) {
textNotInsideSelect.addEventListener('change', (event) => {
this.settings.text_not_inside = event.target.value;
this.onSettingsChangeAllTalk(); // Save the settings after change
this.onSettingsChange(); // Save the settings after change
});
}
@@ -656,12 +595,11 @@ html += `<div class="website-row">
$('#tts_narrate_dialogues').click();
$('#tts_narrate_dialogues').trigger('change');
}
this.onSettingsChangeAllTalk(); // Save the settings after change
this.onSettingsChange(); // Save the settings after change
});
}
// Event Listener for AT Generation Method Dropdown
const atGenerationMethodSelect = document.getElementById('at_generation_method');
const atNarratorEnabledSelect = document.getElementById('at_narrator_enabled');
@@ -680,7 +618,7 @@ html += `<div class="website-row">
atNarratorEnabledSelect.disabled = false;
}
this.settings.at_generation_method = selectedMethod; // Update the setting here
this.onSettingsChangeAllTalk(); // Save the settings after change
this.onSettingsChange(); // Save the settings after change
});
}
@@ -689,9 +627,19 @@ html += `<div class="website-row">
if (languageSelect) {
languageSelect.addEventListener('change', (event) => {
this.settings.language = event.target.value;
this.onSettingsChangeAllTalk(); // Save the settings after change
this.onSettingsChange(); // Save the settings after change
});
}
// Listener for AllTalk Endpoint Input
const atServerInput = document.getElementById('at_server');
if (atServerInput) {
atServerInput.addEventListener('input', (event) => {
this.settings.provider_endpoint = event.target.value;
this.onSettingsChange(); // Save the settings after change
});
}
}
//#############################//
@@ -699,30 +647,16 @@ html += `<div class="website-row">
//#############################//
onSettingsChange() {
// Update settings that SillyTavern will save
this.settings.provider_endpoint = $('#at_server').val();
// Update settings based on the UI elements
//this.settings.provider_endpoint = $('#at_server').val();
this.settings.language = $('#language_options').val();
saveTtsProviderSettings(); // This function should save settings handled by SillyTavern
// Call the function to handle AllTalk specific settings
this.onSettingsChangeAllTalk(); // Save the settings after change
}
onSettingsChangeAllTalk() {
// Update AllTalk specific settings and save to localStorage
this.settings.narrator_enabled = $('#at_narrator_enabled').val() === 'true';
this.settings.narrator_voice = $('#narrator_voice').val() + '.wav'; // assuming you need to append .wav
this.settings.text_not_inside = $('#at_narrator_text_not_inside').val(); // "character" or "narrator"
this.settings.at_generation_method = $('#at_generation_method').val(); // Streaming or standard
this.settings.language = $('#language_options').val(); // Streaming or standard
// Call the save function with the current settings
saveAllTalkSettings({
narrator_voice: this.settings.narrator_voice,
narrator_enabled: this.settings.narrator_enabled,
text_not_inside: this.settings.text_not_inside,
at_generation_method: this.settings.at_generation_method,
language: this.settings.language
});
//this.settings.voiceMap = $('#voicemap').val();
this.settings.at_generation_method = $('#at_generation_method').val();
this.settings.narrator_enabled = $('#at_narrator_enabled').val();
this.settings.at_narrator_text_not_inside = $('#at_narrator_text_not_inside').val();
this.settings.narrator_voice_gen = $('#narrator_voice').val();
// Save the updated settings
saveTtsProviderSettings();
}
//#########################//
@@ -802,38 +736,20 @@ html += `<div class="website-row">
async generateTts(inputText, voiceId) {
try {
if (this.settings.at_generation_method === 'streaming_enabled') {
// For streaming method
// Construct the streaming URL
const streamingUrl = `${this.settings.provider_endpoint}/api/tts-generate-streaming?text=${encodeURIComponent(inputText)}&voice=${encodeURIComponent(voiceId)}.wav&language=${encodeURIComponent(this.settings.language)}&output_file=stream_output.wav`;
const audioElement = new Audio(streamingUrl);
audioElement.play(); // Play the audio stream directly
console.log("Streaming URL:", streamingUrl);
return new Response(null, {
status: 200,
statusText: "OK",
headers: {
"Content-Type": "audio/wav",
"Content-Location": streamingUrl
}
});
// Return the streaming URL directly
return streamingUrl;
} else {
// For standard method
const outputUrl = await this.fetchTtsGeneration(inputText, voiceId);
// Fetch the audio data as a blob from the URL
const audioResponse = await fetch(outputUrl);
if (!audioResponse.ok) {
throw new Error(`HTTP ${audioResponse.status}: Failed to fetch audio data`);
}
const audioBlob = await audioResponse.blob();
if (!audioBlob.type.startsWith('audio/')) {
throw new Error(`Invalid audio data format. Expecting audio/*, got ${audioBlob.type}`);
}
return new Response(audioBlob, {
status: 200,
statusText: "OK",
headers: {
"Content-Type": audioBlob.type
}
});
return audioResponse; // Return the fetch response directly
}
} catch (error) {
console.error("Error in generateTts:", error);
@@ -841,6 +757,7 @@ html += `<div class="website-row">
}
}
//####################//
// Generate Standard //
//####################//
@@ -852,8 +769,8 @@ html += `<div class="website-row">
'text_filtering': "standard",
'character_voice_gen': voiceId + ".wav",
'narrator_enabled': this.settings.narrator_enabled,
'narrator_voice_gen': this.settings.narrator_voice_gen,
'text_not_inside': this.settings.text_not_inside,
'narrator_voice_gen': this.settings.narrator_voice_gen + ".wav",
'text_not_inside': this.settings.at_narrator_text_not_inside,
'language': this.settings.language,
'output_file_name': "st_output",
'output_file_timestamp': "true",
@@ -877,7 +794,6 @@ html += `<div class="website-row">
if (!response.ok) {
const errorText = await response.text();
console.error(`[fetchTtsGeneration] Error Response Text:`, errorText);
// Uncomment the following line if you have a UI element for displaying errors
// toastr.error(response.statusText, 'TTS Generation Failed');
throw new Error(`HTTP ${response.status}: ${errorText}`);
}
@@ -912,27 +828,6 @@ function updateStatus(message) {
case 'Error':
statusElement.style.color = 'red';
break;
// Add more cases as needed
}
}
}
//########################//
// Save/load AT Settings //
//########################//
function saveAllTalkSettings(settingsToSave) {
// Save the specific settings to localStorage
console.log("settings", settingsToSave)
localStorage.setItem('AllTalkSettings', JSON.stringify(settingsToSave));
}
function loadAllTalkSettings() {
// Retrieve the settings from localStorage
const settings = localStorage.getItem('AllTalkSettings');
// If settings exist, parse them back into an object
if (settings) {
return JSON.parse(settings);
}
return null;
}
}
templates/STfiles/index.js
+1046 -1028
View File
File diff suppressed because it is too large. Load diff
templates/STfiles/style.css
+159
View File
@@ -0,0 +1,159 @@
#tts_media_control {
/* order: 100; */
/* width: 40px;
height: 40px;
margin: 0;
padding: 1px; */
outline: none;
border: none;
cursor: pointer;
/* transition: 0.3s;
opacity: 0.7; */
display: flex;
align-items: center;
/* justify-content: center; */
}
#ttsExtensionMenuItem {
transition: 0.3s;
opacity: 0.7;
}
#ttsExtensionMenuItem:hover {
opacity: 1;
filter: brightness(1.2);
}
#tts_media_control:hover {
opacity: 1;
filter: brightness(1.2);
}
.voice_preview {
margin: 0.25rem 0.5rem;
display: flex;
justify-content: space-between;
align-items: center;
gap: 0.5rem;
}
.voice_preview .voice_name {
text-align: left;
flex: 1;
}
.voice_preview .voice_lang {
width: 4rem;
text-align: left;
}
.voice_preview .fa-play {
cursor: pointer;
}
.tts-button {
margin: 0;
outline: none;
border: none;
cursor: pointer;
transition: 0.3s;
opacity: 0.7;
align-items: center;
justify-content: center;
}
.tts-button:hover {
opacity: 1;
}
.tts_block {
display: flex;
align-items: baseline;
column-gap: 5px;
flex-wrap: wrap;
}
.tts_custom_voices {
display: flex;
align-items: baseline;
gap: 5px;
}
.novel_tts_hints {
font-size: calc(0.9 * var(--mainFontSize));
display: flex;
flex-direction: column;
gap: 5px;
margin-bottom: 5px;
}
.at-settings-row {
display: flex;
justify-content: space-between;
align-items: center;
width: 100%;
}
.at-settings-option {
flex: 1;
margin: 0 10px;
}
.at-endpoint-option {
flex: 1;
margin: 0 10px;
margin-right: 25px;
width: 38%;
}
.at-website-row {
display: flex;
justify-content: start;
align-items: center;
margin-top: 10px;
margin-bottom: 10px;
}
.at-website-option {
flex: 1;
margin-right: 10px;
margin-left: 10px;
}
.at-settings-separator {
margin-top: 10px;
margin-bottom: 10px;
padding: 18x;
font-weight: bold;
border-top: 1px solid #e1e1e1; /* Grey line */
border-bottom: 1px solid #e1e1e1; /* Grey line */
text-align: center;
}
.at-status-message {
flex: 1;
margin: 0 10px;
}
.at-model-endpoint-row {
display: flex;
justify-content: space-between;
align-items: center;
width: 100%;
}
.at-model-option, .endpoint-option {
flex: 1;
margin: 0 10px;
margin-left: 10px;
}
.at-endpoint-option {
width: 38%;
}
#at-status_info {
color: lightgreen;
}
templates/tts_chunk_player/tts_chunk_player.html
+95 -90
View File
@@ -6,7 +6,6 @@
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<link rel="icon" href="./favicon.ico" type="image/x-icon">
<title>AllTalk TTS Generator</title>
<style>
body,
textarea,
@@ -637,51 +636,51 @@
import Crunker from './crunker.esm.js';
async function exportMP3() {
const crunker = new Crunker();
const fileUrls = ttsList.map(item => item.fileUrl);
const splitEvery = parseInt(document.getElementById('splitEvery').value) || 500;
const progressIndicator = document.getElementById('progressIndicator');
const progressText = document.getElementById('progressText');
const crunker = new Crunker();
const fileUrls = ttsList.map(item => item.fileUrl);
const splitEvery = parseInt(document.getElementById('splitEvery').value) || 500;
const progressIndicator = document.getElementById('progressIndicator');
const progressText = document.getElementById('progressText');
if (progressIndicator && progressText) {
progressIndicator.style.display = 'block';
progressText.textContent = 'Starting export...';
}
try {
for (let i = 0; i < fileUrls.length; i += splitEvery) {
const currentBatch = Math.floor(i / splitEvery) + 1;
const totalBatches = Math.ceil(fileUrls.length / splitEvery);
if (progressText) {
progressText.textContent = `Processing batch ${currentBatch} of ${totalBatches}`;
progressText.style.color = 'black'; // Set the text color to black
}
const batchUrls = fileUrls.slice(i, Math.min(i + splitEvery, fileUrls.length));
const audioBuffers = await crunker.fetchAudio(...batchUrls);
const combined = crunker.concatAudio(audioBuffers);
const output = crunker.export(combined, "audio/wav");
crunker.download(output.blob, `combined_audio_part_${currentBatch}.wav`);
// Clear audioBuffers array to free up memory
audioBuffers.length = 0;
console.log(`Exported batch ${currentBatch} of ${totalBatches}`);
}
} catch (error) {
console.error('Error combining files:', error);
if (progressText) {
progressText.textContent = 'Error during export!';
}
} finally {
if (progressIndicator && progressText) {
progressIndicator.style.display = 'none';
progressText.textContent = ''; // Reset progress text
progressIndicator.style.display = 'block';
progressText.textContent = 'Starting export...';
}
try {
for (let i = 0; i < fileUrls.length; i += splitEvery) {
const currentBatch = Math.floor(i / splitEvery) + 1;
const totalBatches = Math.ceil(fileUrls.length / splitEvery);
if (progressText) {
progressText.textContent = `Processing batch ${currentBatch} of ${totalBatches}`;
progressText.style.color = 'black'; // Set the text color to black
}
const batchUrls = fileUrls.slice(i, Math.min(i + splitEvery, fileUrls.length));
const audioBuffers = await crunker.fetchAudio(...batchUrls);
const combined = crunker.concatAudio(audioBuffers);
const output = crunker.export(combined, "audio/wav");
crunker.download(output.blob, `combined_audio_part_${currentBatch}.wav`);
// Clear audioBuffers array to free up memory
audioBuffers.length = 0;
console.log(`Exported batch ${currentBatch} of ${totalBatches}`);
}
} catch (error) {
console.error('Error combining files:', error);
if (progressText) {
progressText.textContent = 'Error during export!';
}
} finally {
if (progressIndicator && progressText) {
progressIndicator.style.display = 'none';
progressText.textContent = ''; // Reset progress text
}
}
}
}
window.exportMP3 = exportMP3;
window.exportMP3 = exportMP3;
@@ -710,7 +709,10 @@ window.exportMP3 = exportMP3;
let listPlayIndex = 0;
let currentPage = 1;
let itemsPerPage = 100; // Adjust as needed
var protocol = window.location.protocol;
var host = window.location.hostname;
var port = window.location.port;
var baseUrl = protocol + "//" + host + (port ? ':' + port : '');
////////////////////////////////////
//////// Progress Indicator ////////
@@ -851,7 +853,7 @@ window.exportMP3 = exportMP3;
autoplay_volume: volume
});
const response = await fetch("http://127.0.0.1:7851/api/tts-generate", {
const response = await fetch(baseUrl + "/api/tts-generate", {
method: "POST",
headers: {
"Content-Type": "application/x-www-form-urlencoded",
@@ -921,7 +923,7 @@ window.exportMP3 = exportMP3;
autoplay: "false",
autoplay_volume: "0.1"
});
const response = await fetch("http://127.0.0.1:7851/api/tts-generate", {
const response = await fetch(baseUrl + "/api/tts-generate", {
method: "POST",
headers: {
"Content-Type": "application/x-www-form-urlencoded",
@@ -1051,7 +1053,7 @@ window.exportMP3 = exportMP3;
autoplay_volume: volume,
});
const response = await fetch("http://127.0.0.1:7851/api/tts-generate", {
const response = await fetch(baseUrl + "/api/tts-generate", {
method: "POST",
headers: {
"Content-Type": "application/x-www-form-urlencoded",
@@ -1081,8 +1083,9 @@ window.exportMP3 = exportMP3;
});
// Reinitialize currentSound every time
var srcUrl = baseUrl + "/tts-demo-request?" + queryParams.toString();
currentSound = new Howl({
src: [`http://127.0.0.1:7851/tts-demo-request?${queryParams.toString()}`],
src: [srcUrl],
volume: volume,
html5: true // Force HTML5 Audio to enable streaming
});
@@ -1141,7 +1144,7 @@ window.exportMP3 = exportMP3;
const formData = new FormData();
formData.append("voice", characterVoice);
const response = await fetch("http://127.0.0.1:7851/api/previewvoice/", {
const response = await fetch(baseUrl + "/api/previewvoice/", {
method: "POST",
body: formData,
});
@@ -1531,54 +1534,54 @@ window.exportMP3 = exportMP3;
}
function importFromJSON() {
const input = document.createElement('input');
input.type = 'file';
input.accept = '.json';
const input = document.createElement('input');
input.type = 'file';
input.accept = '.json';
input.onchange = async (e) => {
const file = e.target.files[0];
if (file) {
const reader = new FileReader();
reader.readAsText(file);
reader.onload = (e) => {
try {
const jsonData = JSON.parse(e.target.result);
input.onchange = async (e) => {
const file = e.target.files[0];
if (file) {
const reader = new FileReader();
reader.readAsText(file);
reader.onload = (e) => {
try {
const jsonData = JSON.parse(e.target.result);
ttsList = jsonData;
currentPage = 1; // Reset to the first page
updateUIList();
updatePaginationControls(); // Update pagination controls
} catch (error) {
console.error('Error parsing JSON:', error);
}
};
ttsList = jsonData;
currentPage = 1; // Reset to the first page
updateUIList();
updatePaginationControls(); // Update pagination controls
} catch (error) {
console.error('Error parsing JSON:', error);
}
};
}
};
input.click();
}
function updatePaginationControls() {
const lastPage = Math.ceil(ttsList.length / itemsPerPage);
const pageIndicator = document.getElementById('pageIndicator');
if (pageIndicator) {
pageIndicator.textContent = `Page ${currentPage} of ${lastPage}`;
}
};
input.click();
}
const prevPageButton = document.getElementById('prevPage');
if (prevPageButton) {
prevPageButton.disabled = (currentPage === 1);
}
function updatePaginationControls() {
const lastPage = Math.ceil(ttsList.length / itemsPerPage);
const pageIndicator = document.getElementById('pageIndicator');
if (pageIndicator) {
pageIndicator.textContent = `Page ${currentPage} of ${lastPage}`;
const nextPageButton = document.getElementById('nextPage');
if (nextPageButton) {
nextPageButton.disabled = (currentPage === lastPage);
}
// Similar updates for any additional pagination controls if you have them
// ...
}
const prevPageButton = document.getElementById('prevPage');
if (prevPageButton) {
prevPageButton.disabled = (currentPage === 1);
}
const nextPageButton = document.getElementById('nextPage');
if (nextPageButton) {
nextPageButton.disabled = (currentPage === lastPage);
}
// Similar updates for any additional pagination controls if you have them
// ...
}
@@ -1633,7 +1636,7 @@ function updatePaginationControls() {
async function fetchVoices() {
try {
const response = await fetch("http://127.0.0.1:7851/api/voices");
const response = await fetch(baseUrl + "/api/voices");
const result = await response.json();
const characterVoiceSelect = document.getElementById("character_voice_gen");
@@ -1985,9 +1988,11 @@ function updatePaginationControls() {
window.onload = function () {
setupPage();
hideProgress(); // Ensure the progress indicator is hidden on page load
if (window.location.protocol !== 'http:' && window.location.protocol !== 'https:') {
alert('AllTalk TTS Generator is not designed to be run from your disk folder. Please run it from the correct URL e.g. http://127.0.0.1:7851/static/tts_chunk_player/tts_chunk_player.html');
}
};
</script>
</body>
tts_server.py
+21 -15
View File
@@ -57,6 +57,10 @@ params = load_config(configfile_path)
# check someone hasnt enabled lowvram on a system thats not cuda enabled
params["low_vram"] = "false" if not torch.cuda.is_available() else params["low_vram"]
# Load values for temperature and repetition_penalty
temperature = params["local_temperature"]
repetition_penalty = params["local_repetition_penalty"]
# Define the path to the JSON file
config_file_path = this_dir / "modeldownload.json"
@@ -473,16 +477,16 @@ async def deepspeed(request: Request, new_deepspeed_value: bool):
########################
# TTS VOICE GENERATION METHODS (called from voice_preview and output_modifer)
async def generate_audio(text, voice, language, output_file, streaming=False):
async def generate_audio(text, voice, language, temperature, repetition_penalty, output_file, streaming=False):
# Get the async generator from the internal function
response = generate_audio_internal(text, voice, language, output_file, streaming)
response = generate_audio_internal(text, voice, language, temperature, repetition_penalty, output_file, streaming)
# If streaming, then return the generator as-is, otherwise just exhaust it and return
if streaming:
return response
async for _ in response:
pass
async def generate_audio_internal(text, voice, language, output_file, streaming):
async def generate_audio_internal(text, voice, language, temperature, repetition_penalty, output_file, streaming):
global model
if params["low_vram"] and device == "cpu":
await switch_device()
@@ -504,9 +508,9 @@ async def generate_audio_internal(text, voice, language, output_file, streaming)
"language": language,
"gpt_cond_latent": gpt_cond_latent,
"speaker_embedding": speaker_embedding,
"temperature": float(params["local_temperature"]),
"temperature": float(temperature),
"length_penalty": float(model.config.length_penalty),
"repetition_penalty": float(params["local_repetition_penalty"]),
"repetition_penalty": float(repetition_penalty),
"top_k": int(model.config.top_k),
"top_p": float(model.config.top_p),
"enable_text_splitting": True
@@ -560,9 +564,9 @@ async def generate_audio_internal(text, voice, language, output_file, streaming)
file_path=output_file,
speaker_wav=[f"{this_dir}/voices/{voice}"],
language=language,
temperature=float(params["local_temperature"]),
temperature=temperature,
length_penalty=model.config.length_penalty,
repetition_penalty=float(params["local_repetition_penalty"]),
repetition_penalty=repetition_penalty,
top_k=model.config.top_k,
top_p=model.config.top_p,
)
@@ -602,10 +606,12 @@ async def generate(request: Request):
text = data["text"]
voice = data["voice"]
language = data["language"]
temperature = data["temperature"]
repetition_penalty = data["repetition_penalty"]
output_file = data["output_file"]
streaming = False
# Generation logic
response = await generate_audio(text, voice, language, output_file, streaming)
response = await generate_audio(text, voice, language, temperature, repetition_penalty, output_file, streaming)
if streaming:
return StreamingResponse(response, media_type="audio/wav")
return JSONResponse(
@@ -722,7 +728,7 @@ async def update_settings(
async def tts_demo_request_streaming(text: str, voice: str, language: str, output_file: str):
try:
output_file_path = this_dir / "outputs" / output_file
stream = await generate_audio(text, voice, language, output_file_path, streaming=True)
stream = await generate_audio(text, voice, language, temperature, repetition_penalty, output_file_path, streaming=True)
return StreamingResponse(stream, media_type="audio/wav")
except Exception as e:
print(f"An error occurred: {e}")
@@ -732,7 +738,7 @@ async def tts_demo_request_streaming(text: str, voice: str, language: str, outpu
async def tts_demo_request(request: Request, text: str = Form(...), voice: str = Form(...), language: str = Form(...), output_file: str = Form(...)):
try:
output_file_path = this_dir / "outputs" / output_file
await generate_audio(text, voice, language, output_file_path, streaming=False)
await generate_audio(text, voice, language, temperature, repetition_penalty, output_file_path, streaming=False)
return JSONResponse(content={"output_file_path": str(output_file)}, status_code=200)
except Exception as e:
print(f"An error occurred: {e}")
@@ -794,7 +800,7 @@ async def preview_voice(request: Request, voice: str = Form(...)):
# Generate the audio
output_file_path = this_dir / "outputs" / f"{output_file_name}.wav"
await generate_audio(text, voice, language, output_file_path, streaming=False)
await generate_audio(text, voice, language, temperature, repetition_penalty, output_file_path, streaming=False)
# Generate the URL
output_file_url = f'http://{params["ip_address"]}:{params["port_number"]}/audio/{output_file_name}.wav'
@@ -831,7 +837,7 @@ import hashlib
async def tts_generate_streaming(text: str, voice: str, language: str, output_file: str):
try:
output_file_path = this_dir / "outputs" / output_file
stream = await generate_audio(text, voice, language, output_file_path, streaming=True)
stream = await generate_audio(text, voice, language, temperature, repetition_penalty, output_file_path, streaming=True)
return StreamingResponse(stream, media_type="audio/wav")
except Exception as e:
print(f"An error occurred: {e}")
@@ -841,7 +847,7 @@ async def tts_generate_streaming(text: str, voice: str, language: str, output_fi
async def tts_generate_streaming(request: Request, text: str = Form(...), voice: str = Form(...), language: str = Form(...), output_file: str = Form(...)):
try:
output_file_path = this_dir / "outputs" / output_file
await generate_audio(text, voice, language, output_file_path, streaming=False)
await generate_audio(text, voice, language, temperature, repetition_penalty, output_file_path, streaming=False)
return JSONResponse(content={"output_file_path": str(output_file)}, status_code=200)
except Exception as e:
print(f"An error occurred: {e}")
@@ -1054,7 +1060,7 @@ async def tts_generate(
cleaned_part = re.sub(r'\n+', ' ', cleaned_part)
output_file = this_dir / "outputs" / f"{output_file_name}_{uuid.uuid4()}_{int(time.time())}.wav"
output_file_str = output_file.as_posix()
response = await generate_audio(cleaned_part, voice_to_use, language, output_file_str, streaming)
response = await generate_audio(cleaned_part, voice_to_use, language,temperature, repetition_penalty, output_file_str, streaming)
audio_path = output_file_str
audio_files_all_paragraphs.append(audio_path)
# Combine audio files across paragraphs
@@ -1091,7 +1097,7 @@ async def tts_generate(
cleaned_string = re.sub(r'\n+', ' ', cleaned_string)
else:
cleaned_string = text_input
response = await generate_audio(cleaned_string, character_voice_gen, language, output_file_path, streaming)
response = await generate_audio(cleaned_string, character_voice_gen, language, temperature, repetition_penalty, output_file_path, streaming)
if sounddevice_installed == False or streaming == True:
autoplay = False
if autoplay: