diff --git a/README.md b/README.md index 1c011a9..e045f9d 100644 --- a/README.md +++ b/README.md @@ -13,6 +13,7 @@ AllTalk is an updated version of the Coqui_tts extension for Text Generation web - **Documentation:** Fully documented with a built in webpage. [Screenshot](https://github.com/erew123/alltalk_tts#screenshots) - **Console output:** Clear command line output for any warnings or issues. - **API Suite and 3rd Party support via JSON calls:** Can be used with 3rd party applications via JSON calls. +- **SillyTavern integration:** Full integration with SillyTavern. ## Index @@ -31,10 +32,49 @@ AllTalk is an updated version of the Coqui_tts extension for Text Generation web Please check the below link to find a list of all recent updates and changes. ####     🔄 **Updates list & bug fixes list** can be found [here](https://github.com/erew123/alltalk_tts/issues/25) +## 🟩 Quick Setup (Text-generation-webui & Standalone Installation) +For Windows 10/11 and Linux machines there is a quick setup script. +
+ (Click to expand) QUICK SETUP - Text-Generation-webui
+ +1) To download the AllTalk setup you can either: + - A) On this page select **CODE** > **Download ZIP** then extract it to the text-generation-webui extensions folder
e.g. `\text-generation-webui\extensions\alltalk_tts\`

+ - B) Go to a terminal/console, move into the `\text-generation-webui\extensions\` folder
and `git clone https://github.com/erew123/alltalk_tts`

+3) In a terminal/command prompt, in the text-generation-webui folder you will start its Python environment with either `cmd_windows.bat` or `./cmd_linux.sh` +4) Move into the AllTalk folder e.g. `cd extensions` then `cd alltalk_tts` +5) Start the AllTalk setup script `atsetup.bat` or `./atsetup.sh` +6) Follow the on-screen prompts and install the correct requirements files that you need. It's recommended to test AllTalk works before installing DeepSpeed. + + Any time you need to make changes to AllTalk, or use Finetuning etc, always start the Text-generation-webui Python environment first. + + Finetuning & DeepSpeed have other installation requirements (depending on your OS) so please read any instructions in the setup utility and refer back here to this page for detailed instructions (as needed).

+
+
+ (Click to expand) QUICK SETUP - Standalone Installation
+ +1) To download the AllTalk setup you can either: + - A) On this page select **CODE** > **Download ZIP** then extract it to the folder of your choice
e.g. `C:\myfiles\alltalk_tts\`

+ - B) Go to a terminal/console, move into the folder of your choice e.g `C:\myfiles\` folder
and `git clone https://github.com/erew123/alltalk_tts`

+4) In a terminal/command prompt, move into the AllTalk folder e.g. `cd alltalk_tts` +5) Start the AllTalk setup script `atsetup.bat` or `./atsetup.sh` +6) Follow the on-screen prompts and install the correct requirements files that you need. It's recommended to test AllTalk works before installing DeepSpeed. + + DeepSpeed on Windows machines will be installed as standard. Linux machines have other requirements which are detailed within the setup utility and on this page. + + Finetuning has other installation requirements so please read any instructions in the setup utility and refer back here to this page for detailed instructions.

+
+ + + ## 🟩 Installation on Text generation web UI +On Mac's or if you wish to perform a manual installation, please expand the below. +
+ (Click to expand) MANUAL INSTALLATION - Text-Generation-webui
+ This has been tested on the current Dec 2023 release of Text generation webUI. If you have not updated it for a while, you may wish to update Text generation webUI, [instructions here](https://github.com/oobabooga/text-generation-webui?tab=readme-ov-file#how-to-install) -If you want to watch a video of how to do the below [link here](https://youtu.be/9BPKuwaav5w) +- If you want to watch a video of how to do the below [link here](https://youtu.be/9BPKuwaav5w)
+- If you wish to install AllTalk as a **standalone application** without Text-generation-webui, please go to [this link here](https://github.com/erew123/alltalk_tts?#-running-alltalk-as-a-standalone-app) 1) In a command prompt/terminal window you need to move into your Text generation webUI folder:

`cd text-generation-webui` @@ -62,6 +102,7 @@ If you want to watch a video of how to do the below [link here](https://youtu.be 10) Please read the note below about start-up times and also the note about ensuring your character cards are set up [correctly](https://github.com/erew123/alltalk_tts#the-one-thing-i-cant-easily-work-around) 11) Some extra voices downloadable [here](https://drive.google.com/file/d/1bYdZdr3L69kmzUN3vSiqZmLRD7-A3M47/view?usp=drive_link) +
#### 🟩 Other installation notes On first startup, AllTalk will download the Coqui XTTSv2 2.0.2 model to its **models** folder (1.8GB space required). Check the command prompt/terminal window if you want to know what its doing. After it says "Model Loaded" the Text generation webUI is usually available on its IP address a few seconds later, for you to connect to in your browser. @@ -621,6 +662,22 @@ Retrieve a list of available voices for generating speech. JSON return: `{"voices": ["voice1.wav", "voice2.wav", "voice3.wav"]}` +#### 🟠 Current Settings Endpoint
+Retrieve a list of available voices for generating speech. + +- URL: `http://127.0.0.1:7851/api/currentsettings`
- Method: `GET`
+ + `curl -X GET "http://127.0.0.1:7851/api/currentsettings"` + + JSON return: `{"models_available":[{"name":"Coqui","model_name":"API TTS"},{"name":"Coqui","model_name":"API Local"},{"name":"Coqui","model_name":"XTTSv2 Local"}],"current_model_loaded":"XTTSv2 Local","deepspeed_available":true,"deepspeed_status":true,"low_vram_status":true,"finetuned_model":false} + + name & model_name = listing the currently available models. + current_model_loaded = what model is currently loaded into VRAM. + deepspeed_available = was DeepSpeed detected on startup and available to be activated. + deepspeed_status = If DeepSpeed was detected, is it currently activated. + low_vram_status = Is Low VRAM currently enabled. + finetuned_model = Was a finetuned model detected. (XTTSv2 FT). + #### 🟠 Preview Voice Endpoint Generate a preview of a specified voice with hardcoded settings. @@ -669,8 +726,8 @@ Generate a preview of a specified voice with hardcoded settings. - URL: `http://127.0.0.1:7851/api/tts-generate`
- Method: `POST`
- Content-Type: `application/x-www-form-urlencoded`
-### 🟠 Example command lines -Standard TTS speech Example (standard text) generating a time-stamped file
+### 🟠 Example command lines (Standard Generation) +Standard TTS generation supports Narration and will generate a wav file/blob. Standard TTS speech Example (standard text) generating a time-stamped file
`curl -X POST "http://127.0.0.1:7851/api/tts-generate" -d "text_input=All of this is text spoken by the character. This is text not inside quotes, though that doesnt matter in the slightest" -d "text_filtering=standard" -d "character_voice_gen=female_01.wav" -d "narrator_enabled=false" -d "narrator_voice_gen=male_01.wav" -d "text_not_inside=character" -d "language=en" -d "output_file_name=myoutputfile" -d "output_file_timestamp=true" -d "autoplay=true" -d "autoplay_volume=0.8"`
@@ -773,7 +830,29 @@ Example JSON TTS Generation Response: `{"status":"generate-success","output_file_path":"C:\\text-generation-webui\\extensions\\alltalk_tts\\outputs\\myoutputfile_1704141936.wav","output_file_url":"http://127.0.0.1:7851/audio/myoutputfile_1704141936.wav","output_cache_url":"http://127.0.0.1:7851/audiocache/myoutputfile_1704141936.wav"}` +### 🟠 Example command lines (Streaming Generation) +Streaming TTS generation does NOT support Narration and will generate an audio stream. Streaming TTS speech JavaScript Example:
+``` +// Example parameters +const text = "Here is some text"; +const voice = "female_01.wav"; +const language = "en"; +const outputFile = "stream_output.wav"; +// Encode the text for URL +const encodedText = encodeURIComponent(text); +// Create the streaming URL +const streamingUrl = `http://localhost:7851/api/tts-generate-streaming?text=${encodedText}&voice=${voice}&language=${language}&output_file=${outputFile}`; +// Create and play the audio element +const audioElement = new Audio(streamingUrl); +audioElement.play(); // Play the audio stream directly +``` +- **Text (text):** This is the actual text you want to convert to speech. It should be a string and must be URL-encoded to ensure that special characters (like spaces and punctuation) are correctly transmitted in the URL. Example: `Hello World` becomes `Hello%20World` when URL-encoded.
+- **Voice (voice):** This parameter specifies the voice type to be used for the TTS. The value should match one of the available voice options in AllTalks voices folder. This is a string representing the file, like `female_01.wav`.
+- **Language (language):** This setting determines the language in which the text should be spoken. A two-letter language code (like `en` for English, `fr` for French, etc.).
+- **Output File (output_file):** This parameter names the output file where the audio will be streamed. It should be a string representing the file name, such as `stream_output.wav`. AllTalk will not save this as a file in its outputs folder.
+
+ ### 🔴 Future to-do list -- SillyTavern Support - Possibly add some additional TTS engines (TBD). +- Tagging support within text. - Have a break! diff --git a/atsetup.bat b/atsetup.bat new file mode 100644 index 0000000..387dc70 --- /dev/null +++ b/atsetup.bat @@ -0,0 +1,495 @@ +@echo off +cd /D "%~dp0" + +set PATH=%PATH%;%SystemRoot%\system32 + +:: Generate the ESC character +for /F %%a in ('echo prompt $E ^| cmd') do set "ESC=%%a" + +:: Standard Colors +set "BLACK=%ESC%[30m" +set "RED=%ESC%[31m" +set "GREEN=%ESC%[32m" +set "YELLOW=%ESC%[33m" +set "BLUE=%ESC%[34m" +set "MAGENTA=%ESC%[35m" +set "CYAN=%ESC%[36m" +set "WHITE=%ESC%[37m" +set "L_BLACK=%ESC%[90m" +set "L_RED=%ESC%[91m" +set "L_GREEN=%ESC%[92m" +set "L_YELLOW=%ESC%[93m" +set "L_BLUE=%ESC%[94m" +set "L_MAGENTA=%ESC%[95m" +set "L_CYAN=%ESC%[96m" +set "L_WHITE=%ESC%[97m" +set "RESET=%ESC%[0m" + +@rem Check if curl is available +curl --version >nul 2>&1 +if "%ERRORLEVEL%" NEQ "0" ( + echo curl is not available on this system. Please install curl then re-run the script https://curl.se/ + echo or perform a manual installation of a Conda Python environment. + goto end +) + +:MainMenu +cls +echo. +echo %L_BLUE%ALLTALK WINDOWS SETUP UTILITY%RESET% +echo. +echo INSTALLATION TYPE +echo 1) I am using AllTalk as part of %L_GREEN%Text-generation-webui%RESET% +echo 2) I am using AllTalk as a %L_GREEN%Standalone Application%RESET% +echo. +echo 9) %L_RED%Exit/Quit%RESET% +echo. +set /p UserOption=" Enter your choice: " + +if "%UserOption%"=="1" goto WebUIMenu +if "%UserOption%"=="2" goto StandaloneMenu +if "%UserOption%"=="9" goto End +goto MainMenu + +:WebUIMenu +cls +echo. +echo %L_BLUE%TEXT-GENERATION-WEBUI SETUP%RESET% +echo. +echo Please ensure you have started your Text-generation-webui Python +echo environment. If you have NOT done this, please run %L_GREEN%cmd_windows.bat%RESET% +echo in the %L_GREEN%text-generation-webui%RESET% folder and then re-run this script. +echo. +echo BASE REQUIREMENTS +echo 1) Install the requirements for a %L_GREEN%Nvidia machine%RESET%. +echo 2) Install the requirements for a %L_GREEN%AMD or MAC machine%RESET%. +echo. +echo OPTIONAL +echo 3) Install the Finetuning requirements. +echo 4) Install DeepSpeed v11.2 for CUDA %L_GREEN%11.8%RESET% and Python-3.11.x. +echo 5) Install DeepSpeed v11.2 for CUDA %L_GREEN%12.1%RESET% and Python-3.11.x. +echo 6) Uninstall DeepSpeed. +echo. +echo. OTHER +echo 7) Generate a diagnostics file. +echo. +echo 9) %L_RED%Exit/Quit%RESET% +echo. +set /p WebUIOption="Enter your choice: " +if "%WebUIOption%"=="1" goto InstallNvidiaTextGen +if "%WebUIOption%"=="2" goto InstallOtherTextGen +if "%WebUIOption%"=="3" goto InstallFinetuneTextGen +if "%WebUIOption%"=="4" goto InstallDeepSpeed118TextGen +if "%WebUIOption%"=="5" goto InstallDeepSpeed121TextGen +if "%WebUIOption%"=="6" goto UnInstallDeepSpeed +if "%WebUIOption%"=="7" goto GenerateDiagsTextGen +if "%WebUIOption%"=="9" goto End +goto WebUIMenu + +:StandaloneMenu +cls +echo. +echo %L_BLUE%ALLTALK STANDALONE APPLICATION SETUP%RESET% +echo. +echo BASE REQUIREMENTS +echo 1) Install AllTalk as a Standalone Application +echo. +echo OPTIONAL +echo 2) Delete AllTalk's custom Python environment +echo 3) Install the Finetuning requirements +echo. +echo. OTHER +echo 4) Generate a diagnostics file +echo. +echo 9) %L_RED%Exit/Quit%RESET% +echo. +set /p StandaloneOption="Enter your choice: " +if "%StandaloneOption%"=="1" goto InstallCustomStandalone +if "%StandaloneOption%"=="2" goto DeleteCustomStandalone +if "%StandaloneOption%"=="3" goto InstallFinetuneStandalone +if "%StandaloneOption%"=="4" goto GenerateDiagsStandalone +if "%StandaloneOption%"=="9" goto End +goto StandaloneMenu + +:InstallNvidiaTextGen +pip install -r requirements_nvidia.txt +if %ERRORLEVEL% neq 0 ( + echo. + echo There was an error installing the Nvidia requirements. + echo Press any key to return to the menu. + echo. + pause + goto WebUIMenu +) +Echo. +echo. +Echo Nvidia machine requirements installed successfully. +Echo. +pause +goto WebUIMenu + +:InstallOtherTextGen +pip install -r requirements_other.txt +if %ERRORLEVEL% neq 0 ( + echo. + echo There was an error installing the Other requirements. + echo Press any key to return to the menu. + echo. + pause + goto WebUIMenu +) +Echo. +echo. +Echo Other machine requirements installed successfully. +Echo. +pause +goto WebUIMenu + +:InstallFinetuneTextGen +pip install -r requirements_finetune.txt +if %ERRORLEVEL% neq 0 ( + echo. + echo There was an error installing the Finetune requirements. + echo Press any key to return to the menu. + echo. + pause + goto WebUIMenu +) +Echo. +Echo Finetune requirements installed successfully. Please ensure you have installed CUDA 11.8. +Echo Finetune needs access to %L_GREEN%cublas64_11.dll%RESET% and will error if this is not setup correctly. +Echo Please see here for details %L_GREEN%https://github.com/erew123/alltalk_tts#-finetuning-a-model%RESET% +Echo. +pause +goto WebUIMenu + +:InstallDeepSpeed118TextGen +echo Downloading DeepSpeed... +curl -LO https://github.com/erew123/alltalk_tts/releases/download/deepspeed/deepspeed-0.11.2+cuda118-cp311-cp311-win_amd64.whl +if %ERRORLEVEL% neq 0 ( + echo. + echo Failed to download DeepSpeed wheel file. + echo Please check your internet connection or try again later. + echo Press any key to return to the menu. + echo. + pause + goto WebUIMenu +) +echo DeepSpeed wheel file downloaded successfully. +echo Installing DeepSpeed... +pip install deepspeed-0.11.2+cuda118-cp311-cp311-win_amd64.whl +if %ERRORLEVEL% neq 0 ( + echo. + echo Failed to install DeepSpeed. + echo Please check if the wheel file is compatible with your system. + echo Press any key to return to the menu. + echo. + pause + goto WebUIMenu +) +Echo. +echo. +Echo DeepSpeed installed successfully. +Echo. +del deepspeed-0.11.2+cuda118-cp311-cp311-win_amd64.whl +pause +goto WebUIMenu + +:InstallDeepSpeed121TextGen +echo Downloading DeepSpeed... +curl -LO https://github.com/erew123/alltalk_tts/releases/download/deepspeed/deepspeed-0.11.2+cuda121-cp311-cp311-win_amd64.whl +if %ERRORLEVEL% neq 0 ( + echo. + echo Failed to download DeepSpeed wheel file. + echo Please check your internet connection or try again later. + echo Press any key to return to the menu. + echo. + pause + goto WebUIMenu +) +echo DeepSpeed wheel file downloaded successfully. +echo Installing DeepSpeed... +pip install deepspeed-0.11.2+cuda121-cp311-cp311-win_amd64.whl +if %ERRORLEVEL% neq 0 ( + echo. + echo Failed to install DeepSpeed. + echo Please check if the wheel file is compatible with your system. + echo Press any key to return to the menu. + echo. + pause + goto WebUIMenu +) +Echo. +echo. +Echo DeepSpeed installed successfully. +Echo. +del deepspeed-0.11.2+cuda121-cp311-cp311-win_amd64.whl +pause +goto WebUIMenu + +:UnInstallDeepSpeed +pip uninstall deepspeed +if %ERRORLEVEL% neq 0 ( + echo. + echo There was an error uninstalling DeepSpeed. + echo Press any key to return to the menu. + echo. + pause + goto WebUIMenu +) +echo. +Echo DeepSpeed uninstalled successfully. +Echo. +pause +goto WebUIMenu + +:GenerateDiagsTextGen +Python diagnostics.py +if %ERRORLEVEL% neq 0 ( + echo. + echo There was an error running diagnostics. Have you correctly started your + echo Text-generation-webui Python environment with %L_GREEN%cmd_windows.bat%RESET%? + echo. + pause + goto WebUIMenu +) +Echo. +echo. +Echo Diagnostics.log generated. Please scroll up to look over the log. +Echo. +pause +goto WebUIMenu + +:InstallCustomStandalone +set PATH=%PATH%;%SystemRoot%\system32 + +@rem Check if curl is available +curl --version >nul 2>&1 +if "%ERRORLEVEL%" NEQ "0" ( + echo curl is not available on this system. Please install curl then re-run the script https://curl.se/ + echo or perform a manual installation of a Conda Python environment. + goto end +) +echo "%CD%"| findstr /C:" " >nul && echo This script relies on Miniconda which can not be silently installed under a path with spaces. Please correct your folder names and re-try installation. && goto exit + +@rem Check for special characters in installation path +set "SPCHARMESSAGE=WARNING: Special characters were detected in the installation path! This can cause the installation to fail!" +echo "%CD%"| findstr /R /C:"[!#\$%&()\*+,;<=>?@\[\]\^`{|}~]" >nul && ( + call :PrintBigMessage %SPCHARMESSAGE% +) +set SPCHARMESSAGE= + +@rem fix failed install when installing to a separate drive +set TMP=%cd%\alltalk_environment +set TEMP=%cd%\alltalk_environment + +@rem deactivate existing conda envs as needed to avoid conflicts +(call conda deactivate && call conda deactivate && call conda deactivate) 2>nul + +@rem config +set INSTALL_DIR=%cd%\alltalk_environment +set CONDA_ROOT_PREFIX=%cd%\alltalk_environment\conda +set INSTALL_ENV_DIR=%cd%\alltalk_environment\env +set MINICONDA_DOWNLOAD_URL=https://repo.anaconda.com/miniconda/Miniconda3-py310_23.3.1-0-Windows-x86_64.exe +set conda_exists=F + +@rem figure out whether git and conda need to be installed +call "%CONDA_ROOT_PREFIX%\_conda.exe" --version >nul 2>&1 +if "%ERRORLEVEL%" EQU "0" ( + set conda_exists=T + call "%CONDA_ROOT_PREFIX%\condabin\conda.bat" activate "%INSTALL_ENV_DIR%" + goto RunScript +) + +@rem download and install conda if not installed +echo Downloading Miniconda from %MINICONDA_DOWNLOAD_URL% to %INSTALL_DIR%\miniconda_installer.exe +mkdir "%INSTALL_DIR%" +call curl -Lk "%MINICONDA_DOWNLOAD_URL%" > "%INSTALL_DIR%\miniconda_installer.exe" || ( echo. && echo Miniconda failed to download. && goto end ) +echo Installing Miniconda to %CONDA_ROOT_PREFIX% +start /wait "" "%INSTALL_DIR%\miniconda_installer.exe" /InstallationType=JustMe /NoShortcuts=1 /AddToPath=0 /RegisterPython=0 /NoRegistry=1 /S /D=%CONDA_ROOT_PREFIX% +echo Miniconda version: +call "%CONDA_ROOT_PREFIX%\_conda.exe" --version || ( echo. && echo Miniconda not found. && goto end ) + +@rem create the installer env +echo Packages to install: %PACKAGES_TO_INSTALL% +call "%CONDA_ROOT_PREFIX%\_conda.exe" create --no-shortcuts -y -k --prefix "%INSTALL_ENV_DIR%" python=3.11 || ( echo. && echo Conda environment creation failed. && goto end ) + +@rem check if conda environment was actually created +if not exist "%INSTALL_ENV_DIR%\python.exe" ( echo. && echo Conda environment is empty. && goto end ) + +@rem activate installer env +call "%CONDA_ROOT_PREFIX%\condabin\conda.bat" activate "%INSTALL_ENV_DIR%" || ( echo. && echo Miniconda hook not found. && goto end ) + +@rem Ask user for the type of requirements to install +echo. +echo %L_BLUE%Choose the type of requirements to install%RESET% +echo. +echo 1. Nvidia graphics card machines +echo 2. Other machines (mac, amd, etc) +echo. +set /p UserChoice="Enter your choice (1 or 2): " + +@rem Install requirements based on user choice +if "%UserChoice%" == "1" ( + echo. + echo Installing Nvidia requirements... + echo Downloading Pytorch with CUDA.. + pip install torch==2.1.0 torchvision==0.16.0 torchaudio==2.1.0 --upgrade --force-reinstall --index-url https://download.pytorch.org/whl/cu121 + pip install numpy==1.24.4 + pip install typing-extensions>=4.8.0 + pip install networkx<3.0.0,>=2.5.0 + pip install fsspec>=2023.6.0 + pip install soundfile==0.12.1 + pip install uvicorn==0.24.0.post1 + pip install TTS==0.21.3 + pip install fastapi==0.104.1 + pip install Jinja2==3.1.2 + pip install requests==2.31.0 + pip install tqdm==4.66.1 + pip install importlib-metadata==4.8.1 + pip install packaging==23.2 + pip install pydantic==1.10.13 + pip install sounddevice==0.4.6 + pip install python-multipart==0.0.6 + echo Downloading DeepSpeed... + curl -LO https://github.com/erew123/alltalk_tts/releases/download/DeepSpeed-12.7/deepspeed-0.12.7+d058d4b-cp311-cp311-win_amd64.whl + echo Installing DeepSpeed... + pip install deepspeed-0.12.7+d058d4b-cp311-cp311-win_amd64.whl + del deepspeed-0.12.7+d058d4b-cp311-cp311-win_amd64.whl +) else ( + echo Installing requirements for other machines... + pip install -r requirements_other.txt +) + +@rem Create start_alltalk.bat to run AllTalk +echo @echo off > start_alltalk.bat +echo cd /D "%~dp0" >> start_alltalk.bat +echo set CONDA_ROOT_PREFIX=%cd%\alltalk_environment\conda >> start_alltalk.bat +echo set INSTALL_ENV_DIR=%cd%\alltalk_environment\env >> start_alltalk.bat +echo call "%CONDA_ROOT_PREFIX%\condabin\conda.bat" activate "%INSTALL_ENV_DIR%" >> start_alltalk.bat +echo call python script.py >> start_alltalk.bat +echo exit >> start_alltalk.bat +Echo. +Echo start_alltalk.bat has been created. +Echo You can now start AllTalk with %L_YELLOW%start_alltalk.bat%RESET% +Echo. +pause +goto StandaloneMenu + +:DeleteCustomStandalone +@rem Check if the alltalk_environment directory exists +if not exist "%cd%\alltalk_environment\" ( + echo. + echo %L_GREEN%alltalk_environment%RESET% directory does not exist. No need to delete. + echo. + pause + goto StandaloneMenu +) +@rem Check if a Conda environment is active +if not defined CONDA_PREFIX goto NoCondaEnvDeleteCustomStandalone +@rem Deactivate Conda environment if it's active +Echo Exiting the Conda Environment. Please run %L_GREEN%atsetup.bat%RESET% again and delete the environment. +conda deactivate +:NoCondaEnvDeleteCustomStandalone +echo Deleting "alltalk_environment". Please wait. +rd /s /q "alltalk_environment" +del start_alltalk.bat +if %ERRORLEVEL% neq 0 ( + echo. + echo Failed to delete alltalk_environment folder. + echo Please make sure it is not in use and try again. + echo. + pause + goto StandaloneMenu +) +echo. +Echo. +echo Environment %L_GREEN%alltalk_environment%RESET% deleted. Please set up the environment again. +Echo. +pause +goto StandaloneMenu + +:InstallFinetuneStandalone +cd /D "%~dp0" +set CONDA_ROOT_PREFIX=%cd%\alltalk_environment\conda +set INSTALL_ENV_DIR=%cd%\alltalk_environment\env +@rem Check if the Conda environment exists +if not exist "%INSTALL_ENV_DIR%\python.exe" ( + echo. + echo The Conda environment at "%INSTALL_ENV_DIR%" does not exist. + echo Please install the environment before proceeding. + echo. + pause + goto StandaloneMenu +) +@rem Attempt to activate the Conda environment +call "%CONDA_ROOT_PREFIX%\condabin\conda.bat" activate "%INSTALL_ENV_DIR%" +if errorlevel 1 ( + echo. + echo Failed to activate the Conda environment. + echo Please check your installation and try again. + echo. + pause + goto StandaloneMenu +) +@rem Proceed with installing requirements +pip install -r requirements_finetune.txt +if %ERRORLEVEL% neq 0 ( + echo. + echo There was an error installing the Finetune requirements. + echo Press any key to return to the menu. + echo. + pause + goto StandaloneMenu +) +Echo. +Echo Finetune requirements installed successfully. Please ensure you have installed CUDA 11.8. +Echo Finetune needs access to %L_GREEN%cublas64_11.dll%RESET% and will error if this is not setup correctly. +Echo Please see here for details %L_GREEN%https://github.com/erew123/alltalk_tts#-finetuning-a-model%RESET% +Echo. +pause +goto StandaloneMenu + +:GenerateDiagsStandalone +cd /D "%~dp0" +set CONDA_ROOT_PREFIX=%cd%\alltalk_environment\conda +set INSTALL_ENV_DIR=%cd%\alltalk_environment\env +@rem Check if the Conda environment exists +if not exist "%INSTALL_ENV_DIR%\python.exe" ( + echo. + echo The Conda environment at "%INSTALL_ENV_DIR%" does not exist. + echo Please install the environment before proceeding. + echo. + pause + goto StandaloneMenu +) +@rem Attempt to activate the Conda environment +call "%CONDA_ROOT_PREFIX%\condabin\conda.bat" activate "%INSTALL_ENV_DIR%" +if errorlevel 1 ( + echo. + echo Failed to activate the Conda environment. + echo Please check your installation and try again. + echo. + pause + goto StandaloneMenu +) +@rem Run diagnostics +Python diagnostics.py +if %ERRORLEVEL% neq 0 ( + echo. + echo There was an error running diagnostics. + echo Press any key to return to the menu. + echo. + pause + goto StandaloneMenu +) +Echo. +Echo. +Echo Diagnostics.log generated. Please scroll up to look over the log. +Echo. +pause +goto StandaloneMenu + +:End +echo Exiting AllTalk Setup Utility... +exit /b diff --git a/atsetup.sh b/atsetup.sh new file mode 100644 index 0000000..5712b1c --- /dev/null +++ b/atsetup.sh @@ -0,0 +1,421 @@ +#!/bin/bash + +# ANSI color codes +RED='\033[1;31m' +GREEN='\033[1;32m' +YELLOW='\033[1;33m' +BLUE='\033[1;34m' +CYAN='\033[1;36m' +MAGENTA='\033[1;35m' +WHITE='\033[1;37m' +L_RED='\033[1;91m' +L_GREEN='\033[1;92m' +L_YELLOW='\033[1;93m' +L_BLUE='\033[1;94m' +L_CYAN='\033[1;96m' +L_MAGENTA='\033[1;95m' +NC='\033[0m' # No Color + +# Navigate to the script's directory +cd "$(dirname "$0")" + +# Function to check if curl is installed +check_curl() { + if ! command -v curl &> /dev/null; then + echo "curl is not available on this system. Please install curl then re-run the script https://curl.se/" + echo "or perform a manual installation of a Conda Python environment." + exit 1 + fi +} + +# Main Menu +main_menu() { + while true; do + clear + echo + echo -e " ${L_BLUE}ALLTALK LINUX SETUP UTILITY${NC}" + echo + echo " INSTALLATION TYPE" + echo -e " 1) I am using AllTalk as part of ${L_GREEN}Text-generation-webui${NC}" + echo -e " 2) I am using AllTalk as a ${L_GREEN}Standalone Application${NC}" + echo + echo -e " 9)${L_RED} Exit/Quit${NC}" + echo + read -p " Enter your choice: " user_option + + case $user_option in + 1) webui_menu ;; + 2) standalone_menu ;; + 9) exit 0 ;; + *) echo "Invalid option"; sleep 2 ;; + esac + done +} + +# Text-generation-webui Menu +webui_menu() { + while true; do + clear + echo + echo -e " ${L_BLUE}TEXT-GENERATION-WEBUI SETUP${NC}" + echo + echo -e " Please ensure you have started your Text-generation-webui Python" + echo -e " environment. If you have NOT done this, please run ${L_GREEN}/cmd_linux.sh${NC}" + echo -e " in the ${L_GREEN}text-generation-webui${NC} folder and then re-run this script." + echo + echo " BASE REQUIREMENTS" + echo -e " 1) Install the AllTalk requirements for an ${L_GREEN}Nvidia GPU machine.${NC}" + echo -e " 2) Install the AllTalk requirements for an ${L_GREEN}AMD or MAC machine.${NC}" + echo + echo " OPTIONAL" + echo " 3) Install the Finetuning requirements." + echo " 4) Install DeepSpeed." + echo " 5) Uninstall DeepSpeed." + echo + echo " OTHER" + echo " 6) Generate a diagnostics file." + echo + echo -e " 9)${L_RED} Exit/Quit${NC}" + echo + read -p " Enter your choice: " webui_option + + case $webui_option in + 1) install_nvidia_textgen ;; + 2) install_other_textgen ;; + 3) install_finetune_textgen ;; + 4) install_deepspeed ;; + 5) uninstall_deepspeed ;; + 6) generate_diagnostics_textgen ;; + 9) exit 0 ;; + *) echo "Invalid option"; sleep 2 ;; + esac + done +} + +install_nvidia_textgen() { + local requirements_file="requirements_nvidia.txt" + echo " Installing Finetune requirements from $requirements_file..." + if ! pip install -r "$requirements_file"; then + echo " There was an error installing the Nvidia requirements." + echo " Please check the output for details." + read -p " Press any key to return to the menu. " -n 1 + echo + return + fi + echo " Nvidia requirements installed successfully." + read -p " Press any key to continue. " -n 1 + echo +} + +install_other_textgen() { + local requirements_file="requirements_other.txt" + echo " Installing Other machine requirements from $requirements_file..." + if ! pip install -r "$requirements_file"; then + echo " There was an error installing the Other requirements." + echo " Please check the output for details." + read -p " Press any key to return to the menu. " -n 1 + echo + return + fi + echo " Other machine requirements installed successfully." + read -p " Press any key to continue. " -n 1 + echo +} + +install_finetune_textgen() { + local requirements_file="requirements_finetune.txt" + echo " Installing Finetune requirements from $requirements_file..." + if ! pip install -r "$requirements_file"; then + echo " There was an error installing the Finetune requirements." + echo " Please check the output for details." + read -p " Press any key to return to the menu. " -n 1 + echo + return + fi + echo + echo " Finetune base requirements installed successfully." + echo + echo -e " Finetuning requires access to ${GREEN}Cublas${NC} from the ${GREEN}Nvidia CUDA Toolkit${NC}, so will need installing." + echo + echo -e " Instructions can be found here:" + echo -e " ${GREEN}https://github.com/erew123/alltalk_tts?#-finetuning-a-model${NC}" + echo + read -p " Press any key to continue. " -n 1 + echo +} + +# Function to install DeepSpeed +install_deepspeed() { + clear + echo + echo -e " ${L_BLUE}DEEPSPEED INSTALLATION REQUIREMENTS${NC}" + echo + echo -e " - NVIDIA CUDA Toolkit must be installed from your ${L_GREEN}Linux package manager${NC}" + echo -e " or ${L_GREEN}https://developer.nvidia.com/cuda-toolkit-archive${NC}" + echo -e " - The environment variable ${L_GREEN}CUDA_HOME${NC} must be set in your Python env" + echo -e " e.g. export CUDA_HOME=/usr/local/cuda." + echo -e " - ${L_GREEN}libaio-dev${NC} must be installed." + echo + echo -e " See the following link for full instructions:" + echo -e " ${L_GREEN}https://github.com/erew123/alltalk_tts?#-deepspeed-installation-options${NC}" + echo + echo -e " ${L_BLUE}Please ${L_RED}confirm${L_BLUE} that you have completed these steps before continuing" + echo -e " with the DeepSpeed installation, or the install ${L_RED}will fail.${NC}" + echo + read -p " Have you completed all the above steps? (y/n): " confirm + + if [ "$confirm" != "y" ]; then + echo -e " ${RED}DeepSpeed installation cannot proceed without completing the prerequisites.${NC}" + return + fi + + echo -e "\n ${GREEN}Proceeding with DeepSpeed installation...${NC}" + pip install deepspeed + if [ $? -ne 0 ]; then + echo -e " ${RED}There was an error installing DeepSpeed.${NC}" + return + fi + + echo -e " ${GREEN}DeepSpeed installed successfully.${NC}" + read -p " Press any key to continue. " -n 1 + echo +} + +uninstall_deepspeed() { + echo "Uninstalling DeepSpeed..." + pip uninstall -y deepspeed + if [ $? -ne 0 ]; then + echo " There was an error uninstalling DeepSpeed." + echo " Press any key to return to the menu." + read -n 1 + return + fi + echo " DeepSpeed uninstalled successfully." + echo " Press any key to continue." + read -n 1 +} + +generate_diagnostics_textgen() { + # Run diagnostics + if ! python diagnostics.py; then + echo " There was an error running diagnostics." + read -p " Press any key to return to the menu. " -n 1 + echo + return + fi + + echo " Diagnostics completed successfully." + read -p " Press any key to continue. " -n 1 + echo +} + +# Standalone Menu +standalone_menu() { + while true; do + clear + echo + echo -e " ${L_BLUE}ALLTALK STANDALONE APPLICATION SETUP${NC}" + echo + echo " BASE REQUIREMENTS" + echo " 1) Install AllTalk as a Standalone Application" + echo + echo " OPTIONAL" + echo " 2) Delete AllTalk's custom Python environment" + echo " 3) Install the Finetuning requirements" + echo " 4) Install DeepSpeed." + echo + echo " OTHER" + echo " 5) Generate a diagnostics file" + echo + echo -e " 9)${L_RED} Exit/Quit${NC}" + echo + read -p " Enter your choice: " standalone_option + + case $standalone_option in + 1) install_custom_standalone ;; + 2) delete_custom_standalone ;; + 3) install_finetune_standalone ;; + 4) install_deepspeed ;; + 5) generate_diagnostics_standalone ;; + 9) exit 0 ;; + *) echo "Invalid option"; sleep 2 ;; + esac + done +} + +install_custom_standalone() { + cd "$(dirname "${BASH_SOURCE[0]}")" + + if [[ "$(pwd)" =~ " " ]]; then + echo "This script relies on Miniconda which can not be silently installed under a path with spaces." + exit + fi + + # Deactivate existing conda envs as needed to avoid conflicts + { conda deactivate && conda deactivate && conda deactivate; } 2> /dev/null + + OS_ARCH=$(uname -m) + case "${OS_ARCH}" in + x86_64*) OS_ARCH="x86_64" ;; + arm64* | aarch64*) OS_ARCH="aarch64" ;; + *) echo "Unknown system architecture: $OS_ARCH! This script runs only on x86_64 or arm64" && exit ;; + esac + + # Config + INSTALL_DIR="$(pwd)/alltalk_environment" + CONDA_ROOT_PREFIX="${INSTALL_DIR}/conda" + INSTALL_ENV_DIR="${INSTALL_DIR}/env" + MINICONDA_DOWNLOAD_URL="https://repo.anaconda.com/miniconda/Miniconda3-py310_23.3.1-0-Linux-${OS_ARCH}.sh" + + if [ ! -x "${CONDA_ROOT_PREFIX}/bin/conda" ]; then + echo "Downloading Miniconda from $MINICONDA_DOWNLOAD_URL to ${INSTALL_DIR}/miniconda_installer.sh" + mkdir -p "${INSTALL_DIR}" + curl -L "${MINICONDA_DOWNLOAD_URL}" -o "${INSTALL_DIR}/miniconda_installer.sh" + chmod +x "${INSTALL_DIR}/miniconda_installer.sh" + bash "${INSTALL_DIR}/miniconda_installer.sh" -b -p "${CONDA_ROOT_PREFIX}" + echo "Miniconda installed." + fi + + if [ ! -d "${INSTALL_ENV_DIR}" ]; then + "${CONDA_ROOT_PREFIX}/bin/conda" create -y --prefix "${INSTALL_ENV_DIR}" python=3.11 + echo "Conda environment created at ${INSTALL_ENV_DIR}." + fi + + # Activate the environment and install requirements + source "${CONDA_ROOT_PREFIX}/etc/profile.d/conda.sh" + conda activate "${INSTALL_ENV_DIR}" + + echo + echo -e " ${L_BLUE}Choose the type of requirements to install${NC}" + echo + echo " 1. Nvidia graphics card machines" + echo " 2. Other machines (mac, amd, etc)" + echo + read -p " Enter your choice (1 or 2): " user_choice + + if [ "$user_choice" == "1" ]; then + pip install torch==2.1.0 torchvision==0.16.0 torchaudio==2.1.0 --upgrade --force-reinstall --index-url https://download.pytorch.org/whl/cu121 + pip install numpy==1.24.4 + pip install typing-extensions>=4.8.0 + pip install networkx<3.0.0,>=2.5.0 + pip install fsspec>=2023.6.0 + pip install soundfile==0.12.1 + pip install uvicorn==0.24.0.post1 + pip install TTS==0.21.3 + pip install fastapi==0.104.1 + pip install Jinja2==3.1.2 + pip install requests==2.31.0 + pip install tqdm==4.66.1 + pip install importlib-metadata==4.8.1 + pip install packaging==23.2 + pip install pydantic==1.10.13 + pip install sounddevice==0.4.6 + pip install python-multipart==0.0.6 + else + pip install -r requirements_other.txt + fi + + # Create start_alltalk.sh to run AllTalk + cat << EOF > start_alltalk.sh +#!/bin/bash +source "${CONDA_ROOT_PREFIX}/etc/profile.d/conda.sh" +conda activate "${INSTALL_ENV_DIR}" +python script.py +EOF + chmod +x start_alltalk.sh + echo + echo + echo -e " start_alltalk.sh has been created." + echo -e " You can now start AllTalk with ${L_YELLOW}./start_alltalk.sh${NC}" + echo + read -p " Press any key to continue. " -n 1 +} + + +delete_custom_standalone() { + local env_dir="$PWD/alltalk_environment" + # Check if the alltalk_environment directory exists + if [ ! -d "$env_dir" ]; then + echo " \"$env_dir\" directory does not exist. No need to delete." + read -p " Press any key to continue. " -n 1 + echo + return + fi + # Check if a Conda environment is active and deactivate it + if [ -n "$CONDA_PREFIX" ]; then + echo " Exiting the Conda environment. You may need to start ./atstart.sh again" + conda deactivate + fi + echo "Deleting \"$env_dir\". Please wait." + rm -rf "$env_dir" + if [ -d "$env_dir" ]; then + echo " Failed to delete \"$env_dir\" folder." + echo " Please make sure it is not in use and try again." + else + echo " Environment \"$env_dir\" deleted successfully." + fi + read -p " Press any key to continue. " -n 1 + echo +} + +install_finetune_standalone() { + local env_dir="$PWD/alltalk_environment" + local conda_root_prefix="${env_dir}/conda" + local install_env_dir="${env_dir}/env" + if [ ! -d "${install_env_dir}" ]; then + echo " The Conda environment at '${install_env_dir}' does not exist." + echo " Please install the environment before proceeding." + read -p " Press any key to return to the menu. " -n 1 + echo + return + fi + source "${conda_root_prefix}/etc/profile.d/conda.sh" + conda activate "${install_env_dir}" + local requirements_file="requirements_finetune.txt" + echo "Installing Finetune requirements from $requirements_file..." + if ! pip install -r "$requirements_file"; then + echo " There was an error installing the Finetune requirements." + echo " Please check the output for details." + read -p " Press any key to return to the menu. " -n 1 + echo + return + fi + echo + echo " Finetune base requirements installed successfully." + echo + echo -e " Finetuning requires access to ${GREEN}Cublas${NC} from the ${GREEN}Nvidia CUDA Toolkit${NC}, so will need installing." + echo + echo -e " Instructions can be found here:" + echo -e " ${GREEN}https://github.com/erew123/alltalk_tts?#-finetuning-a-model${NC}" + echo + read -p " Press any key to continue. " -n 1 + echo +} + +generate_diagnostics_standalone() { + local env_dir="$PWD/alltalk_environment" + local conda_root_prefix="${env_dir}/conda" + local install_env_dir="${env_dir}/env" + if [ ! -d "${install_env_dir}" ]; then + echo " The Conda environment at '${install_env_dir}' does not exist." + echo " Please install the environment before proceeding." + read -p " Press any key to return to the menu. " -n 1 + echo + return + fi + source "${conda_root_prefix}/etc/profile.d/conda.sh" + conda activate "${install_env_dir}" + if ! python diagnostics.py; then + echo " There was an error running diagnostics." + read -p " Press any key to return to the menu. " -n 1 + echo + return + fi + echo " Diagnostics completed successfully." + read -p " Press any key to continue. " -n 1 + echo +} + + +# Start the main menu +main_menu diff --git a/diagnostics.py b/diagnostics.py index ca5466a..eca4ad4 100644 --- a/diagnostics.py +++ b/diagnostics.py @@ -215,6 +215,13 @@ def log_system_info(): # Print colored output print(f"{package_name.ljust(max_package_length)} Required: {color_required}{operator} {required_version.ljust(12)}\033[0m Installed: {color_installed}{installed_version}\033[0m") + + print("\nOn Nvidia Graphics cards machines, if your \033[92mInstalled\033[0m version of \033[92mTorch\033[0m and \033[92mTorchaudio\033[0m does") + print("not have \033[92m+cu118\033[0m (Cuda 11.8) or \033[92m+cu121\033[0m (Cuda 12.1) listed after them, you do not have CUDA") + print("installed for Torch or Torchaudio in this Python environment. This will cause you problems") + print("with \033[94mAllTalk\033[0m and \033[94mFinetuning.\033[0m You may have to 'pip install' a new version of torch and") + print("torchaudio, using '\033[94m--upgrade --force-reinstall\033[0m' with the correct version of PyTorch for\033[0m") + print("your Python environment.\033[0m") print("\033[94m\nRequirements file specifier meanings:\033[0m") explanation = textwrap.dedent(""" == Exact version != Any version except < Less than diff --git a/script.py b/script.py index 81b1f96..4e4ff35 100644 --- a/script.py +++ b/script.py @@ -237,10 +237,12 @@ process = subprocess.Popen(["python", script_path]) # Check if the subprocess has started successfully if process.poll() is None: print(f"[{params['branding']}Startup] TTS Subprocess starting") + print(f"[{params['branding']}Startup]") print( - f"[{params['branding']}Startup] Readme available here:", + f"[{params['branding']}Startup] Settings & Documentation:", f"http://{params['ip_address']}:{params['port_number']}", ) + print(f"[{params['branding']}Startup]") else: print( f"[{params['branding']}Startup] \033[91mWarning\033[0m TTS Subprocess Webserver failing to start process" @@ -862,7 +864,8 @@ def ui(): voice.change(lambda x: params.update({"voice": x}), voice, None) language.change(lambda x: params.update({"language": x}), language, None) - # TTS Settings (Not yet parsed to api/implemented) + + # TS Settings (Not yet parsed to api/implemented) # local_temperature_gr.change(lambda x: params.update({"local_temperature": x}), local_temperature_gr, None) # local_repetition_penalty_gr.change(lambda x: params.update({"local_repetition_penalty": x}), local_repetition_penalty_gr, None) @@ -885,4 +888,4 @@ if running_in_standalone: try: time.sleep(1) # Add a small delay to avoid high CPU usage except KeyboardInterrupt: - break # Allow graceful exit on Ctrl+C + break # Allow graceful exit on Ctrl+C \ No newline at end of file diff --git a/templates/STfiles/alltalk.js b/templates/STfiles/alltalk.js new file mode 100644 index 0000000..0d29c6f --- /dev/null +++ b/templates/STfiles/alltalk.js @@ -0,0 +1,930 @@ +import { doExtrasFetch, getApiUrl, modules } from '../../extensions.js'; +import { saveTtsProviderSettings } from './index.js'; + +export { AllTalkTtsProvider }; + +const css = ` +.settings-row { + display: flex; + justify-content: space-between; /* Distribute space evenly between the children */ + align-items: center; + width: 100%; /* Full width to spread out the children */ +} + +.settings-option { + flex: 1; /* Each child will take equal space */ + margin: 0 10px; /* Adjust spacing as needed */ +} + +.endpoint-option { + flex: 1; /* Each child will take equal space */ + margin: 0 10px; /* Adjust spacing as needed */ + margin-right: 25px; + width: 38%; /* Full width to spread out the children */ +} + +.website-row { + display: flex; + justify-content: start; /* Aligns items to the start of the row */ + align-items: center; + margin-top: 10px; /* Top margin */ + margin-bottom: 10px; /* Bottom margin */ +} + +.website-option { + flex: 1; /* Allows each option to take equal space */ + margin-right: 10px; /* Spacing between the two options, adjust as needed */ + margin-left: 10px; /* Adjust this value to align as desired */ + /* Remove the individual margin top and bottom declarations if they are set globally on the .settings-row */ +} + +.settings-separator { + margin-top: 10px; + margin-bottom: 10px; + padding: 18x; + font-weight: bold; + border-top: 1px solid #e1e1e1; /* Grey line */ + border-bottom: 1px solid #e1e1e1; /* Grey line */ + text-align: center; +} + +.status-message { + flex: 1; /* Adjust as needed */ + margin: 0 10px; + /* Additional styling */ +} + +.model-endpoint-row { + display: flex; + justify-content: space-between; + align-items: center; + width: 100%; +} + +.model-option, .endpoint-option { + flex: 1; + margin: 0 10px; + margin-left: 10px; /* Adjust this value to align as desired */ + /* Adjust width as needed to fit in one line */ +} + +.endpoint-option { + /* Adjust width to align with model dropdown */ + width: 38%; +} + +#status_info { + color: lightgreen; /* Or any specific shade of green you prefer */ +} +`; + +const style = document.createElement('style'); +style.type = 'text/css'; +style.appendChild(document.createTextNode(css)); +document.head.appendChild(style); + +class AllTalkTtsProvider { + //########// + // Config // + //########// + + settings; + ready = false; + voices = []; + separator = '. '; + audioElement = document.createElement('audio'); + + languageLabels = { + 'Arabic': 'ar', + 'Brazilian Portuguese': 'pt', + 'Chinese': 'zh-cn', + 'Czech': 'cs', + 'Dutch': 'nl', + 'English': 'en', + 'French': 'fr', + 'German': 'de', + 'Italian': 'it', + 'Polish': 'pl', + 'Russian': 'ru', + 'Spanish': 'es', + 'Turkish': 'tr', + 'Japanese': 'ja', + 'Korean': 'ko', + 'Hungarian': 'hu', + 'Hindi': 'hi', + }; + + Settings = { + provider_endpoint: 'http://localhost:7851', + language: 'en', + voiceMap: {}, + at_generation_method: 'standard_generation', + narrator_enabled: 'false', + at_narrator_text_not_inside: 'narrator', + narrator_voice_gen: 'female_01.wav', + finetuned_model: 'false' + }; + + get settingsHtml() { + let html = `
AllTalk Settings
`; + + html += `
+ +
+ + +
+
`; + + html += `
+ +
+ + +
+ +
+ + +
+ +
`; + + html += `
+
+ + +
+
+ + +
+
`; + + + html += `
+
+ + +
+ +
+ + +
+
`; + + + html += `
+
+ + +
+
+ + +
+
+ Status: Ready +
+
+ +
+
`; + + + html += `
+
+ AllTalk Config & Docs. +
+ +
+ AllTalk Website. +
+
`; + + return html; + } + + + //#################// + // Startup ST & AT // + //#################// + + async loadSettings(settings) { + updateStatus('Offline'); + // Use default settings if no settings are provided + this.settings = Object.keys(settings).length === 0 ? this.Settings : settings; + + try { + // Check if TTS provider is ready + await this.checkReady(); + await this.fetchTtsVoiceObjects(); // Fetch voices only if service is ready + await this.updateSettingsFromServer(); + this.updateNarratorVoicesDropdown(); + this.updateLanguageDropdown(); + this.setupEventListeners(); + this.applySettingsToHTML(); + updateStatus('Ready'); + } catch (error) { + console.error("Error loading settings:", error); + // Set status to Error if there is an issue in loading settings + updateStatus('Offline'); + } + } + + applySettingsToHTML() { + // Load AllTalk specific settings first + const loadedSettings = loadAllTalkSettings(); + // Apply loaded settings or use defaults + this.settings = loadedSettings ? { ...this.Settings, ...loadedSettings } : this.Settings; + const narratorVoiceSelect = document.getElementById('narrator_voice'); + const atNarratorSelect = document.getElementById('at_narrator_enabled'); + const textNotInsideSelect = document.getElementById('at_narrator_text_not_inside'); + const generationMethodSelect = document.getElementById('at_generation_method'); + // Apply settings to Narrator Voice dropdown + if (narratorVoiceSelect && this.settings.narrator_voice) { + narratorVoiceSelect.value = this.settings.narrator_voice.replace('.wav', ''); + this.settings.narrator_voice_gen = this.settings.narrator_voice; + //console.log(this.settings.narrator_voice_gen) + } + // Apply settings to AT Narrator Enabled dropdown + if (atNarratorSelect) { + //console.log(this.settings.narrator_enabled) + // Sync the state with the checkbox in index.js + const ttsPassAsterisksCheckbox = document.getElementById('tts_pass_asterisks'); // Access the checkbox from index.js + const ttsNarrateQuotedCheckbox = document.getElementById('tts_narrate_quoted'); // Access the checkbox from index.js + const ttsNarrateDialoguesCheckbox = document.getElementById('tts_narrate_dialogues'); // Access the checkbox from index.js + // Sync the state with the checkbox in index.js + if (this.settings.narrator_enabled) { + ttsPassAsterisksCheckbox.checked = false; + $('#tts_pass_asterisks').click(); // Simulate a click event + $('#tts_pass_asterisks').trigger('change'); + } + if (!this.settings.narrator_enabled) { + ttsPassAsterisksCheckbox.checked = true; + $('#tts_pass_asterisks').click(); // Simulate a click event + $('#tts_pass_asterisks').trigger('change'); + } + // Uncheck and set tts_narrate_quoted to false if narrator is enabled + if (this.settings.narrator_enabledd) { + ttsNarrateQuotedCheckbox.checked = true; + ttsNarrateDialoguesCheckbox.checked = true; + // Trigger click events instead of change events + $('#tts_narrate_quoted').click(); + $('#tts_narrate_quoted').trigger('change'); + $('#tts_narrate_dialogues').click(); + $('#tts_narrate_dialogues').trigger('change'); + } + atNarratorSelect.value = this.settings.narrator_enabled.toString(); + this.settings.narrator_enabled = this.settings.narrator_enabled.toString(); + } + // Apply settings to the Language dropdown + const languageSelect = document.getElementById('language_options'); + if (languageSelect && this.settings.language) { + languageSelect.value = this.settings.language; + //console.log(this.settings.language) + } + // Apply settings to Text Not Inside dropdown + if (textNotInsideSelect && this.settings.text_not_inside) { + textNotInsideSelect.value = this.settings.text_not_inside; + this.settings.at_narrator_text_not_inside = this.settings.text_not_inside; + //console.log(this.settings.at_narrator_text_not_inside) + } + // Apply settings to Generation Method dropdown + if (generationMethodSelect && this.settings.at_generation_method) { + generationMethodSelect.value = this.settings.at_generation_method; + //console.log(this.settings.at_generation_method) + } + // Additional logic to disable/enable dropdowns based on the selected generation method + const isStreamingEnabled = this.settings.at_generation_method === 'streaming_enabled'; + if (isStreamingEnabled) { + // Disable certain dropdowns when streaming is enabled + if (atNarratorSelect) atNarratorSelect.disabled = true; + if (textNotInsideSelect) textNotInsideSelect.disabled = true; + if (narratorVoiceSelect) narratorVoiceSelect.disabled = true; + } else { + // Enable dropdowns for standard generation + if (atNarratorSelect) atNarratorSelect.disabled = false; + if (textNotInsideSelect) textNotInsideSelect.disabled = !this.settings.narrator_enabled; + if (narratorVoiceSelect) narratorVoiceSelect.disabled = !this.settings.narrator_enabled; + } + const modelSelect = document.getElementById('switch_model'); + if (this.settings.finetuned_model === 'true') { + const ftOption = document.createElement('option'); + ftOption.value = 'XTTSv2 FT'; + ftOption.textContent = 'XTTSv2 FT'; + modelSelect.appendChild(ftOption); + } + //console.log("Settings applied to HTML."); + } + + //##############################// + // Check AT Server is Available // + //##############################// + + async checkReady() { + try { + const response = await fetch(`${this.settings.provider_endpoint}/api/ready`); + // Check if the HTTP request was successful + if (!response.ok) { + throw new Error(`HTTP Error Response: ${response.status} ${response.statusText}`); + } + const statusText = await response.text(); + // Check if the response is 'Ready' + if (statusText === 'Ready') { + this.ready = true; // Set the ready flag to true + console.log('TTS service is ready.'); + } else { + this.ready = false; + console.log('TTS service is not ready.'); + } + } catch (error) { + console.error('Error checking TTS service readiness:', error); + this.ready = false; // Ensure ready flag is set to false in case of error + } + } + + //######################// + // Get Available Voices // + //######################// + + async fetchTtsVoiceObjects() { + const response = await fetch(`${this.settings.provider_endpoint}/api/voices`); + if (!response.ok) { + const errorText = await response.text(); + throw new Error(`HTTP ${response.status}: ${errorText}`); + } + const data = await response.json(); + const voices = data.voices.map(filename => { + const voiceName = filename.replace('.wav', ''); + return { + name: voiceName, + voice_id: voiceName, + preview_url: null, // Preview URL will be dynamically generated + lang: 'en' // Default language + }; + }); + this.voices = voices; // Assign to the class property + return voices; // Also return this list + } + + //##########################################// + // Get Current AT Server Config & Update ST // + //##########################################// + + async updateSettingsFromServer() { + try { + // Fetch current settings + const response = await fetch(`${this.settings.provider_endpoint}/api/currentsettings`); + if (!response.ok) { + throw new Error(`Failed to fetch current settings: ${response.statusText}`); + } + const currentSettings = await response.json(); + // Update internal settings + this.settings.modelsAvailable = currentSettings.models_available; + this.settings.currentModel = currentSettings.current_model_loaded; + this.settings.deepspeed_available = currentSettings.deepspeed_available; + this.settings.deepSpeedEnabled = currentSettings.deepspeed_status; + this.settings.lowVramEnabled = currentSettings.low_vram_status; + this.settings.finetuned_model = currentSettings.finetuned_model + // Update HTML elements + this.updateModelDropdown(); + this.updateCheckboxes(); + } catch (error) { + console.error(`Error updating settings from server: ${error}`); + } + } + + //###################################################// + // Get Current AT Server Config & Update ST (Models) // + //###################################################// + + updateModelDropdown() { + const modelSelect = document.getElementById('switch_model'); + if (modelSelect) { + modelSelect.innerHTML = ''; // Clear existing options + this.settings.modelsAvailable.forEach(model => { + const option = document.createElement('option'); + option.value = model.model_name; + option.textContent = model.model_name; // Use model_name instead of name + option.selected = model.model_name === this.settings.currentModel; + modelSelect.appendChild(option); + }); + } + } + + //#######################################################// + // Get Current AT Server Config & Update ST (DS and LVR) // + //#######################################################// + + updateCheckboxes() { + const deepspeedCheckbox = document.getElementById('deepspeed'); + const lowVramCheckbox = document.getElementById('low_vram'); + if (lowVramCheckbox) lowVramCheckbox.checked = this.settings.lowVramEnabled; + if (deepspeedCheckbox) { + deepspeedCheckbox.checked = this.settings.deepSpeedEnabled; + deepspeedCheckbox.disabled = !this.settings.deepspeed_available; // Disable checkbox if deepspeed is not available + } + } + + //###############################################################// + // Get Current AT Server Config & Update ST (AT Narrator Voices) // + //###############################################################// + + updateNarratorVoicesDropdown() { + const narratorVoiceSelect = document.getElementById('narrator_voice'); + if (narratorVoiceSelect && this.voices) { + // Clear existing options + narratorVoiceSelect.innerHTML = ''; + // Add new options + for (let voice of this.voices) { + const option = document.createElement('option'); + option.value = voice.voice_id; + option.textContent = voice.name; + narratorVoiceSelect.appendChild(option); + } + } + } + + //######################################################// + // Get Current AT Server Config & Update ST (Languages) // + //######################################################// + + updateLanguageDropdown() { + const languageSelect = document.getElementById('language_options'); + if (languageSelect) { + // Ensure default language is set + this.settings.language = this.Settings.language; + + languageSelect.innerHTML = ''; + for (let language in this.languageLabels) { + const option = document.createElement('option'); + option.value = this.languageLabels[language]; + option.textContent = language; + if (this.languageLabels[language] === this.settings.language) { + option.selected = true; + } + languageSelect.appendChild(option); + } + } + } + + //########################################// + // Start AT TTS extenstion page listeners // + //########################################// + + setupEventListeners() { + + let debounceTimeout; + const debounceDelay = 500; // Milliseconds + + // Define the event handler function + const onModelSelectChange = async (event) => { + console.log("Model select change event triggered"); // Debugging statement + const selectedModel = event.target.value; + console.log(`Selected model: ${selectedModel}`); // Debugging statement + // Set status to Processing + updateStatus('Processing'); + try { + const response = await fetch(`${this.settings.provider_endpoint}/api/reload?tts_method=${encodeURIComponent(selectedModel)}`, { + method: 'POST' + }); + if (!response.ok) { + throw new Error(`HTTP Error: ${response.status}`); + } + const data = await response.json(); + console.log("POST response data:", data); // Debugging statement + // Set status to Ready if successful + updateStatus('Ready'); + } catch (error) { + console.error("POST request error:", error); // Debugging statement + // Set status to Error in case of failure + updateStatus('Error'); + } + + // Handle response or error + }; + + const debouncedModelSelectChange = (event) => { + clearTimeout(debounceTimeout); + debounceTimeout = setTimeout(() => { + onModelSelectChange(event); + }, debounceDelay); + }; + + // Switch Model Listener + const modelSelect = document.getElementById('switch_model'); + if (modelSelect) { + // Remove the event listener if it was previously added + modelSelect.removeEventListener('change', debouncedModelSelectChange); + // Add the debounced event listener + modelSelect.addEventListener('change', debouncedModelSelectChange); + } + + // DeepSpeed Listener + const deepspeedCheckbox = document.getElementById('deepspeed'); + if (deepspeedCheckbox) { + deepspeedCheckbox.addEventListener('change', async (event) => { + const deepSpeedValue = event.target.checked ? 'True' : 'False'; + // Set status to Processing + updateStatus('Processing'); + try { + const response = await fetch(`${this.settings.provider_endpoint}/api/deepspeed?new_deepspeed_value=${deepSpeedValue}`, { + method: 'POST' + }); + if (!response.ok) { + throw new Error(`HTTP Error: ${response.status}`); + } + const data = await response.json(); + console.log("POST response data:", data); // Debugging statement + // Set status to Ready if successful + updateStatus('Ready'); + } catch (error) { + console.error("POST request error:", error); // Debugging statement + // Set status to Error in case of failure + updateStatus('Error'); + } + }); + } + + // Low VRAM Listener + const lowVramCheckbox = document.getElementById('low_vram'); + if (lowVramCheckbox) { + lowVramCheckbox.addEventListener('change', async (event) => { + const lowVramValue = event.target.checked ? 'True' : 'False'; + // Set status to Processing + updateStatus('Processing'); + try { + const response = await fetch(`${this.settings.provider_endpoint}/api/lowvramsetting?new_low_vram_value=${lowVramValue}`, { + method: 'POST' + }); + if (!response.ok) { + throw new Error(`HTTP Error: ${response.status}`); + } + const data = await response.json(); + console.log("POST response data:", data); // Debugging statement + // Set status to Ready if successful + updateStatus('Ready'); + } catch (error) { + console.error("POST request error:", error); // Debugging statement + // Set status to Error in case of failure + updateStatus('Error'); + } + }); + } + + // Narrator Voice Dropdown Listener + const narratorVoiceSelect = document.getElementById('narrator_voice'); + if (narratorVoiceSelect) { + narratorVoiceSelect.addEventListener('change', (event) => { + this.settings.narrator_voice_gen = `${event.target.value}.wav`; + this.onSettingsChangeAllTalk(); // Save the settings after change + }); + } + + const textNotInsideSelect = document.getElementById('at_narrator_text_not_inside'); + if (textNotInsideSelect) { + textNotInsideSelect.addEventListener('change', (event) => { + this.settings.text_not_inside = event.target.value; + this.onSettingsChangeAllTalk(); // Save the settings after change + }); + } + + // AT Narrator Dropdown Listener + const atNarratorSelect = document.getElementById('at_narrator_enabled'); + const ttsPassAsterisksCheckbox = document.getElementById('tts_pass_asterisks'); // Access the checkbox from index.js + const ttsNarrateQuotedCheckbox = document.getElementById('tts_narrate_quoted'); // Access the checkbox from index.js + const ttsNarrateDialoguesCheckbox = document.getElementById('tts_narrate_dialogues'); // Access the checkbox from index.js + + if (atNarratorSelect && textNotInsideSelect && narratorVoiceSelect) { + atNarratorSelect.addEventListener('change', (event) => { + const isNarratorEnabled = event.target.value === 'true'; + this.settings.narrator_enabled = isNarratorEnabled; // Update the setting here + textNotInsideSelect.disabled = !isNarratorEnabled; + narratorVoiceSelect.disabled = !isNarratorEnabled; + + // Sync the state with the checkbox in index.js + if (isNarratorEnabled) { + ttsPassAsterisksCheckbox.checked = false; + $('#tts_pass_asterisks').click(); // Simulate a click event + $('#tts_pass_asterisks').trigger('change'); + } + if (!isNarratorEnabled) { + ttsPassAsterisksCheckbox.checked = true; + $('#tts_pass_asterisks').click(); // Simulate a click event + $('#tts_pass_asterisks').trigger('change'); + } + // Uncheck and set tts_narrate_quoted to false if narrator is enabled + if (isNarratorEnabled) { + ttsNarrateQuotedCheckbox.checked = true; + ttsNarrateDialoguesCheckbox.checked = true; + // Trigger click events instead of change events + $('#tts_narrate_quoted').click(); + $('#tts_narrate_quoted').trigger('change'); + $('#tts_narrate_dialogues').click(); + $('#tts_narrate_dialogues').trigger('change'); + } + this.onSettingsChangeAllTalk(); // Save the settings after change + }); + } + + + + // Event Listener for AT Generation Method Dropdown + const atGenerationMethodSelect = document.getElementById('at_generation_method'); + const atNarratorEnabledSelect = document.getElementById('at_narrator_enabled'); + if (atGenerationMethodSelect) { + atGenerationMethodSelect.addEventListener('change', (event) => { + const selectedMethod = event.target.value; + + if (selectedMethod === 'streaming_enabled') { + // Disable and unselect AT Narrator + atNarratorEnabledSelect.disabled = true; + atNarratorEnabledSelect.value = 'false'; + textNotInsideSelect.disabled = true; + narratorVoiceSelect.disabled = true; + } else if (selectedMethod === 'standard_generation') { + // Enable AT Narrator + atNarratorEnabledSelect.disabled = false; + } + this.settings.at_generation_method = selectedMethod; // Update the setting here + this.onSettingsChangeAllTalk(); // Save the settings after change + }); + } + + // Listener for Language Dropdown + const languageSelect = document.getElementById('language_options'); + if (languageSelect) { + languageSelect.addEventListener('change', (event) => { + this.settings.language = event.target.value; + this.onSettingsChangeAllTalk(); // Save the settings after change + }); + } + } + + //#############################// + // Store ST interface settings // + //#############################// + + onSettingsChange() { + // Update settings that SillyTavern will save + this.settings.provider_endpoint = $('#at_server').val(); + this.settings.language = $('#language_options').val(); + saveTtsProviderSettings(); // This function should save settings handled by SillyTavern + // Call the function to handle AllTalk specific settings + this.onSettingsChangeAllTalk(); // Save the settings after change + } + + onSettingsChangeAllTalk() { + // Update AllTalk specific settings and save to localStorage + this.settings.narrator_enabled = $('#at_narrator_enabled').val() === 'true'; + this.settings.narrator_voice = $('#narrator_voice').val() + '.wav'; // assuming you need to append .wav + this.settings.text_not_inside = $('#at_narrator_text_not_inside').val(); // "character" or "narrator" + this.settings.at_generation_method = $('#at_generation_method').val(); // Streaming or standard + this.settings.language = $('#language_options').val(); // Streaming or standard + + // Call the save function with the current settings + saveAllTalkSettings({ + narrator_voice: this.settings.narrator_voice, + narrator_enabled: this.settings.narrator_enabled, + text_not_inside: this.settings.text_not_inside, + at_generation_method: this.settings.at_generation_method, + language: this.settings.language + }); + } + + //#########################// + // ST Handle Reload button // + //#########################// + + async onRefreshClick() { + await this.checkReady(); // Check if the TTS provider is ready + await this.loadSettings(this.settings); // Reload the settings + // Additional actions as needed + } + + //##################// + // Preview AT Voice // + //##################// + + async previewTtsVoice(voiceName) { + try { + // Prepare data for POST request + const postData = new URLSearchParams(); + postData.append("voice", `${voiceName}.wav`); + // Making the POST request + const response = await fetch(`${this.settings.provider_endpoint}/api/previewvoice/`, { + method: "POST", + headers: { + 'Content-Type': 'application/x-www-form-urlencoded' + }, + body: postData, + }); + if (!response.ok) { + const errorText = await response.text(); + console.error(`[previewTtsVoice] Error Response Text:`, errorText); + throw new Error(`HTTP ${response.status}: ${errorText}`); + } + // Assuming the server returns a URL to the .wav file + const data = await response.json(); + if (data.output_file_url) { + // Use an audio element to play the .wav file + const audioElement = new Audio(data.output_file_url); + audioElement.play().catch(e => console.error("Error playing audio:", e)); + } else { + console.warn("[previewTtsVoice] No output file URL received in the response"); + throw new Error("No output file URL received in the response"); + } + + } catch (error) { + console.error("[previewTtsVoice] Exception caught during preview generation:", error); + throw error; + } + } + + //#####################// + // Populate ST voices // + //#####################// + + async getVoice(voiceName, generatePreview = false) { + // Ensure this.voices is populated + if (this.voices.length === 0) { + // Fetch voice objects logic + } + // Find the object where the name matches voiceName + const match = this.voices.find(voice => voice.name === voiceName); + if (!match) { + // Error handling + } + // Generate preview URL only if requested + if (!match.preview_url && generatePreview) { + // Generate preview logic + } + return match; // Return the found voice object + } + + //##########################################// + // Generate TTS Streaming or call Standard // + //##########################################// + + async generateTts(inputText, voiceId) { + try { + if (this.settings.at_generation_method === 'streaming_enabled') { + // For streaming method + const streamingUrl = `${this.settings.provider_endpoint}/api/tts-generate-streaming?text=${encodeURIComponent(inputText)}&voice=${encodeURIComponent(voiceId)}.wav&language=${encodeURIComponent(this.settings.language)}&output_file=stream_output.wav`; + const audioElement = new Audio(streamingUrl); + audioElement.play(); // Play the audio stream directly + console.log("Streaming URL:", streamingUrl); + return new Response(null, { + status: 200, + statusText: "OK", + headers: { + "Content-Type": "audio/wav", + "Content-Location": streamingUrl + } + }); + } else { + // For standard method + const outputUrl = await this.fetchTtsGeneration(inputText, voiceId); + // Fetch the audio data as a blob from the URL + const audioResponse = await fetch(outputUrl); + if (!audioResponse.ok) { + throw new Error(`HTTP ${audioResponse.status}: Failed to fetch audio data`); + } + const audioBlob = await audioResponse.blob(); + if (!audioBlob.type.startsWith('audio/')) { + throw new Error(`Invalid audio data format. Expecting audio/*, got ${audioBlob.type}`); + } + return new Response(audioBlob, { + status: 200, + statusText: "OK", + headers: { + "Content-Type": audioBlob.type + } + }); + } + } catch (error) { + console.error("Error in generateTts:", error); + throw error; + } + } + + //####################// + // Generate Standard // + //####################// + + async fetchTtsGeneration(inputText, voiceId) { + // Prepare the request payload + const requestBody = new URLSearchParams({ + 'text_input': inputText, + 'text_filtering': "standard", + 'character_voice_gen': voiceId + ".wav", + 'narrator_enabled': this.settings.narrator_enabled, + 'narrator_voice_gen': this.settings.narrator_voice_gen, + 'text_not_inside': this.settings.text_not_inside, + 'language': this.settings.language, + 'output_file_name': "st_output", + 'output_file_timestamp': "true", + 'autoplay': "false", + 'autoplay_volume': "0.8" + }).toString(); + + try { + const response = await doExtrasFetch( + `${this.settings.provider_endpoint}/api/tts-generate`, + { + method: 'POST', + headers: { + 'Content-Type': 'application/x-www-form-urlencoded', + 'Cache-Control': 'no-cache', + }, + body: requestBody + } + ); + + if (!response.ok) { + const errorText = await response.text(); + console.error(`[fetchTtsGeneration] Error Response Text:`, errorText); + // Uncomment the following line if you have a UI element for displaying errors + // toastr.error(response.statusText, 'TTS Generation Failed'); + throw new Error(`HTTP ${response.status}: ${errorText}`); + } + const data = await response.json(); + const outputUrl = data.output_file_url; + return outputUrl; // Return only the output_file_url + } catch (error) { + console.error("[fetchTtsGeneration] Exception caught:", error); + throw error; // Rethrow the error for further handling + } + } +} + +//#########################// +// Update Status Messages // +//#########################// + +function updateStatus(message) { + const statusElement = document.getElementById('status_info'); + if (statusElement) { + statusElement.textContent = message; + switch (message) { + case 'Offline': + statusElement.style.color = 'red'; + break; + case 'Ready': + statusElement.style.color = 'lightgreen'; + break; + case 'Processing': + statusElement.style.color = 'blue'; + break; + case 'Error': + statusElement.style.color = 'red'; + break; + // Add more cases as needed + } + } +} + +//########################// +// Save/load AT Settings // +//########################// + +function saveAllTalkSettings(settingsToSave) { + // Save the specific settings to localStorage + console.log("settings", settingsToSave) + localStorage.setItem('AllTalkSettings', JSON.stringify(settingsToSave)); +} + +function loadAllTalkSettings() { + // Retrieve the settings from localStorage + const settings = localStorage.getItem('AllTalkSettings'); + // If settings exist, parse them back into an object + if (settings) { + return JSON.parse(settings); + } + return null; +} diff --git a/templates/STfiles/index.js b/templates/STfiles/index.js new file mode 100644 index 0000000..d07aea6 --- /dev/null +++ b/templates/STfiles/index.js @@ -0,0 +1,1028 @@ +import { callPopup, cancelTtsPlay, eventSource, event_types, name2, saveSettingsDebounced } from '../../../script.js'; +import { ModuleWorkerWrapper, doExtrasFetch, extension_settings, getApiUrl, getContext, modules } from '../../extensions.js'; +import { delay, escapeRegex, getStringHash } from '../../utils.js'; +import { EdgeTtsProvider } from './edge.js'; +import { ElevenLabsTtsProvider } from './elevenlabs.js'; +import { SileroTtsProvider } from './silerotts.js'; +import { CoquiTtsProvider } from './coqui.js'; +import { SystemTtsProvider } from './system.js'; +import { NovelTtsProvider } from './novel.js'; +import { power_user } from '../../power-user.js'; +import { registerSlashCommand } from '../../slash-commands.js'; +import { OpenAITtsProvider } from './openai.js'; +import { XTTSTtsProvider } from './xtts.js'; +import { AllTalkTtsProvider } from './alltalk.js'; +export { talkingAnimation }; + +const UPDATE_INTERVAL = 1000; + +let voiceMapEntries = []; +let voiceMap = {}; // {charName:voiceid, charName2:voiceid2} +let storedvalue = false; +let lastChatId = null; +let lastMessageHash = null; + +const DEFAULT_VOICE_MARKER = '[Default Voice]'; +const DISABLED_VOICE_MARKER = 'disabled'; + +export function getPreviewString(lang) { + const previewStrings = { + 'en-US': 'The quick brown fox jumps over the lazy dog', + 'en-GB': 'Sphinx of black quartz, judge my vow', + 'fr-FR': 'Portez ce vieux whisky au juge blond qui fume', + 'de-DE': 'Victor jagt zwölf Boxkämpfer quer über den großen Sylter Deich', + 'it-IT': 'Pranzo d\'acqua fa volti sghembi', + 'es-ES': 'Quiere la boca exhausta vid, kiwi, piña y fugaz jamón', + 'es-MX': 'Fabio me exige, sin tapujos, que añada cerveza al whisky', + 'ru-RU': 'В чащах юга жил бы цитрус? Да, но фальшивый экземпляр!', + 'pt-BR': 'Vejo xá gritando que fez show sem playback.', + 'pt-PR': 'Todo pajé vulgar faz boquinha sexy com kiwi.', + 'uk-UA': 'Фабрикуймо гідність, лящім їжею, ґав хапаймо, з\'єднавці чаш!', + 'pl-PL': 'Pchnąć w tę łódź jeża lub ośm skrzyń fig', + 'cs-CZ': 'Příliš žluťoučký kůň úpěl ďábelské ódy', + 'sk-SK': 'Vyhŕňme si rukávy a vyprážajme čínske ryžové cestoviny', + 'hu-HU': 'Árvíztűrő tükörfúrógép', + 'tr-TR': 'Pijamalı hasta yağız şoföre çabucak güvendi', + 'nl-NL': 'De waard heeft een kalfje en een pinkje opgegeten', + 'sv-SE': 'Yxskaftbud, ge vårbygd, zinkqvarn', + 'da-DK': 'Quizdeltagerne spiste jordbær med fløde, mens cirkusklovnen Walther spillede på xylofon', + 'ja-JP': 'いろはにほへと　ちりぬるを　わかよたれそ　つねならむ　うゐのおくやま　けふこえて　あさきゆめみし　ゑひもせす', + 'ko-KR': '가나다라마바사아자차카타파하', + 'zh-CN': '我能吞下玻璃而不伤身体', + 'ro-RO': 'Muzicologă în bej vând whisky și tequila, preț fix', + 'bg-BG': 'Щъркелите се разпръснаха по цялото небе', + 'el-GR': 'Ταχίστη αλώπηξ βαφής ψημένη γη, δρασκελίζει υπέρ νωθρού κυνός', + 'fi-FI': 'Voi veljet, miksi juuri teille myin nämä vehkeet?', + 'he-IL': 'הקצינים צעקו: "כל הכבוד לצבא הצבאות!"', + 'id-ID': 'Jangkrik itu memang enak, apalagi kalau digoreng', + 'ms-MY': 'Muzik penyanyi wanita itu menggambarkan kehidupan yang penuh dengan duka nestapa', + 'th-TH': 'เป็นไงบ้างครับ ผมชอบกินข้าวผัดกระเพราหมูกรอบ', + 'vi-VN': 'Cô bé quàng khăn đỏ đang ngồi trên bãi cỏ xanh', + 'ar-SA': 'أَبْجَدِيَّة عَرَبِيَّة', + 'hi-IN': 'श्वेता ने श्वेता के श्वेते हाथों में श्वेता का श्वेता चावल पकड़ा', + }; + const fallbackPreview = 'Neque porro quisquam est qui dolorem ipsum quia dolor sit amet'; + + return previewStrings[lang] ?? fallbackPreview; +} + +let ttsProviders = { + ElevenLabs: ElevenLabsTtsProvider, + Silero: SileroTtsProvider, + XTTSv2: XTTSTtsProvider, + System: SystemTtsProvider, + Coqui: CoquiTtsProvider, + Edge: EdgeTtsProvider, + Novel: NovelTtsProvider, + OpenAI: OpenAITtsProvider, + AllTalk: AllTalkTtsProvider, +}; +let ttsProvider; +let ttsProviderName; + +let ttsLastMessage = null; + +async function onNarrateOneMessage() { + audioElement.src = '/sounds/silence.mp3'; + const context = getContext(); + const id = $(this).closest('.mes').attr('mesid'); + const message = context.chat[id]; + + if (!message) { + return; + } + + resetTtsPlayback(); + ttsJobQueue.push(message); + moduleWorker(); +} + +async function onNarrateText(args, text) { + if (!text) { + return; + } + + audioElement.src = '/sounds/silence.mp3'; + + // To load all characters in the voice map, set unrestricted to true + await initVoiceMap(true); + + const baseName = args?.voice || name2; + const name = (baseName === 'SillyTavern System' ? DEFAULT_VOICE_MARKER : baseName) || DEFAULT_VOICE_MARKER; + + const voiceMapEntry = voiceMap[name] === DEFAULT_VOICE_MARKER + ? voiceMap[DEFAULT_VOICE_MARKER] + : voiceMap[name]; + + if (!voiceMapEntry || voiceMapEntry === DISABLED_VOICE_MARKER) { + toastr.info(`Specified voice for ${name} was not found. Check the TTS extension settings.`); + return; + } + + resetTtsPlayback(); + ttsJobQueue.push({ mes: text, name: name }); + await moduleWorker(); + + // Return back to the chat voices + await initVoiceMap(false); +} + +async function moduleWorker() { + // Primarily determining when to add new chat to the TTS queue + const enabled = $('#tts_enabled').is(':checked'); + $('body').toggleClass('tts', enabled); + if (!enabled) { + return; + } + + const context = getContext(); + const chat = context.chat; + + processTtsQueue(); + processAudioJobQueue(); + updateUiAudioPlayState(); + + // Auto generation is disabled + if (extension_settings.tts.auto_generation == false) { + return; + } + + // no characters or group selected + if (!context.groupId && context.characterId === undefined) { + return; + } + + // Chat changed + if ( + context.chatId !== lastChatId + ) { + currentMessageNumber = context.chat.length ? context.chat.length : 0; + saveLastValues(); + + // Force to speak on the first message in the new chat + if (context.chat.length === 1) { + lastMessageHash = -1; + } + + return; + } + + // take the count of messages + let lastMessageNumber = context.chat.length ? context.chat.length : 0; + + // There's no new messages + let diff = lastMessageNumber - currentMessageNumber; + let hashNew = getStringHash((chat.length && chat[chat.length - 1].mes) ?? ''); + + // if messages got deleted, diff will be < 0 + if (diff < 0) { + // necessary actions will be taken by the onChatDeleted() handler + return; + } + + // if no new messages, or same message, or same message hash, do nothing + if (diff == 0 && hashNew === lastMessageHash) { + return; + } + + // If streaming, wait for streaming to finish before processing new messages + if (context.streamingProcessor && !context.streamingProcessor.isFinished) { + return; + } + + // clone message object, as things go haywire if message object is altered below (it's passed by reference) + const message = structuredClone(chat[chat.length - 1]); + + // if last message within current message, message got extended. only send diff to TTS. + if (ttsLastMessage !== null && message.mes.indexOf(ttsLastMessage) !== -1) { + let tmp = message.mes; + message.mes = message.mes.replace(ttsLastMessage, ''); + ttsLastMessage = tmp; + } else { + ttsLastMessage = message.mes; + } + + // We're currently swiping. Don't generate voice + if (!message || message.mes === '...' || message.mes === '') { + return; + } + + // Don't generate if message doesn't have a display text + if (extension_settings.tts.narrate_translated_only && !(message?.extra?.display_text)) { + return; + } + + // Don't generate if message is a user message and user message narration is disabled + if (message.is_user && !extension_settings.tts.narrate_user) { + return; + } + + // New messages, add new chat to history + lastMessageHash = hashNew; + currentMessageNumber = lastMessageNumber; + + console.debug( + `Adding message from ${message.name} for TTS processing: "${message.mes}"`, + ); + ttsJobQueue.push(message); +} + +function talkingAnimation(switchValue) { + if (!modules.includes('talkinghead')) { + console.debug('Talking Animation module not loaded'); + return; + } + + const apiUrl = getApiUrl(); + const animationType = switchValue ? 'start' : 'stop'; + + if (switchValue !== storedvalue) { + try { + console.log(animationType + ' Talking Animation'); + doExtrasFetch(`${apiUrl}/api/talkinghead/${animationType}_talking`); + storedvalue = switchValue; // Update the storedvalue to the current switchValue + } catch (error) { + // Handle the error here or simply ignore it to prevent logging + } + } + updateUiAudioPlayState(); +} + +function resetTtsPlayback() { + // Stop system TTS utterance + cancelTtsPlay(); + + // Clear currently processing jobs + currentTtsJob = null; + currentAudioJob = null; + + // Reset audio element + audioElement.currentTime = 0; + audioElement.src = ''; + + // Clear any queue items + ttsJobQueue.splice(0, ttsJobQueue.length); + audioJobQueue.splice(0, audioJobQueue.length); + + // Set audio ready to process again + audioQueueProcessorReady = true; +} + +function isTtsProcessing() { + let processing = false; + + // Check job queues + if (ttsJobQueue.length > 0 || audioJobQueue.length > 0) { + processing = true; + } + // Check current jobs + if (currentTtsJob != null || currentAudioJob != null) { + processing = true; + } + return processing; +} + +function debugTtsPlayback() { + console.log(JSON.stringify( + { + 'ttsProviderName': ttsProviderName, + 'voiceMap': voiceMap, + 'currentMessageNumber': currentMessageNumber, + 'audioPaused': audioPaused, + 'audioJobQueue': audioJobQueue, + 'currentAudioJob': currentAudioJob, + 'audioQueueProcessorReady': audioQueueProcessorReady, + 'ttsJobQueue': ttsJobQueue, + 'currentTtsJob': currentTtsJob, + 'ttsConfig': extension_settings.tts, + }, + )); +} +window.debugTtsPlayback = debugTtsPlayback; + +//##################// +// Audio Control // +//##################// + +let audioElement = new Audio(); +audioElement.id = 'tts_audio'; +audioElement.autoplay = true; + +let audioJobQueue = []; +let currentAudioJob; +let audioPaused = false; +let audioQueueProcessorReady = true; + +async function playAudioData(audioBlob) { + // Since current audio job can be cancelled, don't playback if it is null + if (currentAudioJob == null) { + console.log('Cancelled TTS playback because currentAudioJob was null'); + } + const reader = new FileReader(); + reader.onload = function (e) { + const srcUrl = e.target.result; + audioElement.src = srcUrl; + }; + reader.readAsDataURL(audioBlob); + audioElement.addEventListener('ended', completeCurrentAudioJob); + audioElement.addEventListener('canplay', () => { + console.debug('Starting TTS playback'); + audioElement.play(); + }); +} + +window['tts_preview'] = function (id) { + const audio = document.getElementById(id); + + if (audio && !$(audio).data('disabled')) { + audio.play(); + } + else { + ttsProvider.previewTtsVoice(id); + } +}; + +async function onTtsVoicesClick() { + let popupText = ''; + + try { + const voiceIds = await ttsProvider.fetchTtsVoiceObjects(); + + for (const voice of voiceIds) { + popupText += ` +
+ ${voice.lang || ''} + ${voice.name} + +
`; + if (voice.preview_url) { + popupText += ``; + } + } + } catch { + popupText = 'Could not load voices list. Check your API key.'; + } + + callPopup(popupText, 'text'); +} + +function updateUiAudioPlayState() { + if (extension_settings.tts.enabled == true) { + $('#ttsExtensionMenuItem').show(); + let img; + // Give user feedback that TTS is active by setting the stop icon if processing or playing + if (!audioElement.paused || isTtsProcessing()) { + img = 'fa-solid fa-stop-circle extensionsMenuExtensionButton'; + } else { + img = 'fa-solid fa-circle-play extensionsMenuExtensionButton'; + } + $('#tts_media_control').attr('class', img); + } else { + $('#ttsExtensionMenuItem').hide(); + } +} + +function onAudioControlClicked() { + audioElement.src = '/sounds/silence.mp3'; + let context = getContext(); + // Not pausing, doing a full stop to anything TTS is doing. Better UX as pause is not as useful + if (!audioElement.paused || isTtsProcessing()) { + resetTtsPlayback(); + talkingAnimation(false); + } else { + // Default play behavior if not processing or playing is to play the last message. + ttsJobQueue.push(context.chat[context.chat.length - 1]); + } + updateUiAudioPlayState(); +} + +function addAudioControl() { + + $('#extensionsMenu').prepend(` +
+
+ TTS Playback +
`); + $('#ttsExtensionMenuItem').attr('title', 'TTS play/pause').on('click', onAudioControlClicked); + updateUiAudioPlayState(); +} + +function completeCurrentAudioJob() { + audioQueueProcessorReady = true; + currentAudioJob = null; + talkingAnimation(false); //stop lip animation + // updateUiPlayState(); +} + +/** + * Accepts an HTTP response containing audio/mpeg data, and puts the data as a Blob() on the queue for playback + * @param {Response} response + */ +async function addAudioJob(response) { + const audioData = await response.blob(); + if (!audioData.type.startsWith('audio/')) { + throw `TTS received HTTP response with invalid data format. Expecting audio/*, got ${audioData.type}`; + } + audioJobQueue.push(audioData); + console.debug('Pushed audio job to queue.'); +} + +async function processAudioJobQueue() { + // Nothing to do, audio not completed, or audio paused - stop processing. + if (audioJobQueue.length == 0 || !audioQueueProcessorReady || audioPaused) { + return; + } + try { + audioQueueProcessorReady = false; + currentAudioJob = audioJobQueue.pop(); + playAudioData(currentAudioJob); + talkingAnimation(true); + } catch (error) { + console.error(error); + audioQueueProcessorReady = true; + } +} + +//################// +// TTS Control // +//################// + +let ttsJobQueue = []; +let currentTtsJob; // Null if nothing is currently being processed +let currentMessageNumber = 0; + +function completeTtsJob() { + console.info(`Current TTS job for ${currentTtsJob?.name} completed.`); + currentTtsJob = null; +} + +function saveLastValues() { + const context = getContext(); + lastChatId = context.chatId; + lastMessageHash = getStringHash( + (context.chat.length && context.chat[context.chat.length - 1].mes) ?? '', + ); +} + +async function tts(text, voiceId, char) { + let response = await ttsProvider.generateTts(text, voiceId); + + // RVC injection + if (extension_settings.rvc.enabled && typeof window['rvcVoiceConversion'] === 'function') + response = await window['rvcVoiceConversion'](response, char, text); + + addAudioJob(response); + completeTtsJob(); +} + +async function processTtsQueue() { + // Called each moduleWorker iteration to pull chat messages from queue + if (currentTtsJob || ttsJobQueue.length <= 0 || audioPaused) { + return; + } + + console.debug('New message found, running TTS'); + currentTtsJob = ttsJobQueue.shift(); + let text = extension_settings.tts.narrate_translated_only ? (currentTtsJob?.extra?.display_text || currentTtsJob.mes) : currentTtsJob.mes; + + if (extension_settings.tts.skip_codeblocks) { + text = text.replace(/^\s{4}.*$/gm, '').trim(); + text = text.replace(/```.*?```/gs, '').trim(); + } + + if (!extension_settings.tts.pass_asterisks) { + text = extension_settings.tts.narrate_dialogues_only + ? text.replace(/\*[^*]*?(\*|$)/g, '').trim() // remove asterisks content + : text.replaceAll('*', '').trim(); // remove just the asterisks + } + + if (extension_settings.tts.narrate_quoted_only) { + const special_quotes = /[“”]/g; // Extend this regex to include other special quotes + text = text.replace(special_quotes, '"'); + const matches = text.match(/".*?"/g); // Matches text inside double quotes, non-greedily + const partJoiner = (ttsProvider?.separator || ' ... '); + text = matches ? matches.join(partJoiner) : text; + } + + if (typeof ttsProvider?.processText === 'function') { + text = await ttsProvider.processText(text); + } + + // Collapse newlines and spaces into single space + text = text.replace(/\s+/g, ' ').trim(); + + console.log(`TTS: ${text}`); + const char = currentTtsJob.name; + + // Remove character name from start of the line if power user setting is disabled + if (char && !power_user.allow_name2_display) { + const escapedChar = escapeRegex(char); + text = text.replace(new RegExp(`^${escapedChar}:`, 'gm'), ''); + } + + try { + if (!text) { + console.warn('Got empty text in TTS queue job.'); + completeTtsJob(); + return; + } + + const voiceMapEntry = voiceMap[char] === DEFAULT_VOICE_MARKER ? voiceMap[DEFAULT_VOICE_MARKER] : voiceMap[char]; + + if (!voiceMapEntry || voiceMapEntry === DISABLED_VOICE_MARKER) { + throw `${char} not in voicemap. Configure character in extension settings voice map`; + } + const voice = await ttsProvider.getVoice(voiceMapEntry); + const voiceId = voice.voice_id; + if (voiceId == null) { + toastr.error(`Specified voice for ${char} was not found. Check the TTS extension settings.`); + throw `Unable to attain voiceId for ${char}`; + } + tts(text, voiceId, char); + } catch (error) { + console.error(error); + currentTtsJob = null; + } +} + +// Secret function for now +async function playFullConversation() { + const context = getContext(); + const chat = context.chat; + ttsJobQueue = chat; +} +window.playFullConversation = playFullConversation; + +//#############################// +// Extension UI and Settings // +//#############################// + +function loadSettings() { + if (Object.keys(extension_settings.tts).length === 0) { + Object.assign(extension_settings.tts, defaultSettings); + } + for (const key in defaultSettings) { + if (!(key in extension_settings.tts)) { + extension_settings.tts[key] = defaultSettings[key]; + } + } + $('#tts_provider').val(extension_settings.tts.currentProvider); + $('#tts_enabled').prop( + 'checked', + extension_settings.tts.enabled, + ); + $('#tts_narrate_dialogues').prop('checked', extension_settings.tts.narrate_dialogues_only); + $('#tts_narrate_quoted').prop('checked', extension_settings.tts.narrate_quoted_only); + $('#tts_auto_generation').prop('checked', extension_settings.tts.auto_generation); + $('#tts_narrate_translated_only').prop('checked', extension_settings.tts.narrate_translated_only); + $('#tts_narrate_user').prop('checked', extension_settings.tts.narrate_user); + $('#tts_pass_asterisks').prop('checked', extension_settings.tts.pass_asterisks); + $('body').toggleClass('tts', extension_settings.tts.enabled); +} + +const defaultSettings = { + voiceMap: '', + ttsEnabled: false, + currentProvider: 'ElevenLabs', + auto_generation: true, + narrate_user: false, +}; + +function setTtsStatus(status, success) { + $('#tts_status').text(status); + if (success) { + $('#tts_status').removeAttr('style'); + } else { + $('#tts_status').css('color', 'red'); + } +} + +function onRefreshClick() { + Promise.all([ + ttsProvider.onRefreshClick(), + // updateVoiceMap() + ]).then(() => { + extension_settings.tts[ttsProviderName] = ttsProvider.settings; + saveSettingsDebounced(); + setTtsStatus('Successfully applied settings', true); + console.info(`Saved settings ${ttsProviderName} ${JSON.stringify(ttsProvider.settings)}`); + initVoiceMap(); + updateVoiceMap(); + }).catch(error => { + console.error(error); + setTtsStatus(error, false); + }); +} + +function onEnableClick() { + extension_settings.tts.enabled = $('#tts_enabled').is( + ':checked', + ); + updateUiAudioPlayState(); + saveSettingsDebounced(); +} + + +function onAutoGenerationClick() { + extension_settings.tts.auto_generation = !!$('#tts_auto_generation').prop('checked'); + saveSettingsDebounced(); +} + + +function onNarrateDialoguesClick() { + extension_settings.tts.narrate_dialogues_only = !!$('#tts_narrate_dialogues').prop('checked'); + saveSettingsDebounced(); +} + +function onNarrateUserClick() { + extension_settings.tts.narrate_user = !!$('#tts_narrate_user').prop('checked'); + saveSettingsDebounced(); +} + +function onNarrateQuotedClick() { + extension_settings.tts.narrate_quoted_only = !!$('#tts_narrate_quoted').prop('checked'); + saveSettingsDebounced(); +} + + +function onNarrateTranslatedOnlyClick() { + extension_settings.tts.narrate_translated_only = !!$('#tts_narrate_translated_only').prop('checked'); + saveSettingsDebounced(); +} + +function onSkipCodeblocksClick() { + extension_settings.tts.skip_codeblocks = !!$('#tts_skip_codeblocks').prop('checked'); + saveSettingsDebounced(); +} + +function onPassAsterisksClick() { + extension_settings.tts.pass_asterisks = !!$('#tts_pass_asterisks').prop('checked'); + saveSettingsDebounced(); +} + +//##############// +// TTS Provider // +//##############// + +async function loadTtsProvider(provider) { + //Clear the current config and add new config + $('#tts_provider_settings').html(''); + + if (!provider) { + return; + } + + // Init provider references + extension_settings.tts.currentProvider = provider; + ttsProviderName = provider; + ttsProvider = new ttsProviders[provider]; + + // Init provider settings + $('#tts_provider_settings').append(ttsProvider.settingsHtml); + if (!(ttsProviderName in extension_settings.tts)) { + console.warn(`Provider ${ttsProviderName} not in Extension Settings, initiatilizing provider in settings`); + extension_settings.tts[ttsProviderName] = {}; + } + await ttsProvider.loadSettings(extension_settings.tts[ttsProviderName]); + await initVoiceMap(); +} + +function onTtsProviderChange() { + const ttsProviderSelection = $('#tts_provider').val(); + extension_settings.tts.currentProvider = ttsProviderSelection; + loadTtsProvider(ttsProviderSelection); +} + +// Ensure that TTS provider settings are saved to extension settings. +export function saveTtsProviderSettings() { + extension_settings.tts[ttsProviderName] = ttsProvider.settings; + updateVoiceMap(); + saveSettingsDebounced(); + console.info(`Saved settings ${ttsProviderName} ${JSON.stringify(ttsProvider.settings)}`); +} + + +//###################// +// voiceMap Handling // +//###################// + +async function onChatChanged() { + await resetTtsPlayback(); + const voiceMapInit = initVoiceMap(); + await Promise.race([voiceMapInit, delay(1000)]); + ttsLastMessage = null; +} + +async function onChatDeleted() { + const context = getContext(); + + // update internal references to new last message + lastChatId = context.chatId; + currentMessageNumber = context.chat.length ? context.chat.length : 0; + + // compare against lastMessageHash. If it's the same, we did not delete the last chat item, so no need to reset tts queue + let messageHash = getStringHash((context.chat.length && context.chat[context.chat.length - 1].mes) ?? ''); + if (messageHash === lastMessageHash) { + return; + } + lastMessageHash = messageHash; + ttsLastMessage = (context.chat.length && context.chat[context.chat.length - 1].mes) ?? ''; + + // stop any tts playback since message might not exist anymore + await resetTtsPlayback(); +} + +/** + * Get characters in current chat + * @param {boolean} unrestricted - If true, will include all characters in voiceMapEntries, even if they are not in the current chat. + * @returns {string[]} - Array of character names + */ +function getCharacters(unrestricted) { + const context = getContext(); + + if (unrestricted) { + const names = context.characters.map(char => char.name); + names.unshift(DEFAULT_VOICE_MARKER); + return names; + } + + let characters = []; + if (context.groupId === null) { + // Single char chat + characters.push(DEFAULT_VOICE_MARKER); + characters.push(context.name1); + characters.push(context.name2); + } else { + // Group chat + characters.push(DEFAULT_VOICE_MARKER); + characters.push(context.name1); + const group = context.groups.find(group => context.groupId == group.id); + for (let member of group.members) { + // Remove suffix + if (member.endsWith('.png')) { + member = member.slice(0, -4); + } + characters.push(member); + } + } + return characters; +} + +function sanitizeId(input) { + // Remove any non-alphanumeric characters except underscore (_) and hyphen (-) + let sanitized = input.replace(/[^a-zA-Z0-9-_]/g, ''); + + // Ensure first character is always a letter + if (!/^[a-zA-Z]/.test(sanitized)) { + sanitized = 'element_' + sanitized; + } + + return sanitized; +} + +function parseVoiceMap(voiceMapString) { + let parsedVoiceMap = {}; + for (const [charName, voiceId] of voiceMapString + .split(',') + .map(s => s.split(':'))) { + if (charName && voiceId) { + parsedVoiceMap[charName.trim()] = voiceId.trim(); + } + } + return parsedVoiceMap; +} + + + +/** + * Apply voiceMap based on current voiceMapEntries + */ +function updateVoiceMap() { + const tempVoiceMap = {}; + for (const voice of voiceMapEntries) { + if (voice.voiceId === null) { + continue; + } + tempVoiceMap[voice.name] = voice.voiceId; + } + if (Object.keys(tempVoiceMap).length !== 0) { + voiceMap = tempVoiceMap; + console.log(`Voicemap updated to ${JSON.stringify(voiceMap)}`); + } + if (!extension_settings.tts[ttsProviderName].voiceMap) { + extension_settings.tts[ttsProviderName].voiceMap = {}; + } + Object.assign(extension_settings.tts[ttsProviderName].voiceMap, voiceMap); + saveSettingsDebounced(); +} + +class VoiceMapEntry { + name; + voiceId; + selectElement; + constructor(name, voiceId = DEFAULT_VOICE_MARKER) { + this.name = name; + this.voiceId = voiceId; + this.selectElement = null; + } + + addUI(voiceIds) { + let sanitizedName = sanitizeId(this.name); + let defaultOption = this.name === DEFAULT_VOICE_MARKER ? + `` : + ``; + let template = ` +
+ ${this.name} + +
+ `; + $('#tts_voicemap_block').append(template); + + // Populate voice ID select list + for (const voiceId of voiceIds) { + const option = document.createElement('option'); + option.innerText = voiceId.name; + option.value = voiceId.name; + $(`#tts_voicemap_char_${sanitizedName}_voice`).append(option); + } + + this.selectElement = $(`#tts_voicemap_char_${sanitizedName}_voice`); + this.selectElement.on('change', args => this.onSelectChange(args)); + this.selectElement.val(this.voiceId); + } + + onSelectChange(args) { + this.voiceId = this.selectElement.find(':selected').val(); + updateVoiceMap(); + } +} + +/** + * Init voiceMapEntries for character select list. + * @param {boolean} unrestricted - If true, will include all characters in voiceMapEntries, even if they are not in the current chat. + */ +export async function initVoiceMap(unrestricted = false) { + // Gate initialization if not enabled or TTS Provider not ready. Prevents error popups. + const enabled = $('#tts_enabled').is(':checked'); + if (!enabled) { + return; + } + + // Keep errors inside extension UI rather than toastr. Toastr errors for TTS are annoying. + try { + await ttsProvider.checkReady(); + } catch (error) { + const message = `TTS Provider not ready. ${error}`; + setTtsStatus(message, false); + return; + } + + setTtsStatus('TTS Provider Loaded', true); + + // Clear existing voiceMap state + $('#tts_voicemap_block').empty(); + voiceMapEntries = []; + + // Get characters in current chat + const characters = getCharacters(unrestricted); + + // Get saved voicemap from provider settings, handling new and old representations + let voiceMapFromSettings = {}; + if ('voiceMap' in extension_settings.tts[ttsProviderName]) { + // Handle previous representation + if (typeof extension_settings.tts[ttsProviderName].voiceMap === 'string') { + voiceMapFromSettings = parseVoiceMap(extension_settings.tts[ttsProviderName].voiceMap); + // Handle new representation + } else if (typeof extension_settings.tts[ttsProviderName].voiceMap === 'object') { + voiceMapFromSettings = extension_settings.tts[ttsProviderName].voiceMap; + } + } + + // Get voiceIds from provider + let voiceIdsFromProvider; + try { + voiceIdsFromProvider = await ttsProvider.fetchTtsVoiceObjects(); + } + catch { + toastr.error('TTS Provider failed to return voice ids.'); + } + + // Build UI using VoiceMapEntry objects + for (const character of characters) { + if (character === 'SillyTavern System') { + continue; + } + // Check provider settings for voiceIds + let voiceId; + if (character in voiceMapFromSettings) { + voiceId = voiceMapFromSettings[character]; + } else if (character === DEFAULT_VOICE_MARKER) { + voiceId = DISABLED_VOICE_MARKER; + } else { + voiceId = DEFAULT_VOICE_MARKER; + } + const voiceMapEntry = new VoiceMapEntry(character, voiceId); + voiceMapEntry.addUI(voiceIdsFromProvider); + voiceMapEntries.push(voiceMapEntry); + } + updateVoiceMap(); +} + +$(document).ready(function () { + function addExtensionControls() { + const settingsHtml = ` +
+
+
+ TTS +
+
+
+
+
+ Select TTS Provider
+
+ + +
+
+ + + + + + + + +
+
+
+
+
+
+
+ +
+
+
+
+ + `; + $('#extensions_settings').append(settingsHtml); + $('#tts_refresh').on('click', onRefreshClick); + $('#tts_enabled').on('click', onEnableClick); + $('#tts_narrate_dialogues').on('click', onNarrateDialoguesClick); + $('#tts_narrate_quoted').on('click', onNarrateQuotedClick); + $('#tts_narrate_translated_only').on('click', onNarrateTranslatedOnlyClick); + $('#tts_skip_codeblocks').on('click', onSkipCodeblocksClick); + $('#tts_pass_asterisks').on('click', onPassAsterisksClick); + $('#tts_auto_generation').on('click', onAutoGenerationClick); + $('#tts_narrate_user').on('click', onNarrateUserClick); + $('#tts_voices').on('click', onTtsVoicesClick); + for (const provider in ttsProviders) { + $('#tts_provider').append($('