Merge branch 'dev' into patch-1

README.md

@@ -13,6 +13,7 @@ AllTalk is an updated version of the Coqui_tts extension for Text Generation web
 - **Documentation:** Fully documented with a built in webpage. [Screenshot](https://github.com/erew123/alltalk_tts#screenshots)
 - **Console output:** Clear command line output for any warnings or issues.
 - **API Suite and 3rd Party support via JSON calls:** Can be used with 3rd party applications via JSON calls.
+- **SillyTavern integration:** Full integration with SillyTavern.
 
 ## Index
 
@@ -31,10 +32,49 @@ AllTalk is an updated version of the Coqui_tts extension for Text Generation web
 Please check the below link to find a list of all recent updates and changes.
 ####     🔄 **Updates list & bug fixes list** can be found [here](https://github.com/erew123/alltalk_tts/issues/25)
 
+## 🟩 Quick Setup (Text-generation-webui & Standalone Installation)
+For Windows 10/11 and Linux machines there is a quick setup script. 
+<details>
+	<summary>(Click to expand) QUICK SETUP - Text-Generation-webui</summary><br>
+	
+1) To download the AllTalk setup you can either:
+   - A) On this page select **CODE** > **Download ZIP** then extract it to the text-generation-webui extensions folder<br>e.g. `\text-generation-webui\extensions\alltalk_tts\`<br><br>
+   - B) Go to a terminal/console, move into the `\text-generation-webui\extensions\` folder<br>and `git clone https://github.com/erew123/alltalk_tts`<br><br>
+3) In a terminal/command prompt, in the text-generation-webui folder you will start its Python environment with either `cmd_windows.bat` or `./cmd_linux.sh`
+4) Move into the AllTalk folder e.g. `cd extensions` then `cd alltalk_tts`
+5) Start the AllTalk setup script `atsetup.bat` or `./atsetup.sh`
+6) Follow the on-screen prompts and install the correct requirements files that you need. It's recommended to test AllTalk works before installing DeepSpeed.
+
+   Any time you need to make changes to AllTalk, or use Finetuning etc, always start the Text-generation-webui Python environment first.
+
+   Finetuning & DeepSpeed have other installation requirements (depending on your OS) so please read any instructions in the setup utility and refer back here to this page for detailed instructions (as needed).<br><br>
+</details>
+<details>
+	<summary>(Click to expand) QUICK SETUP - Standalone Installation</summary><br>
+	
+1) To download the AllTalk setup you can either:
+   - A) On this page select **CODE** > **Download ZIP** then extract it to the folder of your choice<br>e.g. `C:\myfiles\alltalk_tts\`<br><br>
+   - B) Go to a terminal/console, move into the folder of your choice e.g `C:\myfiles\` folder<br>and `git clone https://github.com/erew123/alltalk_tts`<br><br>
+4) In a terminal/command prompt, move into the AllTalk folder e.g. `cd alltalk_tts`
+5) Start the AllTalk setup script `atsetup.bat` or `./atsetup.sh`
+6) Follow the on-screen prompts and install the correct requirements files that you need. It's recommended to test AllTalk works before installing DeepSpeed.
+
+   DeepSpeed on Windows machines will be installed as standard. Linux machines have other requirements which are detailed within the setup utility and on this page.
+
+   Finetuning has other installation requirements so please read any instructions in the setup utility and refer back here to this page for detailed instructions.<br><br>
+</details>
+
+
+
 ## 🟩 Installation on Text generation web UI
+On Mac's or if you wish to perform a manual installation, please expand the below.
+<details>
+	<summary>(Click to expand) MANUAL INSTALLATION - Text-Generation-webui</summary><br>
+
 This has been tested on the current Dec 2023 release of Text generation webUI. If you have not updated it for a while, you may wish to update Text generation webUI, [instructions here](https://github.com/oobabooga/text-generation-webui?tab=readme-ov-file#how-to-install)
 
-If you want to watch a video of how to do the below [link here](https://youtu.be/9BPKuwaav5w) 
+- If you want to watch a video of how to do the below [link here](https://youtu.be/9BPKuwaav5w)<br>
+- If you wish to install AllTalk as a **standalone application** without Text-generation-webui, please go to [this link here](https://github.com/erew123/alltalk_tts?#-running-alltalk-as-a-standalone-app)
 
 1) In a command prompt/terminal window you need to move into your Text generation webUI folder:<br><br>
 `cd text-generation-webui`
@@ -62,6 +102,7 @@ If you want to watch a video of how to do the below [link here](https://youtu.be
 10) Please read the note below about start-up times and also the note about ensuring your character cards are set up [correctly](https://github.com/erew123/alltalk_tts#the-one-thing-i-cant-easily-work-around)
 
 11) Some extra voices downloadable [here](https://drive.google.com/file/d/1bYdZdr3L69kmzUN3vSiqZmLRD7-A3M47/view?usp=drive_link)
+</details>
 
 #### 🟩 Other installation notes
 On first startup, AllTalk will download the Coqui XTTSv2 2.0.2 model to its **models** folder (1.8GB space required). Check the command prompt/terminal window if you want to know what its doing. After it says "Model Loaded" the Text generation webUI is usually available on its IP address a few seconds later, for you to connect to in your browser.
@@ -621,6 +662,22 @@ Retrieve a list of available voices for generating speech.
 
    JSON return: `{"voices": ["voice1.wav", "voice2.wav", "voice3.wav"]}`
 
+#### 🟠 Current Settings Endpoint<br>
+Retrieve a list of available voices for generating speech.
+
+- URL: `http://127.0.0.1:7851/api/currentsettings`<br> - Method: `GET`<br>
+
+   `curl -X GET "http://127.0.0.1:7851/api/currentsettings"`
+
+   JSON return: `{"models_available":[{"name":"Coqui","model_name":"API TTS"},{"name":"Coqui","model_name":"API Local"},{"name":"Coqui","model_name":"XTTSv2 Local"}],"current_model_loaded":"XTTSv2 Local","deepspeed_available":true,"deepspeed_status":true,"low_vram_status":true,"finetuned_model":false}
+
+  name & model_name = listing the currently available models.
+  current_model_loaded = what model is currently loaded into VRAM.
+  deepspeed_available = was DeepSpeed detected on startup and available to be activated.
+  deepspeed_status = If DeepSpeed was detected, is it currently activated.
+  low_vram_status = Is Low VRAM currently enabled.
+  finetuned_model = Was a finetuned model detected. (XTTSv2 FT).
+
 #### 🟠 Preview Voice Endpoint
 Generate a preview of a specified voice with hardcoded settings.
 
@@ -669,8 +726,8 @@ Generate a preview of a specified voice with hardcoded settings.
 
 - URL: `http://127.0.0.1:7851/api/tts-generate`<br> - Method: `POST`<br> - Content-Type: `application/x-www-form-urlencoded`<br>
 
-### 🟠 Example command lines
+### 🟠 Example command lines (Standard Generation)
-Standard TTS speech Example (standard text) generating a time-stamped file<br>
+Standard TTS generation supports Narration and will generate a wav file/blob. Standard TTS speech Example (standard text) generating a time-stamped file<br>
 
 `curl -X POST "http://127.0.0.1:7851/api/tts-generate" -d "text_input=All of this is text spoken by the character. This is text not inside quotes, though that doesnt matter in the slightest" -d "text_filtering=standard" -d "character_voice_gen=female_01.wav" -d "narrator_enabled=false" -d "narrator_voice_gen=male_01.wav" -d "text_not_inside=character" -d "language=en" -d "output_file_name=myoutputfile" -d "output_file_timestamp=true" -d "autoplay=true" -d "autoplay_volume=0.8"`<br>
 
@@ -773,7 +830,29 @@ Example JSON TTS Generation Response:
 
 `{"status":"generate-success","output_file_path":"C:\\text-generation-webui\\extensions\\alltalk_tts\\outputs\\myoutputfile_1704141936.wav","output_file_url":"http://127.0.0.1:7851/audio/myoutputfile_1704141936.wav","output_cache_url":"http://127.0.0.1:7851/audiocache/myoutputfile_1704141936.wav"}`
 
+### 🟠 Example command lines (Streaming Generation)
+Streaming TTS generation does NOT support Narration and will generate an audio stream. Streaming TTS speech JavaScript Example:<br>
+```
+// Example parameters
+const text = "Here is some text";
+const voice = "female_01.wav";
+const language = "en";
+const outputFile = "stream_output.wav";
+// Encode the text for URL
+const encodedText = encodeURIComponent(text);
+// Create the streaming URL
+const streamingUrl = `http://localhost:7851/api/tts-generate-streaming?text=${encodedText}&voice=${voice}&language=${language}&output_file=${outputFile}`;
+// Create and play the audio element
+const audioElement = new Audio(streamingUrl);
+audioElement.play(); // Play the audio stream directly
+```
+- **Text (text):** This is the actual text you want to convert to speech. It should be a string and must be URL-encoded to ensure that special characters (like spaces and punctuation) are correctly transmitted in the URL. Example: `Hello World` becomes `Hello%20World` when URL-encoded.<br>
+- **Voice (voice):** This parameter specifies the voice type to be used for the TTS. The value should match one of the available voice options in AllTalks voices folder. This is a string representing the file, like `female_01.wav`.<br>
+- **Language (language):** This setting determines the language in which the text should be spoken. A two-letter language code (like `en` for English, `fr` for French, etc.).<br>
+- **Output File (output_file):** This parameter names the output file where the audio will be streamed. It should be a string representing the file name, such as `stream_output.wav`. AllTalk will not save this as a file in its outputs folder.<br>
+<br>
+
 ### 🔴 Future to-do list
-- SillyTavern Support
 - Possibly add some additional TTS engines (TBD).
+- Tagging support within text.
 - Have a break!

atsetup.bat

@@ -0,0 +1,495 @@
+@echo off
+cd /D "%~dp0"
+
+set PATH=%PATH%;%SystemRoot%\system32
+
+:: Generate the ESC character
+for /F %%a in ('echo prompt $E ^| cmd') do set "ESC=%%a"
+
+:: Standard Colors
+set "BLACK=%ESC%[30m"
+set "RED=%ESC%[31m"
+set "GREEN=%ESC%[32m"
+set "YELLOW=%ESC%[33m"
+set "BLUE=%ESC%[34m"
+set "MAGENTA=%ESC%[35m"
+set "CYAN=%ESC%[36m"
+set "WHITE=%ESC%[37m"
+set "L_BLACK=%ESC%[90m"
+set "L_RED=%ESC%[91m"
+set "L_GREEN=%ESC%[92m"
+set "L_YELLOW=%ESC%[93m"
+set "L_BLUE=%ESC%[94m"
+set "L_MAGENTA=%ESC%[95m"
+set "L_CYAN=%ESC%[96m"
+set "L_WHITE=%ESC%[97m"
+set "RESET=%ESC%[0m"
+
+@rem Check if curl is available
+curl --version >nul 2>&1
+if "%ERRORLEVEL%" NEQ "0" (
+    echo curl is not available on this system. Please install curl then re-run the script https://curl.se/ 
+	echo or perform a manual installation of a Conda Python environment.
+    goto end
+)
+
+:MainMenu
+cls
+echo.
+echo    %L_BLUE%ALLTALK WINDOWS SETUP UTILITY%RESET%
+echo.
+echo    INSTALLATION TYPE
+echo    1) I am using AllTalk as part of %L_GREEN%Text-generation-webui%RESET%
+echo    2) I am using AllTalk as a %L_GREEN%Standalone Application%RESET%
+echo.
+echo    9) %L_RED%Exit/Quit%RESET%
+echo.
+set /p UserOption="    Enter your choice: "
+
+if "%UserOption%"=="1" goto WebUIMenu
+if "%UserOption%"=="2" goto StandaloneMenu
+if "%UserOption%"=="9" goto End
+goto MainMenu
+
+:WebUIMenu
+cls
+echo.
+echo    %L_BLUE%TEXT-GENERATION-WEBUI SETUP%RESET%
+echo.
+echo    Please ensure you have started your Text-generation-webui Python 
+echo    environment. If you have NOT done this, please run %L_GREEN%cmd_windows.bat%RESET% 
+echo    in the %L_GREEN%text-generation-webui%RESET% folder and then re-run this script.
+echo.
+echo    BASE REQUIREMENTS
+echo    1) Install the requirements for a %L_GREEN%Nvidia machine%RESET%.
+echo    2) Install the requirements for a %L_GREEN%AMD or MAC machine%RESET%.
+echo.
+echo    OPTIONAL
+echo    3) Install the Finetuning requirements.
+echo    4) Install DeepSpeed v11.2 for CUDA %L_GREEN%11.8%RESET% and Python-3.11.x.
+echo    5) Install DeepSpeed v11.2 for CUDA %L_GREEN%12.1%RESET% and Python-3.11.x.
+echo    6) Uninstall DeepSpeed.
+echo.
+echo.   OTHER
+echo    7) Generate a diagnostics file.
+echo.
+echo    9) %L_RED%Exit/Quit%RESET%
+echo.
+set /p WebUIOption="Enter your choice: "
+if "%WebUIOption%"=="1" goto InstallNvidiaTextGen
+if "%WebUIOption%"=="2" goto InstallOtherTextGen
+if "%WebUIOption%"=="3" goto InstallFinetuneTextGen
+if "%WebUIOption%"=="4" goto InstallDeepSpeed118TextGen
+if "%WebUIOption%"=="5" goto InstallDeepSpeed121TextGen
+if "%WebUIOption%"=="6" goto UnInstallDeepSpeed
+if "%WebUIOption%"=="7" goto GenerateDiagsTextGen
+if "%WebUIOption%"=="9" goto End
+goto WebUIMenu
+
+:StandaloneMenu
+cls
+echo.
+echo    %L_BLUE%ALLTALK STANDALONE APPLICATION SETUP%RESET%
+echo.
+echo    BASE REQUIREMENTS
+echo    1) Install AllTalk as a Standalone Application
+echo.
+echo    OPTIONAL
+echo    2) Delete AllTalk's custom Python environment
+echo    3) Install the Finetuning requirements
+echo.
+echo.   OTHER
+echo    4) Generate a diagnostics file
+echo.
+echo    9) %L_RED%Exit/Quit%RESET%
+echo.
+set /p StandaloneOption="Enter your choice: "
+if "%StandaloneOption%"=="1" goto InstallCustomStandalone
+if "%StandaloneOption%"=="2" goto DeleteCustomStandalone
+if "%StandaloneOption%"=="3" goto InstallFinetuneStandalone
+if "%StandaloneOption%"=="4" goto GenerateDiagsStandalone
+if "%StandaloneOption%"=="9" goto End
+goto StandaloneMenu
+
+:InstallNvidiaTextGen
+pip install -r requirements_nvidia.txt
+if %ERRORLEVEL% neq 0 (
+    echo.
+    echo    There was an error installing the Nvidia requirements.
+    echo    Press any key to return to the menu.
+    echo.
+    pause
+    goto WebUIMenu
+)
+Echo.
+echo.
+Echo    Nvidia machine requirements installed successfully.
+Echo. 
+pause
+goto WebUIMenu
+
+:InstallOtherTextGen
+pip install -r requirements_other.txt
+if %ERRORLEVEL% neq 0 (
+    echo. 
+    echo    There was an error installing the Other requirements.
+    echo    Press any key to return to the menu.
+    echo.
+    pause
+    goto WebUIMenu
+)
+Echo.
+echo.
+Echo    Other machine requirements installed successfully.
+Echo. 
+pause
+goto WebUIMenu
+
+:InstallFinetuneTextGen
+pip install -r requirements_finetune.txt
+if %ERRORLEVEL% neq 0 (
+    echo.
+    echo    There was an error installing the Finetune requirements.
+    echo    Press any key to return to the menu.
+    echo.
+    pause
+    goto WebUIMenu
+)
+Echo.
+Echo    Finetune requirements installed successfully. Please ensure you have installed CUDA 11.8.
+Echo    Finetune needs access to %L_GREEN%cublas64_11.dll%RESET% and will error if this is not setup correctly.
+Echo    Please see here for details %L_GREEN%https://github.com/erew123/alltalk_tts#-finetuning-a-model%RESET%
+Echo.
+pause
+goto WebUIMenu
+
+:InstallDeepSpeed118TextGen
+echo Downloading DeepSpeed...
+curl -LO https://github.com/erew123/alltalk_tts/releases/download/deepspeed/deepspeed-0.11.2+cuda118-cp311-cp311-win_amd64.whl
+if %ERRORLEVEL% neq 0 (
+    echo.
+    echo    Failed to download DeepSpeed wheel file.
+    echo    Please check your internet connection or try again later.
+    echo    Press any key to return to the menu.
+    echo.
+    pause
+    goto WebUIMenu
+)
+echo DeepSpeed wheel file downloaded successfully.
+echo Installing DeepSpeed...
+pip install deepspeed-0.11.2+cuda118-cp311-cp311-win_amd64.whl
+if %ERRORLEVEL% neq 0 (
+    echo. 
+    echo    Failed to install DeepSpeed.
+    echo    Please check if the wheel file is compatible with your system.
+    echo    Press any key to return to the menu.
+    echo.
+    pause
+    goto WebUIMenu
+)
+Echo.
+echo.
+Echo    DeepSpeed installed successfully.
+Echo. 
+del deepspeed-0.11.2+cuda118-cp311-cp311-win_amd64.whl
+pause
+goto WebUIMenu
+
+:InstallDeepSpeed121TextGen
+echo Downloading DeepSpeed...
+curl -LO https://github.com/erew123/alltalk_tts/releases/download/deepspeed/deepspeed-0.11.2+cuda121-cp311-cp311-win_amd64.whl
+if %ERRORLEVEL% neq 0 (
+    echo.
+    echo    Failed to download DeepSpeed wheel file.
+    echo    Please check your internet connection or try again later.
+    echo    Press any key to return to the menu.
+    echo.
+    pause
+    goto WebUIMenu
+)
+echo    DeepSpeed wheel file downloaded successfully.
+echo    Installing DeepSpeed...
+pip install deepspeed-0.11.2+cuda121-cp311-cp311-win_amd64.whl
+if %ERRORLEVEL% neq 0 (
+    echo. 
+    echo    Failed to install DeepSpeed.
+    echo    Please check if the wheel file is compatible with your system.
+    echo    Press any key to return to the menu.
+    echo.
+    pause
+    goto WebUIMenu
+)
+Echo.
+echo.
+Echo    DeepSpeed installed successfully.
+Echo. 
+del deepspeed-0.11.2+cuda121-cp311-cp311-win_amd64.whl
+pause
+goto WebUIMenu
+
+:UnInstallDeepSpeed
+pip uninstall deepspeed
+if %ERRORLEVEL% neq 0 (
+    echo. 
+    echo    There was an error uninstalling DeepSpeed.
+    echo    Press any key to return to the menu.
+    echo.
+    pause
+    goto WebUIMenu
+)
+echo.
+Echo    DeepSpeed uninstalled successfully.
+Echo. 
+pause
+goto WebUIMenu
+
+:GenerateDiagsTextGen
+Python diagnostics.py
+if %ERRORLEVEL% neq 0 (
+    echo.
+    echo    There was an error running diagnostics. Have you correctly started your 
+    echo    Text-generation-webui Python environment with %L_GREEN%cmd_windows.bat%RESET%?
+    echo.
+    pause
+    goto WebUIMenu
+)
+Echo.
+echo.
+Echo    Diagnostics.log generated. Please scroll up to look over the log.
+Echo. 
+pause
+goto WebUIMenu
+
+:InstallCustomStandalone
+set PATH=%PATH%;%SystemRoot%\system32
+
+@rem Check if curl is available
+curl --version >nul 2>&1
+if "%ERRORLEVEL%" NEQ "0" (
+    echo curl is not available on this system. Please install curl then re-run the script https://curl.se/ 
+	echo or perform a manual installation of a Conda Python environment.
+    goto end
+)
+echo "%CD%"| findstr /C:" " >nul && echo This script relies on Miniconda which can not be silently installed under a path with spaces. Please correct your folder names and re-try installation. && goto exit
+
+@rem Check for special characters in installation path
+set "SPCHARMESSAGE=WARNING: Special characters were detected in the installation path! This can cause the installation to fail!"
+echo "%CD%"| findstr /R /C:"[!#\$%&()\*+,;<=>?@\[\]\^`{|}~]" >nul && (
+    call :PrintBigMessage %SPCHARMESSAGE%
+)
+set SPCHARMESSAGE=
+
+@rem fix failed install when installing to a separate drive
+set TMP=%cd%\alltalk_environment
+set TEMP=%cd%\alltalk_environment
+
+@rem deactivate existing conda envs as needed to avoid conflicts
+(call conda deactivate && call conda deactivate && call conda deactivate) 2>nul
+
+@rem config
+set INSTALL_DIR=%cd%\alltalk_environment
+set CONDA_ROOT_PREFIX=%cd%\alltalk_environment\conda
+set INSTALL_ENV_DIR=%cd%\alltalk_environment\env
+set MINICONDA_DOWNLOAD_URL=https://repo.anaconda.com/miniconda/Miniconda3-py310_23.3.1-0-Windows-x86_64.exe
+set conda_exists=F
+
+@rem figure out whether git and conda need to be installed
+call "%CONDA_ROOT_PREFIX%\_conda.exe" --version >nul 2>&1
+if "%ERRORLEVEL%" EQU "0" (
+    set conda_exists=T
+    call "%CONDA_ROOT_PREFIX%\condabin\conda.bat" activate "%INSTALL_ENV_DIR%"
+    goto RunScript
+)
+
+@rem download and install conda if not installed
+echo Downloading Miniconda from %MINICONDA_DOWNLOAD_URL% to %INSTALL_DIR%\miniconda_installer.exe
+mkdir "%INSTALL_DIR%"
+call curl -Lk "%MINICONDA_DOWNLOAD_URL%" > "%INSTALL_DIR%\miniconda_installer.exe" || ( echo. && echo Miniconda failed to download. && goto end )
+echo Installing Miniconda to %CONDA_ROOT_PREFIX%
+start /wait "" "%INSTALL_DIR%\miniconda_installer.exe" /InstallationType=JustMe /NoShortcuts=1 /AddToPath=0 /RegisterPython=0 /NoRegistry=1 /S /D=%CONDA_ROOT_PREFIX%
+echo Miniconda version:
+call "%CONDA_ROOT_PREFIX%\_conda.exe" --version || ( echo. && echo Miniconda not found. && goto end )
+
+@rem create the installer env
+echo Packages to install: %PACKAGES_TO_INSTALL%
+call "%CONDA_ROOT_PREFIX%\_conda.exe" create --no-shortcuts -y -k --prefix "%INSTALL_ENV_DIR%" python=3.11 || ( echo. && echo Conda environment creation failed. && goto end )
+
+@rem check if conda environment was actually created
+if not exist "%INSTALL_ENV_DIR%\python.exe" ( echo. && echo Conda environment is empty. && goto end )
+
+@rem activate installer env
+call "%CONDA_ROOT_PREFIX%\condabin\conda.bat" activate "%INSTALL_ENV_DIR%" || ( echo. && echo Miniconda hook not found. && goto end )
+
+@rem Ask user for the type of requirements to install
+echo.
+echo    %L_BLUE%Choose the type of requirements to install%RESET%
+echo.
+echo    1. Nvidia graphics card machines
+echo    2. Other machines (mac, amd, etc)
+echo.
+set /p UserChoice="Enter your choice (1 or 2): "
+
+@rem Install requirements based on user choice
+if "%UserChoice%" == "1" (
+    echo.
+    echo Installing Nvidia requirements...
+    echo Downloading Pytorch with CUDA..
+    pip install torch==2.1.0 torchvision==0.16.0 torchaudio==2.1.0 --upgrade --force-reinstall --index-url https://download.pytorch.org/whl/cu121
+    pip install numpy==1.24.4
+    pip install typing-extensions>=4.8.0
+    pip install networkx<3.0.0,>=2.5.0
+    pip install fsspec>=2023.6.0
+    pip install soundfile==0.12.1
+    pip install uvicorn==0.24.0.post1
+    pip install TTS==0.21.3
+    pip install fastapi==0.104.1
+    pip install Jinja2==3.1.2
+    pip install requests==2.31.0
+    pip install tqdm==4.66.1
+    pip install importlib-metadata==4.8.1
+    pip install packaging==23.2
+    pip install pydantic==1.10.13
+    pip install sounddevice==0.4.6
+    pip install python-multipart==0.0.6
+    echo Downloading DeepSpeed...
+    curl -LO https://github.com/erew123/alltalk_tts/releases/download/DeepSpeed-12.7/deepspeed-0.12.7+d058d4b-cp311-cp311-win_amd64.whl
+    echo Installing DeepSpeed...
+    pip install deepspeed-0.12.7+d058d4b-cp311-cp311-win_amd64.whl
+    del deepspeed-0.12.7+d058d4b-cp311-cp311-win_amd64.whl
+) else (
+    echo Installing requirements for other machines...
+    pip install -r requirements_other.txt
+)
+
+@rem Create start_alltalk.bat to run AllTalk
+echo @echo off > start_alltalk.bat
+echo cd /D "%~dp0" >> start_alltalk.bat
+echo set CONDA_ROOT_PREFIX=%cd%\alltalk_environment\conda >> start_alltalk.bat
+echo set INSTALL_ENV_DIR=%cd%\alltalk_environment\env >> start_alltalk.bat
+echo call "%CONDA_ROOT_PREFIX%\condabin\conda.bat" activate "%INSTALL_ENV_DIR%" >> start_alltalk.bat
+echo call python script.py >> start_alltalk.bat
+echo exit >> start_alltalk.bat
+Echo.
+Echo    start_alltalk.bat has been created.
+Echo    You can now start AllTalk with %L_YELLOW%start_alltalk.bat%RESET%
+Echo.
+pause
+goto StandaloneMenu
+
+:DeleteCustomStandalone
+@rem Check if the alltalk_environment directory exists
+if not exist "%cd%\alltalk_environment\" (
+    echo.
+    echo    %L_GREEN%alltalk_environment%RESET% directory does not exist. No need to delete.
+    echo.
+    pause
+    goto StandaloneMenu
+)
+@rem Check if a Conda environment is active
+if not defined CONDA_PREFIX goto NoCondaEnvDeleteCustomStandalone
+@rem Deactivate Conda environment if it's active
+Echo    Exiting the Conda Environment. Please run %L_GREEN%atsetup.bat%RESET% again and delete the environment.
+conda deactivate
+:NoCondaEnvDeleteCustomStandalone
+echo Deleting "alltalk_environment". Please wait.
+rd /s /q "alltalk_environment"
+del start_alltalk.bat
+if %ERRORLEVEL% neq 0 (
+    echo.
+    echo    Failed to delete alltalk_environment folder.
+    echo    Please make sure it is not in use and try again.
+    echo.
+    pause
+    goto StandaloneMenu
+)
+echo.
+Echo.
+echo    Environment %L_GREEN%alltalk_environment%RESET% deleted. Please set up the environment again.
+Echo.
+pause
+goto StandaloneMenu
+
+:InstallFinetuneStandalone
+cd /D "%~dp0"
+set CONDA_ROOT_PREFIX=%cd%\alltalk_environment\conda
+set INSTALL_ENV_DIR=%cd%\alltalk_environment\env
+@rem Check if the Conda environment exists
+if not exist "%INSTALL_ENV_DIR%\python.exe" (
+    echo.
+    echo    The Conda environment at "%INSTALL_ENV_DIR%" does not exist.
+    echo    Please install the environment before proceeding.
+    echo.
+    pause
+    goto StandaloneMenu
+)
+@rem Attempt to activate the Conda environment
+call "%CONDA_ROOT_PREFIX%\condabin\conda.bat" activate "%INSTALL_ENV_DIR%"
+if errorlevel 1 (
+    echo. 
+    echo    Failed to activate the Conda environment.
+    echo    Please check your installation and try again.
+    echo.
+    pause
+    goto StandaloneMenu
+)
+@rem Proceed with installing requirements
+pip install -r requirements_finetune.txt
+if %ERRORLEVEL% neq 0 (
+    echo. 
+    echo    There was an error installing the Finetune requirements.
+    echo    Press any key to return to the menu.
+    echo.
+    pause
+    goto StandaloneMenu
+)
+Echo.
+Echo    Finetune requirements installed successfully. Please ensure you have installed CUDA 11.8.
+Echo    Finetune needs access to %L_GREEN%cublas64_11.dll%RESET% and will error if this is not setup correctly.
+Echo    Please see here for details %L_GREEN%https://github.com/erew123/alltalk_tts#-finetuning-a-model%RESET%
+Echo.
+pause
+goto StandaloneMenu
+
+:GenerateDiagsStandalone
+cd /D "%~dp0"
+set CONDA_ROOT_PREFIX=%cd%\alltalk_environment\conda
+set INSTALL_ENV_DIR=%cd%\alltalk_environment\env
+@rem Check if the Conda environment exists
+if not exist "%INSTALL_ENV_DIR%\python.exe" (
+    echo.
+    echo    The Conda environment at "%INSTALL_ENV_DIR%" does not exist.
+    echo    Please install the environment before proceeding.
+    echo. 
+    pause
+    goto StandaloneMenu
+)
+@rem Attempt to activate the Conda environment
+call "%CONDA_ROOT_PREFIX%\condabin\conda.bat" activate "%INSTALL_ENV_DIR%"
+if errorlevel 1 (
+    echo. 
+    echo    Failed to activate the Conda environment.
+    echo    Please check your installation and try again.
+    echo.
+    pause
+    goto StandaloneMenu
+)
+@rem Run diagnostics
+Python diagnostics.py
+if %ERRORLEVEL% neq 0 (
+    echo.
+    echo    There was an error running diagnostics.
+    echo    Press any key to return to the menu.
+    echo.
+    pause
+    goto StandaloneMenu
+)
+Echo.
+Echo.
+Echo    Diagnostics.log generated. Please scroll up to look over the log.
+Echo.
+pause
+goto StandaloneMenu
+
+:End
+echo Exiting AllTalk Setup Utility...
+exit /b

atsetup.sh

@@ -0,0 +1,421 @@
+#!/bin/bash
+
+# ANSI color codes
+RED='\033[1;31m'
+GREEN='\033[1;32m'
+YELLOW='\033[1;33m'
+BLUE='\033[1;34m'
+CYAN='\033[1;36m'
+MAGENTA='\033[1;35m'
+WHITE='\033[1;37m'
+L_RED='\033[1;91m'
+L_GREEN='\033[1;92m'
+L_YELLOW='\033[1;93m'
+L_BLUE='\033[1;94m'
+L_CYAN='\033[1;96m'
+L_MAGENTA='\033[1;95m'
+NC='\033[0m' # No Color
+
+# Navigate to the script's directory
+cd "$(dirname "$0")"
+
+# Function to check if curl is installed
+check_curl() {
+    if ! command -v curl &> /dev/null; then
+        echo "curl is not available on this system. Please install curl then re-run the script https://curl.se/"
+        echo "or perform a manual installation of a Conda Python environment."
+        exit 1
+    fi
+}
+
+# Main Menu
+main_menu() {
+    while true; do
+        clear
+        echo
+        echo -e "    ${L_BLUE}ALLTALK LINUX SETUP UTILITY${NC}"
+        echo
+        echo "    INSTALLATION TYPE"
+        echo -e "    1) I am using AllTalk as part of ${L_GREEN}Text-generation-webui${NC}"
+        echo -e "    2) I am using AllTalk as a ${L_GREEN}Standalone Application${NC}"
+        echo
+        echo -e "    9)${L_RED} Exit/Quit${NC}"
+        echo
+        read -p "    Enter your choice: " user_option
+
+        case $user_option in
+            1) webui_menu ;;
+            2) standalone_menu ;;
+            9) exit 0 ;;
+            *) echo "Invalid option"; sleep 2 ;;
+        esac
+    done
+}
+
+# Text-generation-webui Menu
+webui_menu() {
+    while true; do
+        clear
+        echo
+        echo -e "    ${L_BLUE}TEXT-GENERATION-WEBUI SETUP${NC}"
+        echo
+        echo -e "    Please ensure you have started your Text-generation-webui Python"
+        echo -e "    environment. If you have NOT done this, please run ${L_GREEN}/cmd_linux.sh${NC}"
+        echo -e "    in the ${L_GREEN}text-generation-webui${NC} folder and then re-run this script."
+        echo
+        echo "    BASE REQUIREMENTS"
+        echo -e "    1) Install the AllTalk requirements for an ${L_GREEN}Nvidia GPU machine.${NC}"
+        echo -e "    2) Install the AllTalk requirements for an ${L_GREEN}AMD or MAC machine.${NC}"
+        echo
+        echo "    OPTIONAL"
+        echo "    3) Install the Finetuning requirements."
+        echo "    4) Install DeepSpeed."
+        echo "    5) Uninstall DeepSpeed."
+        echo
+        echo "    OTHER"
+        echo "    6) Generate a diagnostics file."
+        echo
+        echo -e "    9)${L_RED} Exit/Quit${NC}"
+        echo
+        read -p "    Enter your choice: " webui_option
+
+        case $webui_option in
+            1) install_nvidia_textgen ;;
+            2) install_other_textgen ;;
+            3) install_finetune_textgen ;;
+            4) install_deepspeed ;;
+            5) uninstall_deepspeed ;;
+            6) generate_diagnostics_textgen ;;
+            9) exit 0 ;;
+            *) echo "Invalid option"; sleep 2 ;;
+        esac
+    done
+}
+
+install_nvidia_textgen() {
+    local requirements_file="requirements_nvidia.txt"
+    echo "    Installing Finetune requirements from $requirements_file..."
+    if ! pip install -r "$requirements_file"; then
+        echo "    There was an error installing the Nvidia requirements."
+        echo "    Please check the output for details."
+        read -p "    Press any key to return to the menu. " -n 1
+        echo
+        return
+    fi
+    echo "    Nvidia requirements installed successfully."
+    read -p "    Press any key to continue. " -n 1
+    echo
+}
+
+install_other_textgen() {
+    local requirements_file="requirements_other.txt"
+    echo "    Installing Other machine requirements from $requirements_file..."
+    if ! pip install -r "$requirements_file"; then
+        echo "    There was an error installing the Other requirements."
+        echo "    Please check the output for details."
+        read -p "    Press any key to return to the menu. " -n 1
+        echo
+        return
+    fi
+    echo "    Other machine requirements installed successfully."
+    read -p "    Press any key to continue. " -n 1
+    echo
+}
+
+install_finetune_textgen() {
+    local requirements_file="requirements_finetune.txt"
+    echo "    Installing Finetune requirements from $requirements_file..."
+    if ! pip install -r "$requirements_file"; then
+        echo "    There was an error installing the Finetune requirements."
+        echo "    Please check the output for details."
+        read -p "    Press any key to return to the menu. " -n 1
+        echo
+        return
+    fi
+    echo
+    echo "    Finetune base requirements installed successfully."
+    echo
+    echo -e "    Finetuning requires access to ${GREEN}Cublas${NC} from the ${GREEN}Nvidia CUDA Toolkit${NC}, so will need installing."
+    echo
+    echo -e "    Instructions can be found here:"
+    echo -e "    ${GREEN}https://github.com/erew123/alltalk_tts?#-finetuning-a-model${NC}"
+    echo
+    read -p "    Press any key to continue. " -n 1
+    echo
+}
+
+# Function to install DeepSpeed
+install_deepspeed() {
+    clear
+    echo
+    echo -e "    ${L_BLUE}DEEPSPEED INSTALLATION REQUIREMENTS${NC}"
+    echo
+    echo -e "    - NVIDIA CUDA Toolkit must be installed from your ${L_GREEN}Linux package manager${NC}"
+    echo -e "      or ${L_GREEN}https://developer.nvidia.com/cuda-toolkit-archive${NC}"
+    echo -e "    - The environment variable ${L_GREEN}CUDA_HOME${NC} must be set in your Python env"
+    echo -e "      e.g. export CUDA_HOME=/usr/local/cuda."
+    echo -e "    - ${L_GREEN}libaio-dev${NC} must be installed."
+    echo
+    echo -e "    See the following link for full instructions:"
+    echo -e "    ${L_GREEN}https://github.com/erew123/alltalk_tts?#-deepspeed-installation-options${NC}"
+    echo
+    echo -e "    ${L_BLUE}Please ${L_RED}confirm${L_BLUE} that you have completed these steps before continuing" 
+    echo -e "    with the DeepSpeed installation, or the install ${L_RED}will fail.${NC}"
+    echo
+    read -p "    Have you completed all the above steps? (y/n): " confirm
+
+    if [ "$confirm" != "y" ]; then
+        echo -e "    ${RED}DeepSpeed installation cannot proceed without completing the prerequisites.${NC}"
+        return
+    fi
+
+    echo -e "\n    ${GREEN}Proceeding with DeepSpeed installation...${NC}"
+    pip install deepspeed
+    if [ $? -ne 0 ]; then
+        echo -e "    ${RED}There was an error installing DeepSpeed.${NC}"
+        return
+    fi
+
+    echo -e "    ${GREEN}DeepSpeed installed successfully.${NC}"
+    read -p "    Press any key to continue. " -n 1
+    echo
+}
+
+uninstall_deepspeed() {
+    echo "Uninstalling DeepSpeed..."
+    pip uninstall -y deepspeed
+    if [ $? -ne 0 ]; then
+        echo "    There was an error uninstalling DeepSpeed."
+        echo "    Press any key to return to the menu."
+        read -n 1
+        return
+    fi
+    echo "    DeepSpeed uninstalled successfully."
+    echo "    Press any key to continue."
+    read -n 1
+}
+
+generate_diagnostics_textgen() {
+    # Run diagnostics
+    if ! python diagnostics.py; then
+        echo "    There was an error running diagnostics."
+        read -p "    Press any key to return to the menu. " -n 1
+        echo
+        return
+    fi
+
+    echo "    Diagnostics completed successfully."
+    read -p "    Press any key to continue. " -n 1
+    echo
+}
+
+# Standalone Menu
+standalone_menu() {
+    while true; do
+        clear
+        echo
+        echo -e "    ${L_BLUE}ALLTALK STANDALONE APPLICATION SETUP${NC}"
+        echo
+        echo "    BASE REQUIREMENTS"
+        echo "    1) Install AllTalk as a Standalone Application"
+        echo
+        echo "    OPTIONAL"
+        echo "    2) Delete AllTalk's custom Python environment"
+        echo "    3) Install the Finetuning requirements"
+        echo "    4) Install DeepSpeed."
+        echo
+        echo "    OTHER"        
+        echo "    5) Generate a diagnostics file"
+        echo
+        echo -e "    9)${L_RED} Exit/Quit${NC}"
+        echo
+        read -p "    Enter your choice: " standalone_option
+
+        case $standalone_option in
+            1) install_custom_standalone ;;
+            2) delete_custom_standalone ;;
+            3) install_finetune_standalone ;;
+            4) install_deepspeed ;;
+            5) generate_diagnostics_standalone ;;
+            9) exit 0 ;;
+            *) echo "Invalid option"; sleep 2 ;;
+        esac
+    done
+}
+
+install_custom_standalone() {
+    cd "$(dirname "${BASH_SOURCE[0]}")"
+
+    if [[ "$(pwd)" =~ " " ]]; then
+        echo "This script relies on Miniconda which can not be silently installed under a path with spaces."
+        exit
+    fi
+
+    # Deactivate existing conda envs as needed to avoid conflicts
+    { conda deactivate && conda deactivate && conda deactivate; } 2> /dev/null
+
+    OS_ARCH=$(uname -m)
+    case "${OS_ARCH}" in
+        x86_64*)    OS_ARCH="x86_64" ;;
+        arm64* | aarch64*) OS_ARCH="aarch64" ;;
+        *)          echo "Unknown system architecture: $OS_ARCH! This script runs only on x86_64 or arm64" && exit ;;
+    esac
+
+    # Config
+    INSTALL_DIR="$(pwd)/alltalk_environment"
+    CONDA_ROOT_PREFIX="${INSTALL_DIR}/conda"
+    INSTALL_ENV_DIR="${INSTALL_DIR}/env"
+    MINICONDA_DOWNLOAD_URL="https://repo.anaconda.com/miniconda/Miniconda3-py310_23.3.1-0-Linux-${OS_ARCH}.sh"
+
+    if [ ! -x "${CONDA_ROOT_PREFIX}/bin/conda" ]; then
+        echo "Downloading Miniconda from $MINICONDA_DOWNLOAD_URL to ${INSTALL_DIR}/miniconda_installer.sh"
+        mkdir -p "${INSTALL_DIR}"
+        curl -L "${MINICONDA_DOWNLOAD_URL}" -o "${INSTALL_DIR}/miniconda_installer.sh"
+        chmod +x "${INSTALL_DIR}/miniconda_installer.sh"
+        bash "${INSTALL_DIR}/miniconda_installer.sh" -b -p "${CONDA_ROOT_PREFIX}"
+        echo "Miniconda installed."
+    fi
+
+    if [ ! -d "${INSTALL_ENV_DIR}" ]; then
+        "${CONDA_ROOT_PREFIX}/bin/conda" create -y --prefix "${INSTALL_ENV_DIR}" python=3.11
+        echo "Conda environment created at ${INSTALL_ENV_DIR}."
+    fi
+
+    # Activate the environment and install requirements
+    source "${CONDA_ROOT_PREFIX}/etc/profile.d/conda.sh"
+    conda activate "${INSTALL_ENV_DIR}"
+
+    echo
+    echo -e "    ${L_BLUE}Choose the type of requirements to install${NC}"
+    echo
+    echo "    1. Nvidia graphics card machines"
+    echo "    2. Other machines (mac, amd, etc)"
+    echo
+    read -p "    Enter your choice (1 or 2): " user_choice
+
+    if [ "$user_choice" == "1" ]; then
+    	pip install torch==2.1.0 torchvision==0.16.0 torchaudio==2.1.0 --upgrade --force-reinstall --index-url https://download.pytorch.org/whl/cu121
+    	pip install numpy==1.24.4
+	pip install typing-extensions>=4.8.0
+	pip install networkx<3.0.0,>=2.5.0
+    	pip install fsspec>=2023.6.0
+	pip install soundfile==0.12.1
+    	pip install uvicorn==0.24.0.post1
+    	pip install TTS==0.21.3
+    	pip install fastapi==0.104.1
+    	pip install Jinja2==3.1.2
+    	pip install requests==2.31.0
+    	pip install tqdm==4.66.1
+    	pip install importlib-metadata==4.8.1
+    	pip install packaging==23.2
+    	pip install pydantic==1.10.13
+    	pip install sounddevice==0.4.6
+    	pip install python-multipart==0.0.6
+    else
+        pip install -r requirements_other.txt
+    fi
+
+    # Create start_alltalk.sh to run AllTalk
+    cat << EOF > start_alltalk.sh
+#!/bin/bash
+source "${CONDA_ROOT_PREFIX}/etc/profile.d/conda.sh"
+conda activate "${INSTALL_ENV_DIR}"
+python script.py
+EOF
+    chmod +x start_alltalk.sh
+    echo
+    echo
+    echo -e "    start_alltalk.sh has been created."
+    echo -e "    You can now start AllTalk with ${L_YELLOW}./start_alltalk.sh${NC}"
+    echo
+    read -p "    Press any key to continue. " -n 1
+}
+
+
+delete_custom_standalone() {
+    local env_dir="$PWD/alltalk_environment"
+    # Check if the alltalk_environment directory exists
+    if [ ! -d "$env_dir" ]; then
+        echo "    \"$env_dir\" directory does not exist. No need to delete."
+        read -p "    Press any key to continue. " -n 1
+        echo
+        return
+    fi
+    # Check if a Conda environment is active and deactivate it
+    if [ -n "$CONDA_PREFIX" ]; then
+        echo "    Exiting the Conda environment. You may need to start ./atstart.sh again"
+        conda deactivate
+    fi
+    echo "Deleting \"$env_dir\". Please wait."
+    rm -rf "$env_dir"
+    if [ -d "$env_dir" ]; then
+        echo "    Failed to delete \"$env_dir\" folder."
+        echo "    Please make sure it is not in use and try again."
+    else
+        echo "    Environment \"$env_dir\" deleted successfully."
+    fi
+    read -p "    Press any key to continue. " -n 1
+    echo
+}
+
+install_finetune_standalone() {
+    local env_dir="$PWD/alltalk_environment"
+    local conda_root_prefix="${env_dir}/conda"
+    local install_env_dir="${env_dir}/env"
+    if [ ! -d "${install_env_dir}" ]; then
+        echo "    The Conda environment at '${install_env_dir}' does not exist."
+        echo "    Please install the environment before proceeding."
+        read -p "    Press any key to return to the menu. " -n 1
+        echo
+        return
+    fi
+    source "${conda_root_prefix}/etc/profile.d/conda.sh"
+    conda activate "${install_env_dir}"
+    local requirements_file="requirements_finetune.txt"
+    echo "Installing Finetune requirements from $requirements_file..."
+    if ! pip install -r "$requirements_file"; then
+        echo "    There was an error installing the Finetune requirements."
+        echo "    Please check the output for details."
+        read -p "    Press any key to return to the menu. " -n 1
+        echo
+        return
+    fi
+    echo
+    echo "    Finetune base requirements installed successfully."
+    echo
+    echo -e "    Finetuning requires access to ${GREEN}Cublas${NC} from the ${GREEN}Nvidia CUDA Toolkit${NC}, so will need installing."
+    echo
+    echo -e "    Instructions can be found here:"
+    echo -e "    ${GREEN}https://github.com/erew123/alltalk_tts?#-finetuning-a-model${NC}"
+    echo
+    read -p "    Press any key to continue. " -n 1
+    echo
+}
+
+generate_diagnostics_standalone() {
+    local env_dir="$PWD/alltalk_environment"
+    local conda_root_prefix="${env_dir}/conda"
+    local install_env_dir="${env_dir}/env"
+    if [ ! -d "${install_env_dir}" ]; then
+        echo "    The Conda environment at '${install_env_dir}' does not exist."
+        echo "    Please install the environment before proceeding."
+        read -p "    Press any key to return to the menu. " -n 1
+        echo
+        return
+    fi
+    source "${conda_root_prefix}/etc/profile.d/conda.sh"
+    conda activate "${install_env_dir}"
+    if ! python diagnostics.py; then
+        echo "    There was an error running diagnostics."
+        read -p "    Press any key to return to the menu. " -n 1
+        echo
+        return
+    fi
+    echo "    Diagnostics completed successfully."
+    read -p "    Press any key to continue. " -n 1
+    echo
+}
+
+
+# Start the main menu
+main_menu

diagnostics.py

@@ -215,6 +215,13 @@ def log_system_info():
 
             # Print colored output
             print(f"{package_name.ljust(max_package_length)}  Required: {color_required}{operator} {required_version.ljust(12)}\033[0m  Installed: {color_installed}{installed_version}\033[0m")
+
+        print("\nOn Nvidia Graphics cards machines, if your \033[92mInstalled\033[0m version of \033[92mTorch\033[0m and \033[92mTorchaudio\033[0m does")    
+        print("not have \033[92m+cu118\033[0m (Cuda 11.8) or \033[92m+cu121\033[0m (Cuda 12.1) listed after them, you do not have CUDA")
+        print("installed for Torch or Torchaudio in this Python environment. This will cause you problems")
+        print("with \033[94mAllTalk\033[0m and \033[94mFinetuning.\033[0m You may have to 'pip install' a new version of torch and")
+        print("torchaudio, using '\033[94m--upgrade --force-reinstall\033[0m' with the correct version of PyTorch for\033[0m")
+        print("your Python environment.\033[0m")
         print("\033[94m\nRequirements file specifier meanings:\033[0m")
         explanation = textwrap.dedent("""
         == Exact version              != Any version except          < Less than               

script.py

@@ -237,10 +237,12 @@ process = subprocess.Popen(["python", script_path])
 # Check if the subprocess has started successfully
 if process.poll() is None:
     print(f"[{params['branding']}Startup] TTS Subprocess starting")
+    print(f"[{params['branding']}Startup]")
     print(
-        f"[{params['branding']}Startup] Readme available here:",
+        f"[{params['branding']}Startup] Settings & Documentation:",
         f"http://{params['ip_address']}:{params['port_number']}",
     )
+    print(f"[{params['branding']}Startup]")
 else:
     print(
         f"[{params['branding']}Startup] \033[91mWarning\033[0m TTS Subprocess Webserver failing to start process"
@@ -862,7 +864,8 @@ def ui():
     voice.change(lambda x: params.update({"voice": x}), voice, None)
     language.change(lambda x: params.update({"language": x}), language, None)
 
-    # TTS Settings (Not yet parsed to api/implemented)
+
+    # TS Settings (Not yet parsed to api/implemented)
     # local_temperature_gr.change(lambda x: params.update({"local_temperature": x}), local_temperature_gr, None)
     # local_repetition_penalty_gr.change(lambda x: params.update({"local_repetition_penalty": x}), local_repetition_penalty_gr, None)
 
@@ -885,4 +888,4 @@ if running_in_standalone:
         try:
             time.sleep(1)  # Add a small delay to avoid high CPU usage
         except KeyboardInterrupt:
-            break  # Allow graceful exit on Ctrl+C
+            break  # Allow graceful exit on Ctrl+C

templates/STfiles/alltalk.js

@@ -0,0 +1,930 @@
+import { doExtrasFetch, getApiUrl, modules } from '../../extensions.js';
+import { saveTtsProviderSettings } from './index.js';
+
+export { AllTalkTtsProvider };
+
+const css = `
+.settings-row {
+    display: flex;
+    justify-content: space-between; /* Distribute space evenly between the children */
+    align-items: center;
+    width: 100%; /* Full width to spread out the children */
+}
+
+.settings-option {
+    flex: 1; /* Each child will take equal space */
+    margin: 0 10px; /* Adjust spacing as needed */
+}
+
+.endpoint-option {
+    flex: 1; /* Each child will take equal space */
+    margin: 0 10px; /* Adjust spacing as needed */
+    margin-right: 25px;
+    width: 38%; /* Full width to spread out the children */
+}
+
+.website-row {
+    display: flex;
+    justify-content: start; /* Aligns items to the start of the row */
+    align-items: center;
+    margin-top: 10px; /* Top margin */
+    margin-bottom: 10px; /* Bottom margin */
+}
+
+.website-option {
+    flex: 1; /* Allows each option to take equal space */
+    margin-right: 10px; /* Spacing between the two options, adjust as needed */
+    margin-left: 10px; /* Adjust this value to align as desired */
+    /* Remove the individual margin top and bottom declarations if they are set globally on the .settings-row */
+}
+
+.settings-separator {
+    margin-top: 10px;
+    margin-bottom: 10px;
+    padding: 18x;
+    font-weight: bold;
+    border-top: 1px solid #e1e1e1; /* Grey line */
+    border-bottom: 1px solid #e1e1e1; /* Grey line */
+    text-align: center;
+}
+
+.status-message {
+    flex: 1; /* Adjust as needed */
+    margin: 0 10px;
+    /* Additional styling */
+}
+
+.model-endpoint-row {
+    display: flex;
+    justify-content: space-between;
+    align-items: center;
+    width: 100%;
+}
+
+.model-option, .endpoint-option {
+    flex: 1;
+    margin: 0 10px;
+    margin-left: 10px; /* Adjust this value to align as desired */
+    /* Adjust width as needed to fit in one line */
+}
+
+.endpoint-option {
+    /* Adjust width to align with model dropdown */
+    width: 38%;
+}
+
+#status_info {
+    color: lightgreen; /* Or any specific shade of green you prefer */
+}
+`;
+
+const style = document.createElement('style');
+style.type = 'text/css';
+style.appendChild(document.createTextNode(css));
+document.head.appendChild(style);
+
+class AllTalkTtsProvider {
+    //########//
+    // Config //
+    //########//
+
+    settings;
+    ready = false;
+    voices = [];
+    separator = '. ';
+    audioElement = document.createElement('audio');
+
+    languageLabels = {
+        'Arabic': 'ar',
+        'Brazilian Portuguese': 'pt',
+        'Chinese': 'zh-cn',
+        'Czech': 'cs',
+        'Dutch': 'nl',
+        'English': 'en',
+        'French': 'fr',
+        'German': 'de',
+        'Italian': 'it',
+        'Polish': 'pl',
+        'Russian': 'ru',
+        'Spanish': 'es',
+        'Turkish': 'tr',
+        'Japanese': 'ja',
+        'Korean': 'ko',
+        'Hungarian': 'hu',
+        'Hindi': 'hi',
+    };
+
+    Settings = {
+        provider_endpoint: 'http://localhost:7851',
+        language: 'en',
+        voiceMap: {},
+        at_generation_method: 'standard_generation',
+        narrator_enabled: 'false',
+        at_narrator_text_not_inside: 'narrator',
+        narrator_voice_gen: 'female_01.wav',
+        finetuned_model: 'false'
+    };
+
+    get settingsHtml() {
+        let html = `<div class="settings-separator">AllTalk Settings</div>`;
+
+        html += `<div class="settings-row">
+
+        <div class="settings-option">
+            <label for="at_generation_method">AllTalk TTS Generation Method</label>
+                <select id="at_generation_method">
+                <option value="standard_generation">Standard Audio Generation (AT Narrator - Optional)</option>
+                <option value="streaming_enabled">Streaming Audio Generation (AT Narrator - Disabled)</option>
+        </select>
+        </div>
+        </div>`;
+
+        html += `<div class="settings-row">
+
+        <div class="settings-option">
+            <label for="at_narrator_enabled">AT Narrator</label>
+                <select id="at_narrator_enabled">
+                <option value="true">Enabled</option>
+                <option value="false">Disabled</option>
+        </select>
+        </div>
+
+        <div class="settings-option">
+            <label for="at_narrator_text_not_inside">Text Not Inside * or " is</label>
+                <select id="at_narrator_text_not_inside">
+                <option value="narrator">Narrator</option>
+                <option value="character">Character</option>
+        </select>
+        </div>
+
+    </div>`;
+
+        html += `<div class="settings-row">
+        <div class="settings-option">
+            <label for="narrator_voice">Narrator Voice</label>
+            <select id="narrator_voice">`;
+        if (this.voices) {
+            for (let voice of this.voices) {
+                html += `<option value="${voice.voice_id}">${voice.name}</option>`;
+            }
+        }
+        html += `</select>
+        </div>
+        <div class="settings-option">
+            <label for="language_options">Language</label>
+            <select id="language_options">`;
+        for (let language in this.languageLabels) {
+            html += `<option value="${this.languageLabels[language]}" ${this.languageLabels[language] === this.settings?.language ? 'selected="selected"' : ''}>${language}</option>`;
+        }
+        html += `</select>
+        </div>
+    </div>`;
+
+
+        html += `<div class="model-endpoint-row">
+    <div class="model-option">
+            <label for="switch_model">Switch Model</label>
+            <select id="switch_model">
+            <option value="api_tts">API TTS</option>
+            <option value="api_local">API Local</option>
+            <option value="xttsv2_local">XTTSv2 Local</option>
+        </select>
+        </div>
+
+        <div class="endpoint-option">
+            <label for="at_server">AllTalk Endpoint:</label>
+            <input id="at_server" type="text" class="text_pole" maxlength="80" value="${this.Settings.provider_endpoint}"/>
+        </div>
+   </div>`;
+
+
+        html += `<div class="model-endpoint-row">
+    <div class="settings-option">
+        <label for="low_vram">Low VRAM</label>
+        <input id="low_vram" type="checkbox"/>
+    </div>
+    <div class="settings-option">
+        <label for="deepspeed">DeepSpeed</label>
+        <input id="deepspeed" type="checkbox"/>
+    </div>
+    <div class="settings-option status-option">
+        <span>Status: <span id="status_info">Ready</span></span>
+    </div>
+    <div class="settings-option empty-option">
+        <!-- This div remains empty for spacing -->
+    </div>
+</div>`;
+
+
+        html += `<div class="website-row">
+        <div class="website-option">
+        <span>AllTalk <a target="_blank" href="${this.Settings.provider_endpoint}">Config & Docs</a>.</span>
+    </div>
+
+    <div class="website-option">
+        <span>AllTalk <a target="_blank" href="https://github.com/erew123/alltalk_tts/">Website</a>.</span>
+    </div>
+</div>`;
+
+        return html;
+    }
+
+
+    //#################//
+    // Startup ST & AT //
+    //#################//
+
+    async loadSettings(settings) {
+        updateStatus('Offline');
+        // Use default settings if no settings are provided
+        this.settings = Object.keys(settings).length === 0 ? this.Settings : settings;
+
+        try {
+            // Check if TTS provider is ready
+            await this.checkReady();
+            await this.fetchTtsVoiceObjects(); // Fetch voices only if service is ready
+            await this.updateSettingsFromServer();
+            this.updateNarratorVoicesDropdown();
+            this.updateLanguageDropdown();
+            this.setupEventListeners();
+            this.applySettingsToHTML();
+            updateStatus('Ready');
+        } catch (error) {
+            console.error("Error loading settings:", error);
+            // Set status to Error if there is an issue in loading settings
+            updateStatus('Offline');
+        }
+    }
+
+    applySettingsToHTML() {
+        // Load AllTalk specific settings first
+        const loadedSettings = loadAllTalkSettings();
+        // Apply loaded settings or use defaults
+        this.settings = loadedSettings ? { ...this.Settings, ...loadedSettings } : this.Settings;
+        const narratorVoiceSelect = document.getElementById('narrator_voice');
+        const atNarratorSelect = document.getElementById('at_narrator_enabled');
+        const textNotInsideSelect = document.getElementById('at_narrator_text_not_inside');
+        const generationMethodSelect = document.getElementById('at_generation_method');
+        // Apply settings to Narrator Voice dropdown
+        if (narratorVoiceSelect && this.settings.narrator_voice) {
+            narratorVoiceSelect.value = this.settings.narrator_voice.replace('.wav', '');
+            this.settings.narrator_voice_gen = this.settings.narrator_voice;
+            //console.log(this.settings.narrator_voice_gen)
+        }
+        // Apply settings to AT Narrator Enabled dropdown
+        if (atNarratorSelect) {
+            //console.log(this.settings.narrator_enabled)
+            // Sync the state with the checkbox in index.js
+            const ttsPassAsterisksCheckbox = document.getElementById('tts_pass_asterisks'); // Access the checkbox from index.js
+            const ttsNarrateQuotedCheckbox = document.getElementById('tts_narrate_quoted'); // Access the checkbox from index.js
+            const ttsNarrateDialoguesCheckbox = document.getElementById('tts_narrate_dialogues'); // Access the checkbox from index.js
+            // Sync the state with the checkbox in index.js
+            if (this.settings.narrator_enabled) {
+                ttsPassAsterisksCheckbox.checked = false;
+                $('#tts_pass_asterisks').click(); // Simulate a click event
+                $('#tts_pass_asterisks').trigger('change');
+            }
+            if (!this.settings.narrator_enabled) {
+                ttsPassAsterisksCheckbox.checked = true;
+                $('#tts_pass_asterisks').click(); // Simulate a click event
+                $('#tts_pass_asterisks').trigger('change');
+            }
+            // Uncheck and set tts_narrate_quoted to false if narrator is enabled
+            if (this.settings.narrator_enabledd) {
+                ttsNarrateQuotedCheckbox.checked = true;
+                ttsNarrateDialoguesCheckbox.checked = true;
+                // Trigger click events instead of change events
+                $('#tts_narrate_quoted').click();
+                $('#tts_narrate_quoted').trigger('change');
+                $('#tts_narrate_dialogues').click();
+                $('#tts_narrate_dialogues').trigger('change');
+            }
+            atNarratorSelect.value = this.settings.narrator_enabled.toString();
+            this.settings.narrator_enabled = this.settings.narrator_enabled.toString();
+        }
+        // Apply settings to the Language dropdown
+        const languageSelect = document.getElementById('language_options');
+        if (languageSelect && this.settings.language) {
+            languageSelect.value = this.settings.language;
+            //console.log(this.settings.language)
+        }
+        // Apply settings to Text Not Inside dropdown
+        if (textNotInsideSelect && this.settings.text_not_inside) {
+            textNotInsideSelect.value = this.settings.text_not_inside;
+            this.settings.at_narrator_text_not_inside = this.settings.text_not_inside;
+            //console.log(this.settings.at_narrator_text_not_inside)
+        }
+        // Apply settings to Generation Method dropdown
+        if (generationMethodSelect && this.settings.at_generation_method) {
+            generationMethodSelect.value = this.settings.at_generation_method;
+            //console.log(this.settings.at_generation_method)
+        }
+        // Additional logic to disable/enable dropdowns based on the selected generation method
+        const isStreamingEnabled = this.settings.at_generation_method === 'streaming_enabled';
+        if (isStreamingEnabled) {
+            // Disable certain dropdowns when streaming is enabled
+            if (atNarratorSelect) atNarratorSelect.disabled = true;
+            if (textNotInsideSelect) textNotInsideSelect.disabled = true;
+            if (narratorVoiceSelect) narratorVoiceSelect.disabled = true;
+        } else {
+            // Enable dropdowns for standard generation
+            if (atNarratorSelect) atNarratorSelect.disabled = false;
+            if (textNotInsideSelect) textNotInsideSelect.disabled = !this.settings.narrator_enabled;
+            if (narratorVoiceSelect) narratorVoiceSelect.disabled = !this.settings.narrator_enabled;
+        }
+        const modelSelect = document.getElementById('switch_model');
+        if (this.settings.finetuned_model === 'true') {
+            const ftOption = document.createElement('option');
+            ftOption.value = 'XTTSv2 FT';
+            ftOption.textContent = 'XTTSv2 FT';
+            modelSelect.appendChild(ftOption);
+        }
+        //console.log("Settings applied to HTML.");
+    }
+
+    //##############################//
+    // Check AT Server is Available //
+    //##############################//
+
+    async checkReady() {
+        try {
+            const response = await fetch(`${this.settings.provider_endpoint}/api/ready`);
+            // Check if the HTTP request was successful
+            if (!response.ok) {
+                throw new Error(`HTTP Error Response: ${response.status} ${response.statusText}`);
+            }
+            const statusText = await response.text();
+            // Check if the response is 'Ready'
+            if (statusText === 'Ready') {
+                this.ready = true; // Set the ready flag to true
+                console.log('TTS service is ready.');
+            } else {
+                this.ready = false;
+                console.log('TTS service is not ready.');
+            }
+        } catch (error) {
+            console.error('Error checking TTS service readiness:', error);
+            this.ready = false; // Ensure ready flag is set to false in case of error
+        }
+    }
+
+    //######################//
+    // Get Available Voices //
+    //######################//
+
+    async fetchTtsVoiceObjects() {
+        const response = await fetch(`${this.settings.provider_endpoint}/api/voices`);
+        if (!response.ok) {
+            const errorText = await response.text();
+            throw new Error(`HTTP ${response.status}: ${errorText}`);
+        }
+        const data = await response.json();
+        const voices = data.voices.map(filename => {
+            const voiceName = filename.replace('.wav', '');
+            return {
+                name: voiceName,
+                voice_id: voiceName,
+                preview_url: null, // Preview URL will be dynamically generated
+                lang: 'en' // Default language
+            };
+        });
+        this.voices = voices; // Assign to the class property
+        return voices; // Also return this list
+    }
+
+    //##########################################//
+    // Get Current AT Server Config & Update ST //
+    //##########################################//
+
+    async updateSettingsFromServer() {
+        try {
+            // Fetch current settings
+            const response = await fetch(`${this.settings.provider_endpoint}/api/currentsettings`);
+            if (!response.ok) {
+                throw new Error(`Failed to fetch current settings: ${response.statusText}`);
+            }
+            const currentSettings = await response.json();
+            // Update internal settings
+            this.settings.modelsAvailable = currentSettings.models_available;
+            this.settings.currentModel = currentSettings.current_model_loaded;
+            this.settings.deepspeed_available = currentSettings.deepspeed_available;
+            this.settings.deepSpeedEnabled = currentSettings.deepspeed_status;
+            this.settings.lowVramEnabled = currentSettings.low_vram_status;
+            this.settings.finetuned_model = currentSettings.finetuned_model
+            // Update HTML elements
+            this.updateModelDropdown();
+            this.updateCheckboxes();
+        } catch (error) {
+            console.error(`Error updating settings from server: ${error}`);
+        }
+    }
+
+    //###################################################//
+    // Get Current AT Server Config & Update ST (Models) //
+    //###################################################//
+
+    updateModelDropdown() {
+        const modelSelect = document.getElementById('switch_model');
+        if (modelSelect) {
+            modelSelect.innerHTML = ''; // Clear existing options
+            this.settings.modelsAvailable.forEach(model => {
+                const option = document.createElement('option');
+                option.value = model.model_name;
+                option.textContent = model.model_name; // Use model_name instead of name
+                option.selected = model.model_name === this.settings.currentModel;
+                modelSelect.appendChild(option);
+            });
+        }
+    }
+
+    //#######################################################//
+    // Get Current AT Server Config & Update ST (DS and LVR) //
+    //#######################################################//
+
+    updateCheckboxes() {
+        const deepspeedCheckbox = document.getElementById('deepspeed');
+        const lowVramCheckbox = document.getElementById('low_vram');
+        if (lowVramCheckbox) lowVramCheckbox.checked = this.settings.lowVramEnabled;
+        if (deepspeedCheckbox) {
+            deepspeedCheckbox.checked = this.settings.deepSpeedEnabled;
+            deepspeedCheckbox.disabled = !this.settings.deepspeed_available; // Disable checkbox if deepspeed is not available
+        }
+    }
+
+    //###############################################################//
+    // Get Current AT Server Config & Update ST (AT Narrator Voices) //
+    //###############################################################//
+
+    updateNarratorVoicesDropdown() {
+        const narratorVoiceSelect = document.getElementById('narrator_voice');
+        if (narratorVoiceSelect && this.voices) {
+            // Clear existing options
+            narratorVoiceSelect.innerHTML = '';
+            // Add new options
+            for (let voice of this.voices) {
+                const option = document.createElement('option');
+                option.value = voice.voice_id;
+                option.textContent = voice.name;
+                narratorVoiceSelect.appendChild(option);
+            }
+        }
+    }
+
+    //######################################################//
+    // Get Current AT Server Config & Update ST (Languages) //
+    //######################################################//
+
+    updateLanguageDropdown() {
+        const languageSelect = document.getElementById('language_options');
+        if (languageSelect) {
+            // Ensure default language is set
+            this.settings.language = this.Settings.language;
+
+            languageSelect.innerHTML = '';
+            for (let language in this.languageLabels) {
+                const option = document.createElement('option');
+                option.value = this.languageLabels[language];
+                option.textContent = language;
+                if (this.languageLabels[language] === this.settings.language) {
+                    option.selected = true;
+                }
+                languageSelect.appendChild(option);
+            }
+        }
+    }
+
+    //########################################//
+    // Start AT TTS extenstion page listeners //
+    //########################################//
+
+    setupEventListeners() {
+
+        let debounceTimeout;
+        const debounceDelay = 500; // Milliseconds
+
+        // Define the event handler function
+        const onModelSelectChange = async (event) => {
+            console.log("Model select change event triggered"); // Debugging statement
+            const selectedModel = event.target.value;
+            console.log(`Selected model: ${selectedModel}`); // Debugging statement 
+            // Set status to Processing
+            updateStatus('Processing');
+            try {
+                const response = await fetch(`${this.settings.provider_endpoint}/api/reload?tts_method=${encodeURIComponent(selectedModel)}`, {
+                    method: 'POST'
+                });
+                if (!response.ok) {
+                    throw new Error(`HTTP Error: ${response.status}`);
+                }
+                const data = await response.json();
+                console.log("POST response data:", data); // Debugging statement 
+                // Set status to Ready if successful
+                updateStatus('Ready');
+            } catch (error) {
+                console.error("POST request error:", error); // Debugging statement
+                // Set status to Error in case of failure
+                updateStatus('Error');
+            }
+
+            // Handle response or error
+        };
+
+        const debouncedModelSelectChange = (event) => {
+            clearTimeout(debounceTimeout);
+            debounceTimeout = setTimeout(() => {
+                onModelSelectChange(event);
+            }, debounceDelay);
+        };
+
+        // Switch Model Listener
+        const modelSelect = document.getElementById('switch_model');
+        if (modelSelect) {
+            // Remove the event listener if it was previously added
+            modelSelect.removeEventListener('change', debouncedModelSelectChange);
+            // Add the debounced event listener
+            modelSelect.addEventListener('change', debouncedModelSelectChange);
+        }
+
+        // DeepSpeed Listener
+        const deepspeedCheckbox = document.getElementById('deepspeed');
+        if (deepspeedCheckbox) {
+            deepspeedCheckbox.addEventListener('change', async (event) => {
+                const deepSpeedValue = event.target.checked ? 'True' : 'False';
+                // Set status to Processing
+                updateStatus('Processing');
+                try {
+                    const response = await fetch(`${this.settings.provider_endpoint}/api/deepspeed?new_deepspeed_value=${deepSpeedValue}`, {
+                        method: 'POST'
+                    });
+                    if (!response.ok) {
+                        throw new Error(`HTTP Error: ${response.status}`);
+                    }
+                    const data = await response.json();
+                    console.log("POST response data:", data); // Debugging statement
+                    // Set status to Ready if successful
+                    updateStatus('Ready');
+                } catch (error) {
+                    console.error("POST request error:", error); // Debugging statement
+                    // Set status to Error in case of failure
+                    updateStatus('Error');
+                }
+            });
+        }
+
+        // Low VRAM Listener
+        const lowVramCheckbox = document.getElementById('low_vram');
+        if (lowVramCheckbox) {
+            lowVramCheckbox.addEventListener('change', async (event) => {
+                const lowVramValue = event.target.checked ? 'True' : 'False';
+                // Set status to Processing
+                updateStatus('Processing');
+                try {
+                    const response = await fetch(`${this.settings.provider_endpoint}/api/lowvramsetting?new_low_vram_value=${lowVramValue}`, {
+                        method: 'POST'
+                    });
+                    if (!response.ok) {
+                        throw new Error(`HTTP Error: ${response.status}`);
+                    }
+                    const data = await response.json();
+                    console.log("POST response data:", data); // Debugging statement
+                    // Set status to Ready if successful
+                    updateStatus('Ready');
+                } catch (error) {
+                    console.error("POST request error:", error); // Debugging statement
+                    // Set status to Error in case of failure
+                    updateStatus('Error');
+                }
+            });
+        }
+
+        // Narrator Voice Dropdown Listener
+        const narratorVoiceSelect = document.getElementById('narrator_voice');
+        if (narratorVoiceSelect) {
+            narratorVoiceSelect.addEventListener('change', (event) => {
+                this.settings.narrator_voice_gen = `${event.target.value}.wav`;
+                this.onSettingsChangeAllTalk(); // Save the settings after change
+            });
+        }
+
+        const textNotInsideSelect = document.getElementById('at_narrator_text_not_inside');
+        if (textNotInsideSelect) {
+            textNotInsideSelect.addEventListener('change', (event) => {
+                this.settings.text_not_inside = event.target.value;
+                this.onSettingsChangeAllTalk(); // Save the settings after change
+            });
+        }
+
+        // AT Narrator Dropdown Listener
+        const atNarratorSelect = document.getElementById('at_narrator_enabled');
+        const ttsPassAsterisksCheckbox = document.getElementById('tts_pass_asterisks'); // Access the checkbox from index.js
+        const ttsNarrateQuotedCheckbox = document.getElementById('tts_narrate_quoted'); // Access the checkbox from index.js
+        const ttsNarrateDialoguesCheckbox = document.getElementById('tts_narrate_dialogues'); // Access the checkbox from index.js
+
+        if (atNarratorSelect && textNotInsideSelect && narratorVoiceSelect) {
+            atNarratorSelect.addEventListener('change', (event) => {
+                const isNarratorEnabled = event.target.value === 'true';
+                this.settings.narrator_enabled = isNarratorEnabled; // Update the setting here
+                textNotInsideSelect.disabled = !isNarratorEnabled;
+                narratorVoiceSelect.disabled = !isNarratorEnabled;
+
+                // Sync the state with the checkbox in index.js
+                if (isNarratorEnabled) {
+                    ttsPassAsterisksCheckbox.checked = false;
+                    $('#tts_pass_asterisks').click(); // Simulate a click event
+                    $('#tts_pass_asterisks').trigger('change');
+                }
+                if (!isNarratorEnabled) {
+                    ttsPassAsterisksCheckbox.checked = true;
+                    $('#tts_pass_asterisks').click(); // Simulate a click event
+                    $('#tts_pass_asterisks').trigger('change');
+                }
+                // Uncheck and set tts_narrate_quoted to false if narrator is enabled
+                if (isNarratorEnabled) {
+                    ttsNarrateQuotedCheckbox.checked = true;
+                    ttsNarrateDialoguesCheckbox.checked = true;
+                    // Trigger click events instead of change events
+                    $('#tts_narrate_quoted').click();
+                    $('#tts_narrate_quoted').trigger('change');
+                    $('#tts_narrate_dialogues').click();
+                    $('#tts_narrate_dialogues').trigger('change');
+                }
+                this.onSettingsChangeAllTalk(); // Save the settings after change
+            });
+        }
+
+
+
+        // Event Listener for AT Generation Method Dropdown
+        const atGenerationMethodSelect = document.getElementById('at_generation_method');
+        const atNarratorEnabledSelect = document.getElementById('at_narrator_enabled');
+        if (atGenerationMethodSelect) {
+            atGenerationMethodSelect.addEventListener('change', (event) => {
+                const selectedMethod = event.target.value;
+
+                if (selectedMethod === 'streaming_enabled') {
+                    // Disable and unselect AT Narrator
+                    atNarratorEnabledSelect.disabled = true;
+                    atNarratorEnabledSelect.value = 'false';
+                    textNotInsideSelect.disabled = true;
+                    narratorVoiceSelect.disabled = true;
+                } else if (selectedMethod === 'standard_generation') {
+                    // Enable AT Narrator
+                    atNarratorEnabledSelect.disabled = false;
+                }
+                this.settings.at_generation_method = selectedMethod; // Update the setting here
+                this.onSettingsChangeAllTalk(); // Save the settings after change
+            });
+        }
+
+        // Listener for Language Dropdown
+        const languageSelect = document.getElementById('language_options');
+        if (languageSelect) {
+            languageSelect.addEventListener('change', (event) => {
+                this.settings.language = event.target.value;
+                this.onSettingsChangeAllTalk(); // Save the settings after change
+            });
+        }
+    }
+
+    //#############################//
+    // Store ST interface settings //
+    //#############################//
+
+    onSettingsChange() {
+        // Update settings that SillyTavern will save
+        this.settings.provider_endpoint = $('#at_server').val();
+        this.settings.language = $('#language_options').val();
+        saveTtsProviderSettings(); // This function should save settings handled by SillyTavern
+        // Call the function to handle AllTalk specific settings
+        this.onSettingsChangeAllTalk(); // Save the settings after change
+    }
+
+    onSettingsChangeAllTalk() {
+        // Update AllTalk specific settings and save to localStorage
+        this.settings.narrator_enabled = $('#at_narrator_enabled').val() === 'true';
+        this.settings.narrator_voice = $('#narrator_voice').val() + '.wav'; // assuming you need to append .wav
+        this.settings.text_not_inside = $('#at_narrator_text_not_inside').val(); // "character" or "narrator"
+        this.settings.at_generation_method = $('#at_generation_method').val(); // Streaming or standard
+        this.settings.language = $('#language_options').val(); // Streaming or standard
+
+        // Call the save function with the current settings
+        saveAllTalkSettings({
+            narrator_voice: this.settings.narrator_voice,
+            narrator_enabled: this.settings.narrator_enabled,
+            text_not_inside: this.settings.text_not_inside,
+            at_generation_method: this.settings.at_generation_method,
+            language: this.settings.language
+        });
+    }
+
+    //#########################//
+    // ST Handle Reload button //
+    //#########################//
+
+    async onRefreshClick() {
+        await this.checkReady(); // Check if the TTS provider is ready
+        await this.loadSettings(this.settings); // Reload the settings
+        // Additional actions as needed
+    }
+
+    //##################//
+    // Preview AT Voice //
+    //##################//
+
+    async previewTtsVoice(voiceName) {
+        try {
+            // Prepare data for POST request
+            const postData = new URLSearchParams();
+            postData.append("voice", `${voiceName}.wav`);
+            // Making the POST request
+            const response = await fetch(`${this.settings.provider_endpoint}/api/previewvoice/`, {
+                method: "POST",
+                headers: {
+                    'Content-Type': 'application/x-www-form-urlencoded'
+                },
+                body: postData,
+            });
+            if (!response.ok) {
+                const errorText = await response.text();
+                console.error(`[previewTtsVoice] Error Response Text:`, errorText);
+                throw new Error(`HTTP ${response.status}: ${errorText}`);
+            }
+            // Assuming the server returns a URL to the .wav file
+            const data = await response.json();
+            if (data.output_file_url) {
+                // Use an audio element to play the .wav file
+                const audioElement = new Audio(data.output_file_url);
+                audioElement.play().catch(e => console.error("Error playing audio:", e));
+            } else {
+                console.warn("[previewTtsVoice] No output file URL received in the response");
+                throw new Error("No output file URL received in the response");
+            }
+
+        } catch (error) {
+            console.error("[previewTtsVoice] Exception caught during preview generation:", error);
+            throw error;
+        }
+    }
+
+    //#####################//
+    //  Populate ST voices //
+    //#####################//
+
+    async getVoice(voiceName, generatePreview = false) {
+        // Ensure this.voices is populated
+        if (this.voices.length === 0) {
+            // Fetch voice objects logic
+        }
+        // Find the object where the name matches voiceName
+        const match = this.voices.find(voice => voice.name === voiceName);
+        if (!match) {
+            // Error handling
+        }
+        // Generate preview URL only if requested
+        if (!match.preview_url && generatePreview) {
+            // Generate preview logic
+        }
+        return match; // Return the found voice object
+    }
+
+    //##########################################//
+    //  Generate TTS Streaming or call Standard //
+    //##########################################//
+
+    async generateTts(inputText, voiceId) {
+        try {
+            if (this.settings.at_generation_method === 'streaming_enabled') {
+                // For streaming method
+                const streamingUrl = `${this.settings.provider_endpoint}/api/tts-generate-streaming?text=${encodeURIComponent(inputText)}&voice=${encodeURIComponent(voiceId)}.wav&language=${encodeURIComponent(this.settings.language)}&output_file=stream_output.wav`;
+                const audioElement = new Audio(streamingUrl);
+                audioElement.play(); // Play the audio stream directly
+                console.log("Streaming URL:", streamingUrl);
+                return new Response(null, {
+                    status: 200,
+                    statusText: "OK",
+                    headers: {
+                        "Content-Type": "audio/wav",
+                        "Content-Location": streamingUrl
+                    }
+                });
+            } else {
+                // For standard method
+                const outputUrl = await this.fetchTtsGeneration(inputText, voiceId);
+                // Fetch the audio data as a blob from the URL
+                const audioResponse = await fetch(outputUrl);
+                if (!audioResponse.ok) {
+                    throw new Error(`HTTP ${audioResponse.status}: Failed to fetch audio data`);
+                }
+                const audioBlob = await audioResponse.blob();
+                if (!audioBlob.type.startsWith('audio/')) {
+                    throw new Error(`Invalid audio data format. Expecting audio/*, got ${audioBlob.type}`);
+                }
+                return new Response(audioBlob, {
+                    status: 200,
+                    statusText: "OK",
+                    headers: {
+                        "Content-Type": audioBlob.type
+                    }
+                });
+            }
+        } catch (error) {
+            console.error("Error in generateTts:", error);
+            throw error;
+        }
+    }
+
+    //####################//
+    //  Generate Standard //
+    //####################//
+
+    async fetchTtsGeneration(inputText, voiceId) {
+        // Prepare the request payload
+        const requestBody = new URLSearchParams({
+            'text_input': inputText,
+            'text_filtering': "standard",
+            'character_voice_gen': voiceId + ".wav",
+            'narrator_enabled': this.settings.narrator_enabled,
+            'narrator_voice_gen': this.settings.narrator_voice_gen,
+            'text_not_inside': this.settings.text_not_inside,
+            'language': this.settings.language,
+            'output_file_name': "st_output",
+            'output_file_timestamp': "true",
+            'autoplay': "false",
+            'autoplay_volume': "0.8"
+        }).toString();
+
+        try {
+            const response = await doExtrasFetch(
+                `${this.settings.provider_endpoint}/api/tts-generate`,
+                {
+                    method: 'POST',
+                    headers: {
+                        'Content-Type': 'application/x-www-form-urlencoded',
+                        'Cache-Control': 'no-cache',
+                    },
+                    body: requestBody
+                }
+            );
+
+            if (!response.ok) {
+                const errorText = await response.text();
+                console.error(`[fetchTtsGeneration] Error Response Text:`, errorText);
+                // Uncomment the following line if you have a UI element for displaying errors
+                // toastr.error(response.statusText, 'TTS Generation Failed');
+                throw new Error(`HTTP ${response.status}: ${errorText}`);
+            }
+            const data = await response.json();
+            const outputUrl = data.output_file_url;
+            return outputUrl; // Return only the output_file_url
+        } catch (error) {
+            console.error("[fetchTtsGeneration] Exception caught:", error);
+            throw error; // Rethrow the error for further handling
+        }
+    }
+}
+
+//#########################//
+//  Update Status Messages //
+//#########################//
+
+function updateStatus(message) {
+    const statusElement = document.getElementById('status_info');
+    if (statusElement) {
+        statusElement.textContent = message;
+        switch (message) {
+            case 'Offline':
+                statusElement.style.color = 'red';
+                break;
+            case 'Ready':
+                statusElement.style.color = 'lightgreen';
+                break;
+            case 'Processing':
+                statusElement.style.color = 'blue';
+                break;
+            case 'Error':
+                statusElement.style.color = 'red';
+                break;
+            // Add more cases as needed
+        }
+    }
+}
+
+//########################//
+//  Save/load AT Settings //
+//########################//
+
+function saveAllTalkSettings(settingsToSave) {
+    // Save the specific settings to localStorage
+    console.log("settings", settingsToSave)
+    localStorage.setItem('AllTalkSettings', JSON.stringify(settingsToSave));
+}
+
+function loadAllTalkSettings() {
+    // Retrieve the settings from localStorage
+    const settings = localStorage.getItem('AllTalkSettings');
+    // If settings exist, parse them back into an object
+    if (settings) {
+        return JSON.parse(settings);
+    }
+    return null;
+}

templates/admin.html

@@ -222,6 +222,7 @@
         <li><a href="#using-voice-samples">Using Voice Samples</a></li>
         <li><a href="#text-not-inside">Text Not inside</a></li>
         <li><a href="#low-vram">Low VRAM</a></li>
+        <li><a href="#sillytavern-support">Silly Tavern Support</a></li>
         <li><a href="#finetune">Finetuning (Training the model)</a></li>
         <li><a href="#deepspeed">DeepSpeed</a>
             <ul>
@@ -272,8 +273,8 @@
             here</a><br />Update instructions <a href="https://github.com/erew123/alltalk_tts?#-updating"
             target="_blank" rel="noopener">link here</a><br />Help and issues <a
             href="https://github.com/erew123/alltalk_tts?#-help-with-problems" target="_blank" rel="noopener">link
-            here</a><br />TTS Generator <a
+            here</a><br />TTS Generator <a href="http://127.0.0.1:7851/static/tts_chunk_player/tts_chunk_player.html"
-            href="http://127.0.0.1:7851/static/tts_chunk_player/tts_chunk_player.html" target="_blank" rel="noopener">link
+            target="_blank" rel="noopener">link
             here<br /><br /></a><span style="text-decoration: underline;"><strong>Text generation webUI</strong></span>
         <br />Web interface&nbsp;<a href="http://{{ params.ip_address }}:7860" target="_blank" rel="noopener">link
             here</a> <br />Documentation <a href="https://github.com/oobabooga/text-generation-webui/wiki"
@@ -291,7 +292,8 @@
     <p style="padding-left: 30px;"><a href="#toc">Back to top of page</a></p>
 
     <h2 id="demotesttts">Demo/Test TTS</h2>
-    <p style="padding-left: 30px;">If you are wanting to generate bulk quantites of TTS and have control over them, please see the <a href="#tts-generator">AllTalk TTS Generator</a> below.</p>
+    <p style="padding-left: 30px;">If you are wanting to generate bulk quantites of TTS and have control over them,
+        please see the <a href="#tts-generator">AllTalk TTS Generator</a> below.</p>
     <div id="container">
         <form method="post" action="/tts-demo-request" id="ttsForm">
             <label for="text">Text:</label>
@@ -658,6 +660,100 @@
 
     <p style="padding-left: 30px;"><a href="#toc">Back to top of page</a></p>
 
+    <h2 id="sillytavern-support"><strong>SillyTavern Support</strong></h2>
+    <h4>Important note for Text-generation-webui users</h4>
+    <p style="padding-left: 30px;">You <strong><span style="color: #3366ff;">HAVE</span></strong> to disable
+        <strong>Activate TTS</strong> or <strong>Enable TTS</strong> within the Text-generation-webui AllTalk interface,
+        otherwise&nbsp;Text-generation-webui will also generate TTS due to the way it sends out text. You can do this
+        each time you start up&nbsp;Text-generation-webui or set it in the start-up settings at the top of this page.
+    </p>
+    <h4>TTS Generation Methods in SilllyTavern</h4>
+    <p style="padding-left: 30px;">You have 2 types of audio generation options, Streaming and Standard.</p>
+    <p style="padding-left: 30px; text-align: justify;">The Streaming Audio Generation method is designed for speed and
+        is best suited for situations where you just want quick audio playback. This method, however, is limited to
+        using just one voice per TTS generation request. This means a limitation of the Streaming method is the
+        inability to utilize the AllTalk narrator function, making it a straightforward but less nuanced option.</p>
+    <p style="padding-left: 30px; text-align: justify;">On the other hand, the Standard Audio Generation method provides
+        a richer auditory experience. It's slightly slower than the Streaming method but compensates for this with its
+        ability to split text into multiple voices. This functionality is particularly useful in scenarios where
+        differentiating between character dialogues and narration can enhance the storytelling and delivery. The
+        inclusion of the AllTalk narrator functionality in the Standard method allows for a more layered and immersive
+        experience, making it ideal for content where depth and variety in voice narration add significant value.</p>
+    <p style="padding-left: 30px; text-align: justify;">In summary, the choice between Streaming and Standard methods in
+        AllTalk TTS depends on what you want. Streaming is great for quick and simple audio generation, while Standard
+        is preferable for a more dynamic and engaging audio experience.</p>>
+    <ul>
+        <li><strong>AllTalk TTS Generation Method:</strong>
+            <ul>
+                <li>Select between Standard and Streaming Audio Generation methods.</li>
+                <li>This setting impacts the AllTalk narrator functionality.</li>
+            </ul>
+        </li>
+        <li><strong>Language Selection:</strong><br />
+            <ul>
+                <li>Select your preferred TTS generation language from the "Language" dropdown.</li>
+            </ul>
+        </li>
+        <li><strong>Model Switching:</strong>
+            <ul>
+                <li>Switch between different TTS models like API TTS, API Local, XTTSv2 Local, and optionally XTTSv2 FT
+                    if you have a finetuned model available.</li>
+                <li>Fine-tuned model availability (XTTSv2 FT) will only show when a finetuned model is detected by
+                    AllTalk.</li>
+                <li>See TTS Models/Methods for more information (though most people will want to stick with XTTSv2
+                    Local).</li>
+            </ul>
+        </li>
+        <li><strong>DeepSpeed and Low VRAM Options:</strong>
+            <ul>
+                <li>Optimize performance with DeepSpeed and Low VRAM settings.</li>
+                <li>DeepSpeed can offer a 2-3x performance boost on TTS generation.</li>
+                <li>See the relevant sections in this documentation for details.</li>
+            </ul>
+        </li>
+    </ul>
+    <p style="padding-left: 30px;">Changing model or DeepSpeed or Low VRAM <span
+            style="color: #0000ff;"><strong>each</strong></span> take about 15 seconds so you should only change one at
+        a time and wait for <span style="color: #99cc00;">Ready</span> before changing the next setting. To set these
+        options long term you can apply the settings at the top of this page.</p>
+    <h4>AllTalk Narrator</h4>
+    <p style="padding-left: 30px;">Only available on the Standard Audo Generation method.</p>
+    <ul>
+        <li><strong>Narrator Voice Selection:</strong>
+            <ul>
+                <li>Allows users to choose different narrator voices.</li>
+                <li>Access via the "Narrator Voice" dropdown.</li>
+            </ul>
+        </li>
+        <li><strong>AllTalk Narrator:</strong>
+            <ul>
+                <li>Toggle the AllTalk narrator feature.</li>
+                <li>Access via "AT Narrator" dropdown with Enabled/Disabled options.</li>
+            </ul>
+        </li>
+        <li><strong>Text Outside Asterisks Handling:</strong>
+            <ul>
+                <li>Choose how text outside asterisks is interpreted (as Narrator or Character voice).</li>
+                <li>Managed via the "Text Not Inside * or "" dropdown.</li>
+                <li>Note, only available when the AllTalk Narrator is enabled.</li>
+            </ul>
+        </li>
+    </ul>
+    <h4>Usage Notes:</h4>
+    <ul>
+        <li>On startup of SillyTavern, it will pull your current settings from AllTalk (Current model, DeepSpeed status,
+            Low VRAM status and Finetuned model availability).</li>
+        <li>Enabling the narrator automatically unchecks certain checkboxes related to text handling.</li>
+        <li>Changes in model or settings might trigger multiple requests to the server; patience is advised.</li>
+    </ul>
+    <h4>Troubleshooting:</h4>
+    <ul>
+        <li>If experiencing issues, use the Reload button in SillyTavern's TTS extension to reinitialize the connection
+            to AllTalk and check if AllTalk is started correctly.</li>
+    </ul>
+
+    <p style="padding-left: 30px;"><a href="#toc">Back to top of page</a></p>
+
     <h2 id="finetune"><strong>Finetuning (Training the model)</strong></h2>
     <p style="margin-left:40px; text-align:justify">If you have a voice that the model doesnt quite reproduce correctly,
         or indeed you just want to improve the reproduced voice, then finetuning is a way to train your &quot;XTTSv2

tts_server.py

@@ -81,6 +81,17 @@ else:
         f"[{params['branding']}Startup] \033[91mWarning\033[0m modeldownload.config is missing so please re-download it and save it in the alltalk_tts main folder."
     )
 
+##################################################
+#### Check to see if a finetuned model exists ####
+##################################################
+# Set the path to the directory
+trained_model_directory = this_dir / "models" / "trainedmodel"
+# Check if the directory "trainedmodel" exists
+finetuned_model = trained_model_directory.exists()
+# If the directory exists, check for the existence of the required files
+if finetuned_model:
+    required_files = ["model.pth", "config.json", "vocab.json"]
+    finetuned_model = all((trained_model_directory / file).exists() for file in required_files)
 
 ########################
 #### STARTUP CHECKS ####
@@ -98,19 +109,17 @@ except ModuleNotFoundError:
     raise
 
 # DEEPSPEED Import - Check for DeepSpeed and import it if it exists
+deepspeed_available = False
 try:
     import deepspeed
-
+    deepspeed_available = True
-    deepspeed_installed = True
-    print(f"[{params['branding']}Startup] DeepSpeed \033[93mDetected\033[0m")
-    print(
-        f"[{params['branding']}Startup] Activate DeepSpeed in {params['branding']} settings"
-    )
 except ImportError:
-    deepspeed_installed = False
+    pass
-    print(
+if deepspeed_available:
-        f"[{params['branding']}Startup] DeepSpeed \033[93mNot Detected\033[0m. See https://github.com/microsoft/DeepSpeed"
+    print(f"[{params['branding']}Startup] DeepSpeed \033[93mDetected\033[0m")
-    )
+    print(f"[{params['branding']}Startup] Activate DeepSpeed in {params['branding']} settings")
+else:
+    print(f"[{params['branding']}Startup] DeepSpeed \033[93mNot Detected\033[0m. See https://github.com/microsoft/DeepSpeed")
 
 
 @asynccontextmanager
@@ -729,6 +738,11 @@ async def tts_demo_request(request: Request, text: str = Form(...), voice: str =
         print(f"An error occurred: {e}")
         return JSONResponse(content={"error": "An error occurred"}, status_code=500)
 
+
+#####################
+#### Audio feeds ####
+#####################
+
 # Gives web access to the output files
 @app.get("/audio/{filename}")
 async def get_audio(filename: str):
@@ -808,6 +822,34 @@ import numpy as np
 import soundfile as sf
 import sys
 
+##############################
+#### Streaming Generation ####
+##############################
+
+@app.get("/api/tts-generate-streaming", response_class=StreamingResponse)
+async def tts_generate_streaming(text: str, voice: str, language: str, output_file: str):
+    try:
+        output_file_path = this_dir / "outputs" / output_file
+        stream = await generate_audio(text, voice, language, output_file_path, streaming=True)
+        return StreamingResponse(stream, media_type="audio/wav")
+    except Exception as e:
+        print(f"An error occurred: {e}")
+        return JSONResponse(content={"error": "An error occurred"}, status_code=500)
+
+@app.post("/api/tts-generate-streaming", response_class=JSONResponse)
+async def tts_generate_streaming(request: Request, text: str = Form(...), voice: str = Form(...), language: str = Form(...), output_file: str = Form(...)):
+    try:
+        output_file_path = this_dir / "outputs" / output_file
+        await generate_audio(text, voice, language, output_file_path, streaming=False)
+        return JSONResponse(content={"output_file_path": str(output_file)}, status_code=200)
+    except Exception as e:
+        print(f"An error occurred: {e}")
+        return JSONResponse(content={"error": "An error occurred"}, status_code=500)
+
+##############################
+#### Standard Generation ####
+##############################
+
 # Check for PortAudio library on Linux
 try:
     import sounddevice as sd
@@ -1053,6 +1095,38 @@ async def tts_generate(
         return JSONResponse(content={"status": "generate-failure", "error": "An error occurred"}, status_code=500)
 
 
+##########################
+#### Current Settings ####
+##########################
+# Define the available models
+models_available = [
+    {"name": "Coqui", "model_name": "API TTS"},
+    {"name": "Coqui", "model_name": "API Local"},
+    {"name": "Coqui", "model_name": "XTTSv2 Local"}
+]
+
+@app.get('/api/currentsettings')
+def get_current_settings():
+    # Determine the current model loaded
+    if params["tts_method_api_tts"]:
+        current_model_loaded = "API TTS"
+    elif params["tts_method_api_local"]:
+        current_model_loaded = "API Local"
+    elif params["tts_method_xtts_local"]:
+        current_model_loaded = "XTTSv2 Local"
+    else:
+        current_model_loaded = None  # or a default value if no method is active
+
+    settings = {
+        "models_available": models_available,
+        "current_model_loaded": current_model_loaded,
+        "deepspeed_available": deepspeed_available,
+        "deepspeed_status": params["deepspeed_activate"],
+        "low_vram_status": params["low_vram"],
+        "finetuned_model": finetuned_model
+    }
+    return settings  # Automatically converted to JSON by Fas
+
 #############################
 #### Word Add-in Sharing ####
 #############################