Merge pull request #31 from erew123/dev

3x new API endpoints & Play audio at command prompt/terminal

README.md

@@ -26,7 +26,9 @@ AllTalk is an updated version of the Coqui_tts extension for Text Generation web
 - 🔴 [Future to-do list & Upcoming updates](https://github.com/erew123/alltalk_tts?#-future-to-do-list)
 
 #### Updates
-Dec 25th - Applied a small update to avoid a possible race condition on file naming with small sentences when generating narrator/character speech. Please update if you are experiencing missing small bits of missing audio on narrator/character.
+**Dec 28th**- Finetuning has been updated to compact models and make the final steps easier (buttons basically). Narrator has been updated. If you wish to compact existing finetuned models, pre the update please see [here](https://github.com/erew123/alltalk_tts/issues/28)
+
+**Dec 25th** - Applied a small update to avoid a possible race condition on file naming with small sentences when generating narrator/character speech. Please update if you are experiencing missing small bits of missing audio on narrator/character.
 
 ####     🔄 **Minor updates/bug fixes list** can be found [here](https://github.com/erew123/alltalk_tts/issues/25)
 
@@ -497,9 +499,67 @@ Deepspeed and other such things can be installed. Please read the relevant instr
 ### 🟠Overview
 The Text-to-Speech (TTS) Generation API allows you to generate speech from text input using various configuration options. This API supports both character and narrator voices, providing flexibility for creating dynamic and engaging audio content.
 
-- URL: `http://127.0.0.1:7851/api/tts-generate`<br>
+#### 🟠 Ready Endpoint<br>
-- Method: `POST`<br>
+Check if the Text-to-Speech (TTS) service is ready to accept requests.
-- Content-Type: `application/x-www-form-urlencoded`<br>
+
+- URL: `http://127.0.0.1:7851/api/ready`<br> - Method: `GET`<br> 
+
+   `curl -X GET "http://127.0.0.1:7851/api/ready"`
+
+  Response: `Ready`
+
+#### 🟠 Voices List Endpoint<br>
+Retrieve a list of available voices for generating speech.
+
+- URL: `http://127.0.0.1:7851/api/voices`<br> - Method: `GET`<br>
+
+   `curl -X GET "http://127.0.0.1:7851/api/voices"`
+
+   JSON return: `{"voices": ["voice1.wav", "voice2.wav", "voice3.wav"]}`
+
+#### 🟠 Preview Voice Endpoint
+Generate a preview of a specified voice with hardcoded settings.
+
+- URL: `http://127.0.0.1:7851/api/previewvoice/`<br> - Method: `POST`<br> - Content-Type: `application/x-www-form-urlencoded`<br>
+
+   `curl -X POST "http://127.0.0.1:7851/api/previewvoice/" -F "voice=female_01.wav"`
+
+   Replace `female_01.wav` with the name of the voice sample you want to hear.
+
+   JSON return: `{"status": "generate-success", "output_file_path": "/path/to/outputs/api_preview_voice.wav", "output_file_url": "http://127.0.0.1:7851/audio/api_preview_voice.wav"}`
+
+#### 🟠 Switching Model Endpoint<br>
+
+- URL: `http://127.0.0.1:7851/api/reload`<br> - Method: `POST`<br><br>
+   `curl -X POST "http://127.0.0.1:7851/api/reload?tts_method=API%20Local"`<br>
+   `curl -X POST "http://127.0.0.1:7851/api/reload?tts_method=API%20TTS"`<br>
+   `curl -X POST "http://127.0.0.1:7851/api/reload?tts_method=XTTSv2%20Local"`<br>
+
+   Switch between the 3 models respectively.
+
+   JSON return `{"status": "model-success"}`
+
+#### 🟠 Switch DeepSpeed Endpoint<br>
+
+- URL: `http://127.0.0.1:7851/api/deepspeed`<br> - Method: `POST`<br><br>
+   `curl -X POST "http://127.0.0.1:7851/api/deepspeed?new_deepspeed_value=True"`
+
+   Replace True with False to disable DeepSpeed mode.
+
+   JSON return `{"status": "deepspeed-success"}`
+
+#### 🟠 Switching Low VRAM Endpoint<br>
+
+- URL: `http://127.0.0.1:7851/api/lowvramsetting`<br> - Method: `POST`<br><br>
+   `curl -X POST "http://127.0.0.1:7851/api/lowvramsetting?new_low_vram_value=True"`
+
+   Replace True with False to disable Low VRAM mode.
+
+   JSON return `{"status": "lowvram-success"}`
+
+### 🟠 TTS Generation Endpoint
+
+- URL: `http://127.0.0.1:7851/api/tts-generate`<br> - Method: `POST`<br> - Content-Type: `application/x-www-form-urlencoded`<br>
 
 ### 🟠 Example command lines
 Standard TTS speech Example (standard text) generating a time-stamped file<br>
@@ -556,22 +616,22 @@ Example:
 
 🟠 **language**: Choose the language for TTS. Options:
 
-`ar Arabic`<br>
+`ar` Arabic<br>
-`zh-cn Chinese (Simplified)`<br>
+`zh-cn` Chinese (Simplified)<br>
-`cs Czech`<br>
+`cs` Czech<br>
-`nl Dutch`<br>
+`nl` Dutch<br>
-`en English`<br>
+`en` English<br>
-`fr French`<br>
+`fr` French<br>
-`de German`<br>
+`de` German<br>
-`hu Hungarian`<br>
+`hu` Hungarian<br>
-`it Italian`<br>
+`it` Italian<br>
-`ja Japanese`<br>
+`ja` Japanese<br>
-`ko Korean`<br>
+`ko` Korean<br>
-`pl Polish`<br>
+`pl` Polish<br>
-`pt Portuguese`<br>
+`pt` Portuguese<br>
-`ru Russian`<br>
+`ru` Russian<br>
-`es Spanish`<br>
+`es` Spanish<br>
-`tr Turkish`<br>
+`tr` Turkish<br>
 
 `-d "language=en"`<br>
 
@@ -584,12 +644,12 @@ Example:
 `-d "output_file_timestamp=true"`<br>
 `-d "output_file_timestamp=false"`
 
-🟠 **autoplay**: Feature not yet available. Enable or disable autoplay. Still needs to be specified in the JSON request.
+🟠 **autoplay**: Enable or disable playing the generated TTS to your standard sound output device at time of TTS generation.
 
 `-d "autoplay=true"`<br>
 `-d "autoplay=false"`
 
-🟠 **autoplay_volume**: Feature not yet available. Set the autoplay volume. Should be between 0.1 and 1.0. Still needs to be specified in the JSON request.
+🟠 **autoplay_volume**: Set the autoplay volume. Should be between 0.1 and 1.0. Needs to be specified in the JSON request even if autoplay is false.
 
 `-d "autoplay_volume=0.8"`
 
@@ -604,29 +664,6 @@ Example JSON TTS Generation Response:
 
 `{"status": "generate-success", "output_file_path": "C:\text-generation-webui\extensions\alltalk_tts\outputs\myoutputfile_1703149973.wav", "output_file_url": "http://127.0.0.1:7851/audio/myoutputfile_1703149973.wav"}`
 
-🟠 **Switching Model**<br><br>
-`curl -X POST "http://127.0.0.1:7851/api/reload?tts_method=API%20Local"`<br>
-`curl -X POST "http://127.0.0.1:7851/api/reload?tts_method=API%20TTS"`<br>
-`curl -X POST "http://127.0.0.1:7851/api/reload?tts_method=XTTSv2%20Local"`<br>
-
-Switch between the 3 models respectively.
-
-JSON return `{"status": "model-success"}`
-
-🟠 **Switch DeepSpeed**<br><br>
-`curl -X POST "http://127.0.0.1:7851/api/deepspeed?new_deepspeed_value=True"`
-
-Replace True with False to disable DeepSpeed mode.
-
-JSON return `{"status": "deepspeed-success"}`
-
-🟠 **Switching Low VRAM**<br><br>
-`curl -X POST "http://127.0.0.1:7851/api/lowvramsetting?new_low_vram_value=True"`
-
-Replace True with False to disable Low VRAM mode.
-
-JSON return `{"status": "lowvram-success"}`
-
 ### 🔴 Future to-do list
 - Voice output within the command prompt/terminal (TBD).
 - Correct a few spelling mistakes in the documentation.

modeldownload.json

@@ -8,6 +8,7 @@
         "model.pth": "https://huggingface.co/coqui/XTTS-v2/resolve/v2.0.2/model.pth?download=true",
         "dvae.pth": "https://huggingface.co/coqui/XTTS-v2/resolve/v2.0.2/dvae.pth?download=true",
         "mel_stats.pth": "https://huggingface.co/coqui/XTTS-v2/resolve/v2.0.2/mel_stats.pth?download=true",
+        "speakers_xtts.pth": "https://huggingface.co/coqui/XTTS-v2/resolve/v2.0.2/speakers_xtts.pth?download=true",
         "vocab.json": "https://huggingface.co/coqui/XTTS-v2/resolve/v2.0.2/vocab.json?download=true"
     }
-}
+}

requirements_nvidia.txt

@@ -11,3 +11,5 @@ tqdm>=4.66.1
 importlib-metadata>=4.8.1
 packaging>=23.2
 pydantic>=1.10.13
+sounddevice>=0.4.6
+python-multipart>=0.0.6

requirements_other.txt

@@ -11,3 +11,5 @@ tqdm>=4.66.1
 importlib-metadata>=4.8.1
 packaging>=23.2
 pydantic>=1.10.13
+sounddevice>=0.4.6
+python-multipart>=0.0.6

script.py

@@ -252,7 +252,7 @@ else:
 timeout = 120  # Adjust the timeout as needed
 
 # Introduce a delay before starting the check loop
-time.sleep(25)  # Wait 25 secs before checking if the tts_server.py has started up.
+time.sleep(26)  # Wait 26 secs before checking if the tts_server.py has started up.
 start_time = time.time()
 while time.time() - start_time < timeout:
     try:
@@ -264,7 +264,7 @@ while time.time() - start_time < timeout:
         print(
             f"[{params['branding']}Startup] \033[91mWarning\033[0m TTS Subprocess has NOT started up yet, Will keep trying for 120 seconds maximum. Please wait."
         )
-    time.sleep(1)
+    time.sleep(4)
 else:
     print(
         f"[{params['branding']}Startup] Startup timed out. Check the server logs for more information."
@@ -494,7 +494,7 @@ def output_modifier(string, state):
     if not params["activate"]:
         return string
     original_string = string
-    print("ORGINAL STRING: ", original_string)
+    #print("ORGINAL STRING: ", original_string)
     cleaned_string = before_audio_generation(string, params)
     if cleaned_string is None:
         return

tts_server.py

@@ -619,6 +619,53 @@ async def get_audio(filename: str):
     audio_path = this_dir / "outputs" / filename
     return FileResponse(audio_path)
 
+
+#########################
+#### VOICES LIST API ####
+#########################
+# Define the new endpoint
+@app.get("/api/voices")
+async def get_voices():
+    wav_files = list_files(this_dir / "voices")
+    return {"voices": wav_files}
+
+###########################
+#### PREVIEW VOICE API ####
+###########################
+@app.post("/api/previewvoice/", response_class=JSONResponse)
+async def preview_voice(request: Request, voice: str = Form(...)):
+    try:
+        # Hardcoded settings
+        language = "en"
+        output_file_name = "api_preview_voice"
+
+        # Clean the voice filename for inclusion in the text
+        clean_voice_filename = re.sub(r'\.wav$', '', voice.replace(' ', '_'))
+        clean_voice_filename = re.sub(r'[^a-zA-Z0-9]', ' ', clean_voice_filename)
+        
+        # Generate the audio
+        text = f"Hello, this is a preview of voice {clean_voice_filename}."
+
+        # Generate the audio
+        output_file_path = this_dir / "outputs" / f"{output_file_name}.wav"
+        await generate_audio(text, voice, language, output_file_path)
+
+        # Generate the URL
+        output_file_url = f'http://{params["ip_address"]}:{params["port_number"]}/audio/{output_file_name}.wav'
+
+        # Return the response with both local file path and URL
+        return JSONResponse(
+            content={
+                "status": "generate-success",
+                "output_file_path": str(output_file_path),
+                "output_file_url": str(output_file_url),
+            },
+            status_code=200,
+        )
+    except Exception as e:
+        print(f"An error occurred: {e}")
+        return JSONResponse(content={"error": "An error occurred"}, status_code=500)
+
 ########################
 #### GENERATION API ####
 ########################
@@ -627,9 +674,29 @@ import re
 import uuid
 import numpy as np
 import soundfile as sf
+import sys
+
+# Check for PortAudio library on Linux
+try:
+    import sounddevice as sd
+    sounddevice_installed=True
+except OSError:
+    print(f"[{params['branding']}Startup] \033[91mInfo\033[0m PortAudio library not found. If you wish to play TTS in standalone mode through the API suite")
+    print(f"[{params['branding']}Startup] \033[91mInfo\033[0m please install PortAudio. This will not affect any other features or use of Alltalk.")
+    print(f"[{params['branding']}Startup] \033[91mInfo\033[0m If you don't know what the API suite is, then this message is nothing to worry about.")
+    sounddevice_installed=False
+    if sys.platform.startswith('linux'):
+        print(f"[{params['branding']}Startup] \033[91mInfo\033[0m On Linux, you can use the following command to install PortAudio:")
+        print(f"[{params['branding']}Startup] \033[91mInfo\033[0m sudo apt-get install portaudio19-dev")
+
 from typing import Union, Dict
 from pydantic import BaseModel, ValidationError, Field
 
+def play_audio(file_path, volume):
+    data, fs = sf.read(file_path)
+    sd.play(volume * data, fs)
+    sd.wait()
+
 class Request(BaseModel):
     # Define the structure of the 'Request' class if needed
     pass
@@ -827,6 +894,10 @@ async def tts_generate(
             else:
                 cleaned_string = text_input
             await generate_audio(cleaned_string, character_voice_gen, language, output_file_path)
+        if sounddevice_installed == False:
+            autoplay = False
+        if autoplay:
+            play_audio(output_file_path, autoplay_volume)
         return JSONResponse(content={"status": "generate-success", "output_file_path": str(output_file_path), "output_file_url": str(output_file_url)}, status_code=200)
     except Exception as e:
         return JSONResponse(content={"status": "generate-failure", "error": "An error occurred"}, status_code=500)
@@ -1673,6 +1744,33 @@ simple_webpage = """
 <p style="padding-left: 30px;"><span style="color: #3366ff;">curl -X POST "http://127.0.0.1:7851/api/lowvramsetting?new_low_vram_value=True"</span></p>
 <p style="padding-left: 30px;">Replace True with False to disable Low VRAM mode.</p>
 <p style="padding-left: 30px;">JSON return <span style="color: #339966;">{"status": "lowvram-success"}</span></p>
+<h4>Ready Endpoint</strong></h4>
+<p>Check if the Text-to-Speech (TTS) service is ready to accept requests.</p>
+<ul>
+  <li>URL: <span style="color: #3366ff;">http://127.0.0.1:7851/api/ready</span></li>
+  <li>Method: <span style="color: #3366ff;">GET</span></li>
+  <li>Response: <span style="color: #339966;">Ready</span></li>
+</ul>
+<p style="padding-left: 30px;"><span style="color: #3366ff;">curl -X GET "http://127.0.0.1:7851/api/ready"</span></p>
+<h4>Voices List Endpoint</strong></h4>
+<p>Retrieve a list of available voices for generating speech.</p>
+<ul>
+  <li>URL: <span style="color: #3366ff;">http://127.0.0.1:7851/api/voices</span></li>
+  <li>Method: <span style="color: #3366ff;">GET</span></li>
+</ul>
+<p style="padding-left: 30px;"><span style="color: #3366ff;">curl -X GET "http://127.0.0.1:7851/api/voices"</span></p>
+<p style="padding-left: 30px;">JSON return: <span style="color: #339966;">{"voices": ["voice1.wav", "voice2.wav", "voice3.wav"]}</span></p>
+<h4><strong>Preview Voice Endpoint</strong></h4>
+<p>Generate a preview of a specified voice with hardcoded settings.</p>
+<ul>
+  <li>URL: <span style="color: #3366ff;">http://127.0.0.1:7851/api/previewvoice/</span></li>
+  <li>Method: <span style="color: #3366ff;">POST</span></li>
+  <li>Content-Type: <span style="color: #3366ff;">application/x-www-form-urlencoded</span></li>
+</ul>
+<p style="padding-left: 30px;"><span style="color: #3366ff;">curl -X POST "http://127.0.0.1:7851/api/previewvoice/" -F "voice=female_01.wav"</span></p>
+<p style="padding-left: 30px;">Replace <span style="color: #3366ff;">female_01.wav</span> with the name of the voice sample you want to hear.</p>
+<p style="padding-left: 30px;">JSON return: <span style="color: #339966;">{"status": "generate-success", "output_file_path": "/path/to/outputs/api_preview_voice.wav", "output_file_url": "http://127.0.0.1:7851/audio/api_preview_voice.wav"}</span></p>
+
 <p><a href="#toc">Back to top of page<br /></a></p>
 
 <h2 id="debugging-and-tts-generation-information"><strong>Debugging and TTS Generation Information</strong></h2>
@@ -1717,10 +1815,20 @@ template = Template(simple_webpage)
 # Render the template with the dynamic values
 rendered_html = template.render(params=params)
 
+###############################
+#### Internal script ready ####
+###############################
 @app.get("/ready")
 async def ready():
     return Response("Ready endpoint")
 
+############################
+#### External API ready ####
+############################
+@app.get("/api/ready")
+async def ready():
+    return Response("Ready")
+
 @app.get("/")
 async def read_root():
     return HTMLResponse(content=rendered_html, status_code=200)