Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
97f0d6441a | ||
|
|
2ea6cb6304 | ||
|
|
5bed68a963 | ||
|
|
be9862d0b0 | ||
|
|
f5a811f0f8 | ||
|
|
dc9d343115 | ||
|
|
c92aa435e5 | ||
|
|
1f425e6bac | ||
|
|
0854544a47 | ||
|
|
52a7922e98 | ||
|
|
558583b16f |
No files matched your search
@@ -25,6 +25,10 @@ AllTalk is an updated version of the Coqui_tts extension for Text Generation web
|
|||||||
- 🔴 [Future to-do list & Upcoming updates](https://github.com/erew123/alltalk_tts?#-future-to-do-list)
|
- 🔴 [Future to-do list & Upcoming updates](https://github.com/erew123/alltalk_tts?#-future-to-do-list)
|
||||||
|
|
||||||
#### Updates
|
#### Updates
|
||||||
|
Dec 25th - Applied a small update to avoid a possible race condition on file naming with small sentences when generating narrator/character speech. Please update if you are experiencing missing small bits of missing audio on narrator/character.
|
||||||
|
|
||||||
|
#### 🔄 **Minor updates/bug fixes list** can be found [here](https://github.com/erew123/alltalk_tts/issues/25)
|
||||||
|
|
||||||
The latest build (13 Dec 2023) has had the entire text filtering engine and narration engine rebuilt from scratch. It's highly complicated how its actually working, but the end result it a much clearer TTS output and much better control over the narrator option and how to handle text that isnt within quotes or asterisks. It does however mean you need to ensure your character card is set up correctly if using the narrator function. Details are below in the installation notes.
|
The latest build (13 Dec 2023) has had the entire text filtering engine and narration engine rebuilt from scratch. It's highly complicated how its actually working, but the end result it a much clearer TTS output and much better control over the narrator option and how to handle text that isnt within quotes or asterisks. It does however mean you need to ensure your character card is set up correctly if using the narrator function. Details are below in the installation notes.
|
||||||
|
|
||||||
DeepSpeed **v11.2** can be installed within the **default text-generation-webui Python 3.11 environment**. Installs in custom Python environments are possible, but can be more complicated. Instructions [here](https://github.com/erew123/alltalk_tts##deepspeed-installation-options) (or scroll down).
|
DeepSpeed **v11.2** can be installed within the **default text-generation-webui Python 3.11 environment**. Installs in custom Python environments are possible, but can be more complicated. Instructions [here](https://github.com/erew123/alltalk_tts##deepspeed-installation-options) (or scroll down).
|
||||||
@@ -145,6 +149,7 @@ Assuming its all working fine and you are happy, you can delete the old alltalk_
|
|||||||
| |  |
|
| |  |
|
||||||
|
|
||||||
## 🟨 Help with problems
|
## 🟨 Help with problems
|
||||||
|
#### 🔄 **Minor updates/bug fixes list** can be found [here](https://github.com/erew123/alltalk_tts/issues/25)
|
||||||
|
|
||||||
#### 🟨 How to make a diagnostics report file
|
#### 🟨 How to make a diagnostics report file
|
||||||
<details>
|
<details>
|
||||||
@@ -203,6 +208,19 @@ Other possibilities for this issue are:
|
|||||||
11) You have built yourself a custom Python environment and something is funky with it. This is very hard to diagnose as its not a standard environment. You may want to updating text-generation-webui and re installing its requirements file (whichever one you use that comes down with text-generation-webui).
|
11) You have built yourself a custom Python environment and something is funky with it. This is very hard to diagnose as its not a standard environment. You may want to updating text-generation-webui and re installing its requirements file (whichever one you use that comes down with text-generation-webui).
|
||||||
</details>
|
</details>
|
||||||
|
|
||||||
|
#### 🟨 I think AllTalks requirements file has installed something another extension doesn't like
|
||||||
|
<details>
|
||||||
|
<summary>Click to expand</summary><br>
|
||||||
|
|
||||||
|
Ive paid very close attention to **not** impact what Text-generation-webui is requesting on a factory install. This is one of the requirements of submitting an extension to Text-generation-webui. If you want to look at a comparison of a factory fresh text-generation-webui installed packages (with cuda 12.1, though AllTalk's requirements were set on cuda 11.8) you can find that comparison [here](https://github.com/erew123/alltalk_tts/issues/23). This comparison shows that AllTalk is requesting the same package version numbers as Text-generation-webui or even lower version numbers (meaning AllTalk will not update them to a later version). What other extensions do, I cant really account for that.
|
||||||
|
|
||||||
|
I will note that the TTS engine downgrades Pandas data validator to 1.5.3 though its unlikely to cause any issues. You can upgrade it back to text-generation-webui default (december 2023) with `pip install pandas==2.1.4` when inside of the python environment. I have noticed no ill effects from it being a lower or higher version, as far as AllTalk goes. This is also the same behaviour as the Coqui_tts extension that comes with Text-generation-webui.
|
||||||
|
|
||||||
|
Other people are reporting issues with extensions not starting with errors about Pydantic e.g. ```pydantic.errors.PydanticImportError: BaseSettings` has been moved to the pydantic-settings package. See https://docs.pydantic.dev/2.5/migration/#basesettings-has-moved-to-pydantic-settings for more details.```
|
||||||
|
|
||||||
|
Im not sure if the Pydantic version has been recently updated by the Text-generation-webui installer, but this is nothing to do with AllTalk. The other extension you are having an issue with, need to be updated to work with Pydantic 2.5.x. AllTalk was updated in mid december to work with 2.5.x. I am not specifically condoning doing this, as it may have other knock on effects, but within the text-gen Python environment, you can use `pip install pydantic==2.5.0` or `pip install pydantic==1.10.13` to change the version of Pydantic installed.
|
||||||
|
</details>
|
||||||
|
|
||||||
#### 🟨 I activated DeepSpeed in the settings page, but I didnt install DeepSpeed yet and now I have issues starting up
|
#### 🟨 I activated DeepSpeed in the settings page, but I didnt install DeepSpeed yet and now I have issues starting up
|
||||||
<details>
|
<details>
|
||||||
<summary>Click to expand</summary><br>
|
<summary>Click to expand</summary><br>
|
||||||
@@ -293,6 +311,8 @@ As mentioned you must have a small portion of the Nvidia CUDA Toolkit **11.8** i
|
|||||||
12) When you have finished finetuning, the final tab will tell you what to do with your files and how to move your newly trained model to the correct location on disk.
|
12) When you have finished finetuning, the final tab will tell you what to do with your files and how to move your newly trained model to the correct location on disk.
|
||||||
|
|
||||||
## 🔵🟢🟡 DeepSpeed Installation Options
|
## 🔵🟢🟡 DeepSpeed Installation Options
|
||||||
|
**NOTE**: You **DO NOT** need to set Text-generation-webUI's **--deepspeed** setting for AllTalk to be able to use DeepSpeed. These are two completely separate things and incorrectly setting that on Text-generation-webUI may cause other complications.
|
||||||
|
|
||||||
#### 🔵 Linux Installation
|
#### 🔵 Linux Installation
|
||||||
<details>
|
<details>
|
||||||
<summary>Click to expand: Linux DeepSpeed installation</summary>
|
<summary>Click to expand: Linux DeepSpeed installation</summary>
|
||||||
|
|||||||
@@ -10,4 +10,4 @@ requests>=2.31.0
|
|||||||
tqdm>=4.66.1
|
tqdm>=4.66.1
|
||||||
importlib-metadata>=4.8.1
|
importlib-metadata>=4.8.1
|
||||||
packaging>=23.2
|
packaging>=23.2
|
||||||
pydantic>=2.5.0
|
pydantic>=1.10.13
|
||||||
@@ -10,4 +10,4 @@ requests>=2.31.0
|
|||||||
tqdm>=4.66.1
|
tqdm>=4.66.1
|
||||||
importlib-metadata>=4.8.1
|
importlib-metadata>=4.8.1
|
||||||
packaging>=23.2
|
packaging>=23.2
|
||||||
pydantic>=2.5.0
|
pydantic>=1.10.13
|
||||||
@@ -501,7 +501,7 @@ def output_modifier(string, state):
|
|||||||
if process_lock.acquire(blocking=False):
|
if process_lock.acquire(blocking=False):
|
||||||
try:
|
try:
|
||||||
if params["narrator_enabled"]:
|
if params["narrator_enabled"]:
|
||||||
# print(original_string)
|
#print(original_string)
|
||||||
# Do Some basic stripping and remove any double CR's
|
# Do Some basic stripping and remove any double CR's
|
||||||
processed_string = (
|
processed_string = (
|
||||||
original_string
|
original_string
|
||||||
@@ -522,11 +522,11 @@ def output_modifier(string, state):
|
|||||||
.replace('"', '"<')
|
.replace('"', '"<')
|
||||||
)
|
)
|
||||||
#capturing another outlier in inital character paragraph
|
#capturing another outlier in inital character paragraph
|
||||||
# print("processed string 1 is:", processed_string)
|
#print("processed string 1 is:", processed_string)
|
||||||
processed_string = processed_string.replace('"<. *', '"< *"')
|
processed_string = processed_string.replace('"<. *', '"< *"')
|
||||||
processed_string = processed_string.replace('< *"', '< *')
|
processed_string = processed_string.replace('< *"', '< *')
|
||||||
processed_string = processed_string.replace('. *', '< *')
|
processed_string = processed_string.replace('. *', '< *')
|
||||||
# print("processed string 2 is:", processed_string)
|
#print("processed string 2 is:", processed_string)
|
||||||
# Set up a tracking of the individual wav files.
|
# Set up a tracking of the individual wav files.
|
||||||
audio_files_all_paragraphs = []
|
audio_files_all_paragraphs = []
|
||||||
# Split the line using " and ".* " (so end of sentences, leaving special characters added to the start of all OTHER sentences, bar possibly the first one if its starting with a *
|
# Split the line using " and ".* " (so end of sentences, leaving special characters added to the start of all OTHER sentences, bar possibly the first one if its starting with a *
|
||||||
@@ -541,11 +541,11 @@ def output_modifier(string, state):
|
|||||||
if '<' in part and '< *' not in part and '<*' not in part and '< *' not in part and '< ' not in part and '< ' not in part:
|
if '<' in part and '< *' not in part and '<*' not in part and '< *' not in part and '< ' not in part and '< ' not in part:
|
||||||
cleaned_part = html.unescape(part.replace('<', ''))
|
cleaned_part = html.unescape(part.replace('<', ''))
|
||||||
voice_to_use = params["voice"]
|
voice_to_use = params["voice"]
|
||||||
#Narrator will always be an * or < with an * a position or two after it.
|
#Narrator will always be an * or < with an * a position or two after it.
|
||||||
elif '<*' in part or '< *' in part or '< *' in part or '*' in part:
|
elif '<*' in part or '< *' in part or '< *' in part or '*' in part:
|
||||||
cleaned_part = html.unescape(part.replace('<*', '').replace('< *', '').replace('< *', '').replace('*', ''))
|
cleaned_part = html.unescape(part.replace('<*', '').replace('< *', '').replace('< *', '').replace('*', ''))
|
||||||
voice_to_use = params["narrator_voice"]
|
voice_to_use = params["narrator_voice"]
|
||||||
#If the other two dont capture it, aka, the AI gave no * or " on the line, use non_quoted_text_is aka user interface, user can choose Char or Narrator
|
#If the other two dont capture it, aka, the AI gave no * or " on the line, use non_quoted_text_is aka user interface, user can choose Char or Narrator
|
||||||
elif non_quoted_text_is:
|
elif non_quoted_text_is:
|
||||||
cleaned_part = html.unescape(part.replace('< ', '').replace('< ', '').replace('< ', ''))
|
cleaned_part = html.unescape(part.replace('< ', '').replace('< ', '').replace('< ', ''))
|
||||||
voice_to_use = params["voice"]
|
voice_to_use = params["voice"]
|
||||||
@@ -576,9 +576,14 @@ def output_modifier(string, state):
|
|||||||
)
|
)
|
||||||
cleaned_part = html.unescape(processed_string)
|
cleaned_part = html.unescape(processed_string)
|
||||||
# Process the part and give it a non-character name if being used vai API or standalone.
|
# Process the part and give it a non-character name if being used vai API or standalone.
|
||||||
output_filename = get_output_filename(state)
|
if "character_menu" in state:
|
||||||
|
output_file = Path(f'{params["output_folder_wav"]}/{state["character_menu"]}_{int(time.time())}.wav')
|
||||||
|
else:
|
||||||
|
output_file = Path(f'{params["output_folder_wav"]}/TTSOUT_{int(time.time())}.wav')
|
||||||
|
output_file_str = output_file.as_posix()
|
||||||
|
output_file = get_output_filename(state)
|
||||||
generate_response = send_generate_request(
|
generate_response = send_generate_request(
|
||||||
cleaned_part, params["voice"], language_code, output_filename
|
cleaned_part, params["voice"], language_code, output_file_str
|
||||||
)
|
)
|
||||||
audio_path = generate_response.get("data", {}).get("audio_path")
|
audio_path = generate_response.get("data", {}).get("audio_path")
|
||||||
final_output_file = audio_path
|
final_output_file = audio_path
|
||||||
@@ -618,11 +623,10 @@ def output_modifier(string, state):
|
|||||||
|
|
||||||
|
|
||||||
def get_output_filename(state):
|
def get_output_filename(state):
|
||||||
# Check if character name exists and if not, just call it TTSOUT_
|
|
||||||
if "character_menu" in state:
|
if "character_menu" in state:
|
||||||
return Path(f'{params["output_folder_wav"]}/{state["character_menu"]}_{int(time.time())}.wav').as_posix()
|
return Path(f'{params["output_folder_wav"]}/{state["character_menu"]}_{str(uuid.uuid4())[:8]}.wav').as_posix()
|
||||||
else:
|
else:
|
||||||
return Path(f'{params["output_folder_wav"]}/TTSOUT_{int(time.time())}.wav').as_posix()
|
return Path(f'{params["output_folder_wav"]}/TTSOUT_{str(uuid.uuid4())[:8]}.wav').as_posix()
|
||||||
|
|
||||||
|
|
||||||
###############################################
|
###############################################
|
||||||
|
|||||||
+7
-3
@@ -22,7 +22,6 @@ from fastapi.templating import Jinja2Templates
|
|||||||
from fastapi.staticfiles import StaticFiles
|
from fastapi.staticfiles import StaticFiles
|
||||||
from jinja2 import Template
|
from jinja2 import Template
|
||||||
from contextlib import asynccontextmanager
|
from contextlib import asynccontextmanager
|
||||||
from pydantic import field_validator
|
|
||||||
|
|
||||||
###########################
|
###########################
|
||||||
#### STARTUP VARIABLES ####
|
#### STARTUP VARIABLES ####
|
||||||
@@ -647,11 +646,10 @@ class JSONInput(BaseModel):
|
|||||||
autoplay: bool = Field(..., description="autoplay needs to be a true or false value.")
|
autoplay: bool = Field(..., description="autoplay needs to be a true or false value.")
|
||||||
autoplay_volume: float = Field(..., ge=0.1, le=1.0, description="autoplay_volume needs to be from 0.1 to 1.0")
|
autoplay_volume: float = Field(..., ge=0.1, le=1.0, description="autoplay_volume needs to be from 0.1 to 1.0")
|
||||||
|
|
||||||
@field_validator("autoplay_volume")
|
|
||||||
@classmethod
|
@classmethod
|
||||||
def validate_autoplay_volume(cls, value):
|
def validate_autoplay_volume(cls, value):
|
||||||
if not (0.1 <= value <= 1.0):
|
if not (0.1 <= value <= 1.0):
|
||||||
raise ValidationError("Autoplay volume must be between 0.1 and 1.0")
|
raise ValueError("Autoplay volume must be between 0.1 and 1.0")
|
||||||
return value
|
return value
|
||||||
|
|
||||||
|
|
||||||
@@ -1054,6 +1052,12 @@ simple_webpage = """
|
|||||||
<p style="padding-left: 30px; text-align: justify;">AllTalk is a web interface, based around the Coqui TTS voice cloning/speech generation system. To generate TTS, you can use the provided gradio interface or interact with the server using JSON/CURL commands. </p>
|
<p style="padding-left: 30px; text-align: justify;">AllTalk is a web interface, based around the Coqui TTS voice cloning/speech generation system. To generate TTS, you can use the provided gradio interface or interact with the server using JSON/CURL commands. </p>
|
||||||
<p style="padding-left: 30px; text-align: justify;"><span style="color: #ff0000;">Note:</span> When loading up a new character in Text generation webUI it may look like nothing is happening for 20-30 seconds. Its actually processing the introduction section (greeting message) of the text and once that is completed, it will appear. You can see the activity occuring in the console window. Refreshing the page multiple times will try force the TTS engine to keep re-generating the text, so please just wait and check the console if needed.</p>
|
<p style="padding-left: 30px; text-align: justify;"><span style="color: #ff0000;">Note:</span> When loading up a new character in Text generation webUI it may look like nothing is happening for 20-30 seconds. Its actually processing the introduction section (greeting message) of the text and once that is completed, it will appear. You can see the activity occuring in the console window. Refreshing the page multiple times will try force the TTS engine to keep re-generating the text, so please just wait and check the console if needed.</p>
|
||||||
<p style="padding-left: 30px; text-align: justify;"><span style="color: #ff0000;">Note: </span>Ensure that your RP character card has asterisks around anything for the narration and double quotes around anything spoken. There is a complication ONLY with the greeting card so, ensuring it has the correct use of quotes and asterisks will help make sure that the greeting card sounds correct. I will aim to address this issue in a future update. In Text-generation-webUI <span style="color: #3366ff;">parameters menu</span> > <span style="color: #3366ff;">character tab</span> > <span style="color: #3366ff;">greeting</span> make sure that anything in there that is the narrator is in asterisks and anything spoken is in double quotes, then hit the <span style="color: #3366ff;">save</span> (💾) button.</p>
|
<p style="padding-left: 30px; text-align: justify;"><span style="color: #ff0000;">Note: </span>Ensure that your RP character card has asterisks around anything for the narration and double quotes around anything spoken. There is a complication ONLY with the greeting card so, ensuring it has the correct use of quotes and asterisks will help make sure that the greeting card sounds correct. I will aim to address this issue in a future update. In Text-generation-webUI <span style="color: #3366ff;">parameters menu</span> > <span style="color: #3366ff;">character tab</span> > <span style="color: #3366ff;">greeting</span> make sure that anything in there that is the narrator is in asterisks and anything spoken is in double quotes, then hit the <span style="color: #3366ff;">save</span> (💾) button.</p>
|
||||||
|
<p style="padding-left: 30px;"><span style="text-decoration: underline;"><strong>AllTalk <span style="color: #ff0000; text-decoration: underline;">Minor Bug Fixes Changelog & Known issues</span></strong></span></p>
|
||||||
|
<p style="padding-left: 30px;">If I squash any minor bugs or find any issues, I will try to apply an update asap. If you think something isnt working correctly or you have a problem, check these two links below first.</p>
|
||||||
|
<ul>
|
||||||
|
<li>Minor bug fixes changelog <a href="https://github.com/erew123/alltalk_tts/issues/25" target="_blank" rel="noopener">link here</a>.</li>
|
||||||
|
<li>Help and known issues <a href="https://github.com/erew123/alltalk_tts?#-help-with-problems" target="_blank" rel="noopener">link here</a>.</li>
|
||||||
|
</ul>
|
||||||
<p style="padding-left: 30px;"><span style="text-decoration: underline;"><strong>AllTalk</strong></span> <br />Github <a href="https://github.com/erew123/alltalk_tts" target="_blank" rel="noopener">link here</a><br />Update instructions <a href="https://github.com/erew123/alltalk_tts?#-updating" target="_blank" rel="noopener">link here</a><br />Help and issues <a href="https://github.com/erew123/alltalk_tts?#-help-with-problems" target="_blank" rel="noopener">link here<br /><br /></a><span style="text-decoration: underline;"><strong>Text generation webUI</strong></span> <br />Web interface <a href="http://{{ params["ip_address"] }}:7860" target="_blank" rel="noopener">link here</a> <br />Documentation <a href="https://github.com/oobabooga/text-generation-webui/wiki" target="_blank" rel="noopener">link here</a></p>
|
<p style="padding-left: 30px;"><span style="text-decoration: underline;"><strong>AllTalk</strong></span> <br />Github <a href="https://github.com/erew123/alltalk_tts" target="_blank" rel="noopener">link here</a><br />Update instructions <a href="https://github.com/erew123/alltalk_tts?#-updating" target="_blank" rel="noopener">link here</a><br />Help and issues <a href="https://github.com/erew123/alltalk_tts?#-help-with-problems" target="_blank" rel="noopener">link here<br /><br /></a><span style="text-decoration: underline;"><strong>Text generation webUI</strong></span> <br />Web interface <a href="http://{{ params["ip_address"] }}:7860" target="_blank" rel="noopener">link here</a> <br />Documentation <a href="https://github.com/oobabooga/text-generation-webui/wiki" target="_blank" rel="noopener">link here</a></p>
|
||||||
<p style="padding-left: 30px;"><a href="#toc">Back to top of page</a></p>
|
<p style="padding-left: 30px;"><a href="#toc">Back to top of page</a></p>
|
||||||
<h2 id="server-information">Server Information</h2>
|
<h2 id="server-information">Server Information</h2>
|
||||||
|
|||||||
Reference in new issue
Block a user