From 3647d69007a4ae18c118d5ea53b1c970706ca6bf Mon Sep 17 00:00:00 2001 From: erew123 <35898566+erew123@users.noreply.github.com> Date: Fri, 5 Jan 2024 13:18:02 +0000 Subject: [PATCH 1/4] Update README.md --- README.md | 31 +++++++++++++++++++++---------- 1 file changed, 21 insertions(+), 10 deletions(-) diff --git a/README.md b/README.md index fa07a40..08c8d35 100644 --- a/README.md +++ b/README.md @@ -505,27 +505,38 @@ def get_shm_size(): ## 🟦 Running AllTalk as a standalone app AllTalk will run as a standalone app, as long as you install its requirements files into whatever Python environment you are using. You can follow the steps to install the AllTalk's requirements into whatever Python environment you wish. Because I dont know what Python environment you are wanting to use, I can only give you a loose set of installation instructions. -Please note, at time of writing, the TTS engine requires Python **3.9.x** to **3.11.x** [TTS Engine details here](https://pypi.org/project/TTS/). AllTalk and its requirements are tested on Python 3.11.x. +Please note, at time of writing, the TTS engine requires Python **3.9.x** to **3.11.x** [TTS Engine details here](https://pypi.org/project/TTS/). AllTalk and its requirements are tested on Python **3.11.x**. #### 🟦 (Option 1) I already have AllTalk installed as an extension of Text-generation-webui If you already have AllTalk as a extension of Text-generation-webui, and wish to run it as standalone, load Text-generation-webui's Python environment `cmd_windows.bat`, `./cmd_linux.sh`, `cmd_macos.sh` or `cmd_wsl.bat`, move into the AllTalk folder `cd extensions` > `cd alltalk_tts` and start AllTalk with `python script.py`. There is nothing beyond this you would need to do. #### 🟦 (Option 2) I wish to do a custom install of AllTalk -1) In your chosen disk location git clone this repository:

-`git clone https://github.com/erew123/alltalk_tts` +#### 🔹 A very quick understanding of Python Environments. +Different Python applications have different requirements, some of those requirement’s conflict with other Python applications requirements. To work around this problem, you can create different Python environments that remain separated from one another. A simple way of looking at Python environments, is just like how your house has different rooms for specific purposes (Kitchen, Bathroom, Bedroom etc). You can create a Python environment that is built/customised specifically for your current applications needs/purposes and will **not** interfere with any other Python applications environments/installations.

If you are adept at managing Python environments, have an existing Python environment and know that you won’t cause any conflicts by installing AllTalk's requirements within that Python environment, then load up your Python environment and install the requirements. For everyone else, here is a basic guide on installing AllTalk in its own custom Python environment (there will be small variations between OS's, but the principle is the same). -2) Move into the **alltalk_tts** folder:

-`cd alltalk_tts` +**Note:** A standard VENV can cause module path issues, hence Conda is the correct method to create a Python environment for AllTalk. -3) If you have a custom Python environment, now is the time to load it! +#### 🔹 Building a custom Python Environment with Miniconda +1) Open a terminal/command prompt, and confirm that both `python --version` and `pip` both work. Neither of them should give an error message. If they do you may need to install Python (and maybe Pip).

[Python Website](https://www.python.org/downloads/)
[Pip Website](https://pip.pypa.io/en/stable/installation/)

Once you have those working you can now continue on.

-4) Install one of the two requirements files. Whichever one of the two is correct for your machine type:

+2) Assuming you don't already have Miniconda installed, we will need to download and install Minoconda as this is what we will use to build our custom Python environment.

[Miniconda Website](https://docs.conda.io/projects/miniconda/en/latest/)

Download the Miniconda version that is correct for your OS and then install it. Miniconda will create some items in your Start Menu or Application Launcher called something like `Anaconda Prompt`

You will start the `Anaconda Prompt` and your prompt in your terminal/command prompt will say something like:

`(base) C:\users\myaccount\>`

The important bit is that it has the `(base)`, the location after that doesn't matter. `(base)` signifies we are in the base conda Python environment and can now create custom Python environments with Conda.

+ +3) In your Anaconda Prompt command prompt/terminal, move into the folder where you want to download AllTalk to and then git clone this repository. For simplicity I am going to assume that location is `c:\myfiles\`. So you will:

`cd myfiles`

`git clone https://github.com/erew123/alltalk_tts`

+ +4) Now we will create our custom Conda Python 3.11.5 environment and give it the name alltalkenv by typing the following at the prompt:

`conda create --name alltalkenv python=3.11.5`

You will be prompted if you want to continue and you say yes.

Once that process has completed, we now need to change from the `(base)` Conda Python environment, to our newly created `(alltalkenv)` environment. So type the following:

`conda activate alltalkenv`

Your prompt should now change from `(base) C:\myfiles\>` to `(alltalkenv) C:\myfiles\>`

+ +5) **If you are NOT using an Nvidia card and CUDA, skip to step 6**. To force PyTorch to install with CUDA, perform the following:

`pip cache purge` (Clear the current Pip cache)

`pip uninstall torch torchaudio` (Uninstall Torch and Torchaudio from the `alltalkenv` environment)

+You can now either install the CUDA 11.8 or 12.1 version of Torch and Torchaudio with **one** of the following

**CUDA 11.8** > `pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu118`

**CUDA 12.1** > `pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu121`

+ +6) Move into the alltalk_tts folder and install one of the two requirements files. Whichever one of the two is correct for your machine type:

`cd alltalk_tts`

**Nvidia graphics card machines** - `pip install -r requirements_nvidia.txt`

-**Other machines (mac, amd etc)** - `pip install -r requirements_other.txt` +**Other machines (mac, amd etc)** - `pip install -r requirements_other.txt`

-5) Start AllTalk with `python script.py` +7) Start AllTalk with `python script.py`

-Deepspeed and other such things can be installed. Please read the relevant instructions for those items, however, make the relevant changes to load your correct Python environment when installing any requirements files and starting AllTalk. +🔹Anytime you wish to use AllTalk, update its requirements, install some other requirements such as DeepSpeed, you will need to start its Conda Python Environment in the terminal/prompt, which you will do with `conda activate alltalkenv` as long as the Conda executable is accessible at the command prompt/terminal you are in.

It is fully possible to create a batch file or script file that you can use as a launcher from your Start Menu/Application Launcher that will start the environment and AllTalk for you. + +Deepspeed and other such things can be installed. Please read the relevant instructions for those items, however, make the relevant changes to load your correct Python environment when installing any requirements files and starting AllTalk.

## 🟠 API Suite and JSON-CURL ### 🟠Overview From 2c14847708abf7c2227b49b60144a238568c0f2c Mon Sep 17 00:00:00 2001 From: erew123 <35898566+erew123@users.noreply.github.com> Date: Fri, 5 Jan 2024 13:24:20 +0000 Subject: [PATCH 2/4] Update script.py added Hungarian characters --- script.py | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/script.py b/script.py index 54f5fe2..d874aef 100644 --- a/script.py +++ b/script.py @@ -600,7 +600,7 @@ def output_modifier(string, state): # Replace multiple exclamation marks, question marks, or other punctuation with a single instance cleaned_part = re.sub(r'([!?.])\1+', r'\1', part) # Further clean to remove any other unwanted characters - cleaned_part = re.sub(r'[^a-zA-Z0-9\s\.,;:!?\-\'"\u0400-\u04FFÀ-ÿ\$]', '', cleaned_part) + cleaned_part = re.sub(r'[^a-zA-Z0-9\s\.,;:!?\-\'"\u0400-\u04FFÀ-ÿ\u0150\u0151\u0170\u0171\$]', '', cleaned_part) # Remove all newline characters (single or multiple) cleaned_part = re.sub(r'\n+', ' ', cleaned_part) @@ -622,7 +622,7 @@ def output_modifier(string, state): # Replace multiple instances of certain punctuation marks with a single instance cleaned_part = re.sub(r'([!?.])\1+', r'\1', cleaned_part) # Further clean to remove any other unwanted characters - cleaned_part = re.sub(r'[^a-zA-Z0-9\s\.,;:!?\-\'"\u0400-\u04FFÀ-ÿ\$]', '', cleaned_part) + cleaned_part = re.sub(r'[^a-zA-Z0-9\s\.,;:!?\-\'"\u0400-\u04FFÀ-ÿ\u0150\u0151\u0170\u0171\$]', '', cleaned_part) # Remove all newline characters (single or multiple) cleaned_part = re.sub(r'\n+', ' ', cleaned_part) # Process the part and give it a non-character name if being used vai API or standalone. From 002c5ad9db53040510527c64f447a72a3e3a8a70 Mon Sep 17 00:00:00 2001 From: erew123 <35898566+erew123@users.noreply.github.com> Date: Fri, 5 Jan 2024 13:26:15 +0000 Subject: [PATCH 3/4] Update tts_server.py Added Hungarian characters --- tts_server.py | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/tts_server.py b/tts_server.py index a34e8a8..ab15968 100644 --- a/tts_server.py +++ b/tts_server.py @@ -1006,7 +1006,7 @@ async def tts_generate( # Replace multiple exclamation marks, question marks, or other punctuation with a single instance cleaned_part = re.sub(r'([!?.])\1+', r'\1', part) # Further clean to remove any other unwanted characters - cleaned_part = re.sub(r'[^a-zA-Z0-9\s\.,;:!?\-\'"\u0400-\u04FFÀ-ÿ\$]', '', cleaned_part) + cleaned_part = re.sub(r'[^a-zA-Z0-9\s\.,;:!?\-\'"\u0400-\u04FFÀ-ÿ\u0150\u0151\u0170\u0171\$]', '', cleaned_part) # Remove all newline characters (single or multiple) cleaned_part = re.sub(r'\n+', ' ', cleaned_part) output_file = this_dir / "outputs" / f"{output_file_name}_{uuid.uuid4()}_{int(time.time())}.wav" @@ -1030,13 +1030,13 @@ async def tts_generate( cleaned_string = html.unescape(standard_filtering(text_input)) cleaned_string = re.sub(r'([!?.])\1+', r'\1', text_input) # Further clean to remove any other unwanted characters - cleaned_string = re.sub(r'[^a-zA-Z0-9\s\.,;:!?\-\'"\u0400-\u04FFÀ-ÿ\$]', '', cleaned_string) + cleaned_string = re.sub(r'[^a-zA-Z0-9\s\.,;:!?\-\'"\u0400-\u04FFÀ-ÿ\u0150\u0151\u0170\u0171\$]', '', cleaned_string) # Remove all newline characters (single or multiple) cleaned_string = re.sub(r'\n+', ' ', cleaned_string) elif text_filtering == "standard": cleaned_string = re.sub(r'([!?.])\1+', r'\1', text_input) # Further clean to remove any other unwanted characters - cleaned_string = re.sub(r'[^a-zA-Z0-9\s\.,;:!?\-\'"\u0400-\u04FFÀ-ÿ\$]', '', cleaned_string) + cleaned_string = re.sub(r'[^a-zA-Z0-9\s\.,;:!?\-\'"\u0400-\u04FFÀ-ÿ\u0150\u0151\u0170\u0171\$]', '', cleaned_string) # Remove all newline characters (single or multiple) cleaned_string = re.sub(r'\n+', ' ', cleaned_string) else: From 7718bfd3ea0e6495674c8eab71953048eb2af358 Mon Sep 17 00:00:00 2001 From: erew123 <35898566+erew123@users.noreply.github.com> Date: Fri, 5 Jan 2024 13:27:49 +0000 Subject: [PATCH 4/4] Update admin.html Added Hungarian characters --- templates/admin.html | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/templates/admin.html b/templates/admin.html index f815e56..f1c7ca0 100644 --- a/templates/admin.html +++ b/templates/admin.html @@ -416,7 +416,7 @@ let cleanedText = textInput.replace(/ \- | \– /g, ' '); // Replace hyphens cleanedText = cleanedText.replace(/%/g, " percent"); //cleanedText = cleanedText.replace(/([!?.])\1+/g, '$1'); - cleanedText = cleanedText.replace(/[^a-zA-Z0-9\s\.,;:!?\-\'"À-ÿ\u0400-\u04FF]/g, ''); + cleanedText = cleanedText.replace(/[^a-zA-Z0-9\s\.,;:!?\-\'"À-ÿ\u0400-\u04FF\u0150\u0151\u0170\u0171]/g, ''); cleanedText = cleanedText.replace(/\n+/g, ' '); cleanedText = cleanedText.replace(/\.\s'/g, ".'"); // Remove space between period and single quote //cleanedText = cleanedText.replace(/'/g, ""); // Remove all single quotes