From 0bec11ebd2799e0d110b07a773aeb6f331a73bd3 Mon Sep 17 00:00:00 2001
From: erew123 <35898566+erew123@users.noreply.github.com>
Date: Tue, 1 Oct 2024 12:03:03 +0100
Subject: [PATCH] Update README.md
---
README.md | 430 ++++++++++++------------------------------------------
1 file changed, 91 insertions(+), 339 deletions(-)
diff --git a/README.md b/README.md
index 591b3f6..eabd3b0 100644
--- a/README.md
+++ b/README.md
@@ -1,373 +1,125 @@
# AllTalk TTS v2 BETA
-This is the BETA of v2. To be clear, that means they may well be bugs, issues, missing/incomplete documentation, a variety of potential problems etc (This is what BETA refers to in the software world for those whom dont know). This means you may need a bit of technical know how to deal with things that could come up and should only continue if you feel comfortable with that.
-
Github **discussions** on the BETA are [here in the discussion board](https://github.com/erew123/alltalk_tts/discussions/245)
-For **Issues/Bugs/Support**, please open an issue ticket [here in the issues area](https://github.com/erew123/alltalk_tts/issues)
+**Issues/Bugs/Support**, please open an issue ticket [here in the issues area](https://github.com/erew123/alltalk_tts/issues)
-**Please note, my available time has become VERY limited due to unexpected family commitments. So please DYOR and refer to the TTS manufacturer (listed in the interface) for issues with their TTS engines.**
+**Documentation**, please refer to the built in documentation or the [Wiki here ](https://github.com/erew123/alltalk_tts/wiki)
-#### If you are going to run the BETA, please read all the info below....
-
-The BETA has been tested on Windows 11 and Lunux (Ubuntu). The Standalone installations `atsetup` **should** take care of everything for you, all the way through to having DeepSpeed up and running on both Windows and Linux.
-
-Seperate from Standalone installations, installation within the Text-generation-webui Python environment should also work, however, I have had to various bits of code last minute and not had an opportuninty to deeply test this yet.
-
-### Screenshots
-Please go see [here](https://github.com/erew123/alltalk_tts/discussions/237)
-
-### New Features include
-
-- Multiple TTS engines. Currently XTTS, Piper, Parler and VITS. New engines are easy to install with some coding experience. In time I will add in all engine Coqui supports as they should be easy.
- - Each engine has its own custom settings
- - You can download the models for each engine within its settings area.
- - FYI there is no limit on how many XTTS models AllTalk will now find/work with.
-- AllTalk starts up on 0.0.0.0, meaning it will bind to ALL available IP addresses. There is no setting its IP address any more.
-- Updated Coqui TTS engine.
-- XTTS multiple audio sample TTS generation to improve voice reproduction https://github.com/erew123/alltalk_tts/commit/b7aa3a7c5c8df260edede52dd1b31a7b40d9b202
-- Gradio web interface (see the screenshots)
-- Retrieval based Voice Conversion, AKA, RVC voice Pipeline, re-written to work on Python 3.11.
-- TTS Generator will now use any TTS engine.
-- Vastly updated API Suite. **V2 is not 100% compatable with v1. (See here for an explanation https://github.com/erew123/alltalk_tts/issues/166)**.
- - New SillyTavern extension is included in the `/system/` folder.
- - New remote TGWUI extension if you want to run your TTS elsewhere from the AllTalk server.
-- OpenAI compatable endpoint/API. Meaning this should work as an alternative endpoint with any software that can send a TTS generation request to OpenAI.
-- Fully customisable API Settings. I have **NOT** tested any limits of any TTS engines.
-- Multi Engine Manager (MEM) (Run Multiple TTS Engine instances simultaneously with a queue system). See [here for details](https://github.com/erew123/alltalk_tts/issues/63#issuecomment-2365329723)
-- Updated Finetuning for XTTS models
-- Audio Transcoding to about 6x formats (mp3, opus, etc)
-- About 50 gradio interface themes if you arent happy with the standard Gradio one.
-- Documentaton has been about 80% updated. Its built into the interface.
-- Lots and lots of other things too numerous to mention.
-
-There is a welcome screen that covers a few other bits, along with the built in documentation.
-
-### To install the BETA build, read the notes below for the type of system you have, the read the Quick Setup section to perform the install.
+**Please note, my available time has become VERY limited due to unexpected family commitments. So please DYOR, look at the [Wiki here ](https://github.com/erew123/alltalk_tts/wiki) and refer to the TTS manufacturer (Links in the Gradio interface) for issues specific to THEIR TTS engines.**
---
-### Windows Systems Requirements
-Before installing AllTalk you will need to have or install the following:
+## AllTalk V2 Core Functionality
+- Comprehensive setup utilities for Windows & Linux
+- Multiple TTS engine support (XTTS, Piper, Parler, VITS)
+- Easy integration of new TTS engines (some coding required)
+- Customizable settings for each TTS engine
+- In-app model downloads for each engine
+- Gradio web interface for easy management
+- Standalone application or integration with Text-generation-webui, SillyTavern, KoboldCPP, HomeAssistant
+- Narrator function for using different voices for characters and narration
+- Audio Transcoding to multiple formats (mp3, opus, etc.)
+- About 50 Gradio interface themes
+- Custom start-up settings
+- Clear console output for troubleshooting
+- Binds to all available IP addresses (0.0.0.0)
+- Fully customizable Global API settings
+- Retrieval-based Voice Conversion (RVC) pipeline
-- **Git** to clone Github repositories. [Instructions here](https://github.com/erew123/alltalk_tts/wiki/Install-%E2%80%90-WINDOWS-%E2%80%90-Git)
-- **Microsoft C++ Build Tools** and **Windows SDK** for Python to function correctly. [Instructions here](https://github.com/erew123/alltalk_tts/wiki/Install-%E2%80%90-WINDOWS-%E2%80%90-Python-C-&-SDK-Requirements)
-- **Espeak-ng** for multiple TTS engines to function. [Instructions here](https://github.com/erew123/alltalk_tts/wiki/Install-%E2%80%90-WINDOWS-%E2%80%90-Espeak%E2%80%90ng)
+#### Documentation and Support
+- Built-in documentation with web interface
+- GitHub Wiki documentation
-### Linux Systems
-You need to install a few bits (depending on your Linux flavour), otherwise DeepSpeed will fail and some TTS engines not work. At your terminal type the following:
-- **Debian-based systems** `sudo apt install libaio-dev espeak-ng ffmpeg gcc g++`
-- **RPM-based systems** `sudo yum install libaio-devel espeak-ng ffmpeg gcc g++`
+#### Performance and Optimization (Depending on TTS engine used)
+- DeepSpeed integration for 2-3x performance boost
+- Low VRAM mode for systems with limited GPU memory
-As mentioned, the **atsetup.sh** install for Standalone installations **should** install deepspeed automatically.
+#### Voice Customization and Enhancement
+- Model Finetuning for improved voice reproduction
+- XTTS Multiple audio sample TTS generation for better voice reproduction
-TGWUI - If however you are wanting to install in TGWUI and you can look here for a simpler way to get DeepSpeed installed here https://github.com/erew123/alltalk_tts/releases/tag/DeepSpeed-14.2-Linux, however, after 20+ hours wrestling with the conda python toolkit installations (on Linux), it looks like Nvidia may have a fault in their installation that can break 6x of the symlinks in the conda environment folder. They are reasonably easy to fix **if needed** and the script in `/system/config/fixsymlinks.sh` **may** do it for you, though I havnt tested on every OS with every eventuality etc.
+#### Bulk Operations and Management
+- Bulk TTS Generator/Editor for large-scale audio production
-### Mac Systems
-I do not have a Mac system personally, so cannot test things out. In theory, all the TTS engines **should** work on Mac, however Ive been unable to write an installation routine for AllTalk. I have a loose idea of what to do. You would need to setup a miniconda for mac with Python 3.11.9 and Pytorch 2.2.1. You would need to install (brew) espeak-ng and ffmpeg (like Linux). I think at that point in time, once you have a working environment, you should be able to use the `requirements_standalone.txt` file with pip, though you may want to remove the nvidia cudnn bits, as mac cant use those. You would have to `conda install pytorch::faiss-cpu` and also `conda install conda-forge::ffmpeg`. The sticking point, if you wanted to use RVC is that you need a version of Fairseq that would work on Mac and Python 3.11.x. The only way I can think of doing that would be to compile a build of this https://github.com/VarunGumma/fairseq (if its possible), though I have no idea how to do that on a mac. So using RVC on Mac may be a no-no, or it may be possible for someone who wants to try figuring out Fairseq 12.2+ on mac.
+#### API and Integration
+- Comprehensive API Suite
+- OpenAI-compatible endpoint/API for broader software compatibility
+- JSON call support for third-party applications
-### ALL Systems - General requirements
-When I say general requirements, this is what I have tested it on. If you vary outside of this setup, expect issues. I will not have time to figure out why it doesnt work on Python version x and Pytorch version x, I dont have the resource/time available.
-- **Python version** 3.11.9
-- **PyTorch version** 2.2.1 with CUDA 12.1
-- **Standalone Installation Disk space** Used **during** installation may go as high as **23GB** of space. This is cleared down at the end of installation. Windows users can expect about 12GB of disk space before you install any models. Linux users can expect about 16GB of space, due to the extra need for the CUDA Toolkit.
-- **TGWUI Installation Disk Space** Will add about 2-3 GB onto the TGWUI Python environment.
+#### Experimental Features
+- Multi Engine Manager (MEM) for running multiple TTS instances simultaneously & queuing requests between them.
+
+#### Screenshots
+Screenshots are available [here](https://github.com/erew123/alltalk_tts/discussions/237)
---
-### 🟩 Quick Setup (Text-generation-webui & Standalone Installation)
+### 🟥 Platform-Specific Notes
-Quick setup scripts are available for users on Windows 10/11 and Linux. Instructional videos for both setup processes are linked below.
+#### Windows
+- Requires Git, Microsoft C++ Build Tools, and Windows SDK.
+- DeepSpeed works with NVIDIA GPUs.
-- Ensure that **Git** is installed on your system as it is required for cloning the repository. If you do not have Git installed, visit [**Git's official website**](https://git-scm.com/downloads) to download and install it.
-- **Important**: Do not use dashes or spaces in your folder path (e.g. avoid `/my folder-is-this/alltalk_tts-main`) as this causes issues with Python.
+#### Linux
+- Requires specific packages based on your distribution (Debian-based or RPM-based).
+- DeepSpeed works with NVIDIA GPUs.
+- Limited experimental support for AMD GPUs (mainly for XTTS).
-#### **NOTE. IF YOU HAVE AN EXISTING ALLTALK INSTALL** this will want to create a folder called `alltalk_tts` so you will need to re-name your old `alltalk_tts` to something else OR put this in a different folder/location.
-#### **NOTE**. I am not saying this is a over-the-top of an old alltalk upgrade ATM. It needs the Python environment rebuilding, hence a fresh install is needed for now.
+#### Mac (Theoretical)
+- Installation process is untested and theoretical.
+- No GPU acceleration for AllTalk TTS engines.
+- Some TTS engines may have limited or no support on Mac.
----
-#### Python/PIP has recently made changes to the package installer and I have/am doing what I can to keep the requirements files working, however, this is a complicated time to say the least as some external requirements change and this can cause installation issues. Please raise a ticket if you have an error that a module is missing.
----
+For more details on Mac support limitations, please refer to our [Mac Support Disclaimer](https://github.com/erew123/alltalk_tts/wiki/Install-%E2%80%90-Manual-Installation-Guide).
-
-QUICK SETUP - Text-Generation-webui
-
+#### GPU Support
+GPU support is provided by the developer of the individual TTS engine. If their engine support X GPU, then I can support X GPU, if it doesnt support X GPU, then I cannot support X GPU.
-For a step-by-step video guide, click [here](https://www.youtube.com/watch?v=icn2XS5rUH8).
-
-To set up AllTalk within Text-generation-webui, follow either method:
-
-1. **Download AllTalk Setup**:
- - **Via Terminal/Console (Recommended)**:
- - `cd \text-generation-webui\extensions\`
- - `git clone -b alltalkbeta https://github.com/erew123/alltalk_tts`
-
-2. **Start Python Environment**:
- - In the text-generation-webui folder, start the environment with the appropriate command:
- - Windows: `cmd_windows.bat`
- - Linux: `./cmd_linux.sh`
-
- > If you're unfamiliar with Python environments and wish to learn more, consider reviewing **Understanding Python Environments Simplified** in the Help section.
-
-3. **Run AllTalk Setup Script**:
- - Navigate to the AllTalk directory and execute the setup script:
- - `cd extensions`
- - `cd alltalk_tts`
- - Windows: `atsetup.bat`
- - Linux: `./atsetup.sh`
-
-4. **Install Requirements**:
- - Follow the on-screen instructions to install the necessary requirements. It's recommended to test AllTalk's functionality before installing DeepSpeed.
-
-> **Note**: Always activate the Text-generation-webui Python environment before making any adjustments or using Fine-tuning. Additional instructions for Fine-tuning and DeepSpeed can be found within the setup utility and on this documentation page.
-
-
-
-
-QUICK SETUP - Standalone Installation
-
-
-For a step-by-step video guide, click [here](https://www.youtube.com/watch?v=AQYCccDRbaY).
-
-To perform a Standalone installation of AllTalk:
-
-1. **Get AllTalk Setup**:
- - **Via Terminal/Console (Recommended)**:
- - Navigate to your preferred directory: `cd C:\myfiles\`
- - Clone the AllTalk repository: `git clone -b alltalkbeta https://github.com/erew123/alltalk_tts`
-
-2. **Start AllTalk Setup**:
- - Open a terminal/command prompt, move to the AllTalk directory, and run the setup script:
- - `cd alltalk_tts`
- - Windows: `atsetup.bat`
- - Linux: `./atsetup.sh`
-
-3. **Follow the Setup Prompts**:
- - Select Standalone Installation and then Option 1 and follow any on-screen instructions to install the required files. DeepSpeed is automatically installed, but will only work on Nvidia GPU's.
-
-> If you're unfamiliar with Python environments and wish to learn more, consider reviewing **Understanding Python Environments Simplified** in the Help section.
-
-
-
-
-MANUAL Windows Setup - Standalone Installation
-
-
-# Manual Windows Setup Instructions for AllTalk TTS
-
-If for some reason the Quick Setup instructions above are not what you wish to do, you can follow these instructions to perform a manual installation.
-
-## Prerequisites
-1. Ensure you have curl installed on your system. If not, download and install it from https://curl.se/
-2. Make sure your installation path does not contain spaces or special characters.
-3. Install Microsoft C++ development tools for Python as detailed in [Windows & Python requirements for compiling packages here](https://github.com/erew123/alltalk_tts?tab=readme-ov-file#installation-and-setup-issues)
-4. Ensure you have installed Espeak-ng as detailed above [here](https://github.com/erew123/alltalk_tts/tree/alltalkbeta?tab=readme-ov-file#windows-systems)
-
-## Manual Installation Steps
-
-#### If at any step you get an error code or message, you can retry that step or research on the internet what the error may be.
-
-Error messages may vary, but could look like `('Connection broken: IncompleteRead(2742643 bytes read, 984095 more expected)', IncompleteRead(2742643 bytes read, 984095 more expected))` which would be a failure to fully download something from Conda (as an example). In such a situation you would re-try to the step or research on the internet, or even try later as it could be a transient fault.
-
-These are **copy/paste** instructions, for Windows and you can click on the **double square button at the right of each step to copy it**. You then **right click with your mouse on the Command prompt** window to paste it in.
-
-**Download AllTalk to your preferred folder**:
- - **Open a Windows Command Prompt from your Start Menu**:
- - Navigate to your preferred directory:
- ```
- cd C:\myfiles\
- ```
- - Clone the AllTalk repository:
- ```
- git clone -b alltalkbeta https://github.com/erew123/alltalk_tts
- ```
-
-1. After cloning AllTalk into a folder, navigate into to the directory where you cloned AllTalk to.
- ```
- cd C:\myfiles\alltalk_tts
- ```
-
-2. Create the alltalk_environment directory and navigate into it:
- ```
- mkdir alltalk_environment
- ```
- ```
- cd alltalk_environment
- ```
-
-3. Download Miniconda:
- ```
- curl -Lk "https://repo.anaconda.com/miniconda/Miniconda3-py311_24.4.0-0-Windows-x86_64.exe" > miniconda_installer.exe
- ```
-
-4. Install Miniconda:
- ```
- start /wait "" miniconda_installer.exe /InstallationType=JustMe /NoShortcuts=1 /AddToPath=0 /RegisterPython=0 /NoRegistry=1 /S /D=%cd%\conda
- ```
-
-5. Navigate to the conda folder:
- ```
- cd conda
- ```
-
-6. Create a new conda environment:
- ```
- .\condabin\conda create --no-shortcuts -y -k --prefix ..\env python=3.11.9
- ```
-
-7. Activate the new environment:
- ```
- call .\condabin\conda.bat activate ..\env
- ```
-
-8. Install PyTorch 2.2.1:
- ```
- .\Scripts\conda install -y pytorch==2.2.1 torchvision==0.17.1 torchaudio==2.2.1 pytorch-cuda=12.1 -c pytorch -c nvidia
- ```
-
-9. Install Faiss:
- ```
- .\Scripts\conda install -y pytorch::faiss-cpu
- ```
-
-10. Install FFmpeg:
- ```
- .\Scripts\conda install -y conda-forge::ffmpeg
- ```
-
-11. Navigate back to the `alltalk_tts` folder where you cloned AllTalk to, then you can install requirements from the requirements file (adjust path as needed):
- ```
- cd..
- ```
- ```
- cd..
- ```
- You should now be back in your `alltalk_tts` folder e.g. `C:\myfiles\alltalk_tts`
- ```
- pip install -r system\requirements\requirements_standalone.txt
- ```
-
-12. Update Gradio:
- ```
- pip install --upgrade gradio==4.32.2
- ```
-
-13. Download and install DeepSpeed:
- ```
- curl -LO https://github.com/erew123/alltalk_tts/releases/download/DeepSpeed-14.0/deepspeed-0.14.0+ce78a63-cp311-cp311-win_amd64.whl
- ```
- ```
- pip install deepspeed-0.14.0+ce78a63-cp311-cp311-win_amd64.whl
- ```
- ```
- del deepspeed-0.14.0+ce78a63-cp311-cp311-win_amd64.whl
- ```
-
-14. Install Parler:
- ```
- pip install -r system\requirements\requirements_parler.txt
- ```
-
-Ignore the `ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
-coqui-tts 0.24.1 requires transformers<4.41.0,>=4.33.0, but you have transformers 4.43.3 which is incompatible.` message
-
-
-15. Clean the conda environment:
- ```
- .\alltalk_environment\conda\Scripts\conda clean --all --force-pkgs-dirs -y
- ```
-
-16. Decide whether to downgrade transformers for XTTS streaming support:
- - If you want XTTS streaming support, run the following, ignoring the error message similar to the one on Step 14:
- ```
- pip install transformers==4.40.2
- ```
- - If you don't need XTTS streaming support, keep transformers at version 4.43.3.
-
-Note: XTTS streaming may not work with versions of transformers other than 4.40.2.
-
-
-# Detailed Guide to creating the AllTalk Batch Files
-
-These batch files are used to start different parts of the AllTalk program. You'll need to edit them to match where you installed AllTalk on your computer.
-
-## What You Need to Change
-
-1. The parts of the script that points to the drive letter and the folder path to the `alltalk_tts` folder e.g. if your path was `C:\myfiles\alltalk_tts`, you would change `{drive_letter}:\{path_to_alltalk_folder}` to `C:\myfiles\alltalk_tts`
-2. The Python script name at the end of some files
-
-## Steps to Edit the Files
-
-1. Open Notepad and create a file. You will insert the **Main Example** from below and change the `{drive_letter}:\{path_to_alltalk_folder}` to match your correct folder path.
-2. Change all instances of `{drive_letter}:\{path_to_alltalk_folder}` to your AllTalk installation folder.
-3. Add and change the Python script call name at the end of the file (except for start_environment.bat). (call's are detailed below)
-4. Save the file with the correct file after making these changes and ensure they are BAT files and not TXT files.
-
-## Main Example (for all files)
-
-```batch
-@echo off
-cd /D "{drive_letter}:\{path_to_alltalk_folder}\alltalk_tts\"
-set CONDA_ROOT_PREFIX={drive_letter}:\{path_to_alltalk_folder}\alltalk_tts\alltalk_environment\conda
-set INSTALL_ENV_DIR={drive_letter}:\{path_to_alltalk_folder}\alltalk_tts\alltalk_environment\env
-call "{drive_letter}:\{path_to_alltalk_folder}\alltalk_tts\alltalk_environment\conda\condabin\conda.bat" activate "{drive_letter}:\{path_to_alltalk_folder}\alltalk_tts\alltalk_environment\env"
-```
-
-Replace `{drive_letter}` and `{path_to_alltalk_folder}` with your specific path.
-For example: `C:\myfiles\alltalk_tts\`
-
-## Specific Changes for Each File
-
-After the common part above, each file (except start_environment.bat) has a line to run a Python script. Add the following line to the bottom of each file as follows:
-
-### start_alltalk.bat
-```
-call python script.py
-```
-
-### start_diagnostics.bat
-```
-call python diagnostics.py
-```
-
-### start_finetune.bat
-```
-call python finetune.py
-```
-
-### start_environment.bat
-This file doesn't run a Python script, so no additional line is needed.
-
-## Example of a Fully Edited File
-
-Here's how `start_alltalk.bat` might look after editing, assuming `c:\myfiles\alltalk_tts` is the path you are using:
-
-```batch
-@echo off
-cd /D "C:\myfiles\alltalk_tts\"
-set CONDA_ROOT_PREFIX=C:\myfiles\alltalk_tts\alltalk_environment\conda
-set INSTALL_ENV_DIR=C:\myfiles\alltalk_tts\alltalk_environment\env
-call "C:\myfiles\alltalk_tts\alltalk_environment\conda\condabin\conda.bat" activate "C:\myfiles\alltalk_tts\alltalk_environment\env"
-call python script.py
-```
-
-## Important Notes
-
-- Make sure to keep the quotes (`"`) around the file paths.
-- Don't change anything else in the files unless you're sure you know what you're doing.
-- Ensure the files do not have a TXT extension on the name, but are Windows BAT files.
-- After making these changes, the batch files should work correctly with your AllTalk installation.
-
-
+- NVIDIA GPUs: Full support on Windows and Linux.
+- AMD GPUs: Limited experimental support on Linux (mainly for XTTS).
+- Intel ARC GPUs: No specific support currently.
+- Apple Silicon (M1/M2): No GPU acceleration for AllTalk TTS engines currently.
---
+### 🟩 Quick Setup (Recommended for most users)
+
+For a fast and straightforward installation and recommended:
+
+- [Standalone Installation](https://github.com/erew123/alltalk_tts/wiki/Install-%E2%80%90-Standalone-Installation)
+ - For users who want to run AllTalk TTS as a standalone application.
+ - Available for Windows and Linux.
+
+- [Text-generation-webui Installation](https://github.com/erew123/alltalk_tts/wiki/Install-%E2%80%90-Text%E2%80%90generation%E2%80%90webui-Installation)
+ - For users who want to integrate AllTalk TTS as a part of [Text-generation-webui](https://github.com/oobabooga/text-generation-webui).
+ - Available for Windows and Linux.
+ - Includes information on the Remote Extension option.
+
+These methods use scripts that automate most of the installation process, making it easier for users to get started quickly.
+
+### 🟩 Manual Installation
+
+For users who prefer more control over the installation process or need to troubleshoot:
+
+- [Manual Installation Guide](https://github.com/erew123/alltalk_tts/wiki/Install-%E2%80%90-Manual-Installation-Guide)
+ - Detailed step-by-step instructions for manual installation.
+ - Available for Windows, Linux, and Mac (theoretical).
+ - Recommended for advanced users or those with specific setup requirements.
+
+### 🟩 Google Colab Installation
+
+For users who want to run AllTalk TTS in a cloud environment:
+
+- [Google Colab Installation](https://github.com/erew123/alltalk_tts/wiki/Google-COLAB)
+ - Instructions for setting up and running AllTalk TTS in Google Colab.
+ - Ideal for users who want to try AllTalk TTS without installing it locally.
+
+---
### 🟪 Updating
As long as you did the `git clone` method to setup initially, you will be able to go into the folder and use `git pull` to download updates.
-There was an issue with the `.gitignore` file, so if you cannot git pull on a build that is pre 11 June 2024, you can `git reset --hard origin/alltalkbeta` and then git pull (your config file will be reset).
-
---
### 🟨 Diagnostics Help with Issues/Start-up problems etc.