From 51a8e93df7879c7b847515e18270f3be072aaa73 Mon Sep 17 00:00:00 2001 From: "q5sys (JT)" Date: Thu, 11 Jan 2024 21:21:30 -0500 Subject: [PATCH] Update README.md Add specific section for deepspeed installation on linux for standalone app. Related to https://github.com/erew123/alltalk_tts/issues/62 --- README.md | 27 ++++++++++++++++++++++++--- 1 file changed, 24 insertions(+), 3 deletions(-) diff --git a/README.md b/README.md index 72baa97..1c011a9 100644 --- a/README.md +++ b/README.md @@ -361,6 +361,8 @@ If you've compacted and moved your model, its highly unlikely you would want to ➡️DeepSpeed requires an Nvidia Graphics card!⬅️ +- #### DeepSpeed Installation for Text generation webUI + 1) Preferably use your built in package manager to install CUDA tools. Alternatively download and install the Nvidia Cuda Toolkit for Linux [Nvidia Cuda Toolkit 11.8 or 12.1](https://developer.nvidia.com/cuda-toolkit-archive)

2) Open a terminal console.

3) Install libaio-dev (however your Linux version installs things) e.g. `sudo apt install libaio-dev`

@@ -372,13 +374,32 @@ If you've compacted and moved your model, its highly unlikely you would want to `export CUDA_HOME=/etc/alternatives/cuda`

**every** time you run `./cmd_linux.sh`.

If you try to start DeepSpeed with the CUDA_HOME path set incorrectly, expect an error similar to `[Errno 2] No such file or directory: /home/yourname/text-generation-webui/installer_files/env/bin/nvcc`

-9) Now install deepspeed with pip install deepspeed

-10) You can now start Text generation webUI `python server.py` ensuring to activate your extensions.

+8) Now install deepspeed with pip install deepspeed

+9) You can now start Text generation webUI `python server.py` ensuring to activate your extensions.

Just to reiterate, starting Text-generation-webUI with `./start_linux.sh` will overwrite the CUDA_HOME variable unless you have permanently changed it, hence always starting it with `./cmd_linux.sh` **then** setting the environment variable manually (step 7) and **then** `python server.py`, which is how you would need to run it each time, unless you permanently set the environment variable for CUDA_HOME within Text-generation-webUI's standard Python environment.

**Removal** - If it became necessary to uninstall DeepSpeed, you can do so with `./cmd_linux.sh` and then `pip uninstall deepspeed`

+ +- #### DeepSpeed Installation for standalone alltalk_tts app + +1) Preferably use your built in package manager to install CUDA tools. Alternatively download and install the Nvidia Cuda Toolkit for Linux [Nvidia Cuda Toolkit 11.8 or 12.1](https://developer.nvidia.com/cuda-toolkit-archive)

+2) Open a terminal console.

+3) Install libaio-dev (however your Linux version installs things) e.g. `sudo apt install libaio-dev` If you are using an RPM based distribution the package will probably be named `libaio-devel`.

+4) Navigate to your alltalk_tts folder e.g. `cd alltext_tts`

+5) Start the alltalk_tts Python environment `conda activate alltalkenv`

+6) You will need to set the **CUDA_HOME** environment variable. Details to change it each time are on the next step. Below is a link to Conda's manual and changing environment variables permanently though its possible changing it permanently could affect other extensions, you would have to test.

+[Conda manual - Environment variables](https://conda.io/projects/conda/en/latest/user-guide/tasks/manage-environments.html#set-env-vars)

+7) Set the environment path for CUDA_HOME e.g. `export CUDA_HOME=/usr/local/cuda-12.1/bin` or `export CUDA_HOME=/etc/alternative/cuda`. Verify the path on your Linux distro before setting this variable.
+If you try to start DeepSpeed with the CUDA_HOME path set incorrectly, expect an error similar to `[Errno 2] No such file or directory` or `CUDA_HOME does not exist`

+If you set the `env` variable and you receive an error of `[Errno 2] No such file or directory: '/usr/local/cuda-12.1/bin/bin/nvcc'` with a double `bin/bin` path, you can remove the env variable with `unset CUDA_HOME` and add the path to your linux PATH env variable with: `export PATH=$PATH:/usr/local/cuda-12.1/bin`. (Verify the path on your Linux distro before setting this variable.) However if you are using other applications that use CUDA, you will want to verify that configuring the path did not break those applications.

+8) Now install deepspeed with `pip install deepspeed`

+9) You can now start the alltalk_tts webUI with `python script.py`

+

+**Removal** - If it became necessary to uninstall DeepSpeed, you can do so by entering your `alltalkenv` enviroment `conda activate alltalkenv` and then running `pip uninstall deepspeed`

+ + - + #### 🟢🟡 Windows Installation DeepSpeed v11.2 will work on the current default text-generation-webui Python 3.11 environment! You have 2x options for how to setup DeepSpeed on Windows. A quick way (🟢Option 1) and a long way (🟡Option 2).