Merge pull request #517 from Paladinium/alltalkbeta_docker_improvements

Refactoring docker images, updating CUDA to 12.6 and pytorch 2.6
2025-02-12 22:11:40 +01:00 · 2025-02-12 22:01:34 +01:00 · 2025-02-11 22:41:41 +01:00 · 2025-02-11 16:41:20 -05:00 · 2025-01-30 06:26:44 +00:00 · 2025-01-28 19:54:37 +00:00
628 changed files with 72157 additions and 7351 deletions
@@ -0,0 +1 @@
 ko_fi: erew123
@@ -0,0 +1,33 @@
 ---
 name: Support request OR Bug report
 about: Create a report to help us improve
 title: ''
 labels: ''
 assignees: ''
 ---
 🔴 If you have installed AllTalk in a custom Python environment, I will only be able to provide limited assistance/support. AllTalk draws on a variety of scripts and libraries that are not written or managed by myself, and they may fail, error or give strange results in custom built python environments.
 🔴 Please generate a diagnostics report and upload the "diagnostics.log" as this helps me understand your configuration.
 https://github.com/erew123/alltalk_tts/tree/main?#-how-to-make-a-diagnostics-report-file
 **Describe the bug**
 A clear and concise description of what the bug is.
 **To Reproduce**
 Steps to reproduce the behaviour:
 **Screenshots**
 If applicable, add screenshots to help explain your problem.
 **Text/logs**
 If applicable, copy/paste in your logs here from the console.
 **Desktop (please complete the following information):**
 AllTalk was updated: [approx. date]
 Custom Python environment: [yes/no give details if yes]
 Text-generation-webUI was updated: [approx. date]
 **Additional context**
 Add any other context about the problem here.
@@ -0,0 +1,20 @@
 ---
 name: Feature request
 about: Suggest an idea for this project
 title: ''
 labels: ''
 assignees: ''
 ---
 **Is your feature request related to a problem? Please describe.**
 A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]
 **Describe the solution you'd like**
 A clear and concise description of what you want to happen.
 **Describe alternatives you've considered**
 A clear and concise description of any alternative solutions or features you've considered.
 **Additional context**
 Add any other context or screenshots about the feature request here.
@@ -0,0 +1,33 @@
 ---
 name: Support request OR Bug report
 about: Create a report to help us improve
 title: ''
 labels: ''
 assignees: ''
 ---
 🔴 If you have installed AllTalk in a custom Python environment, I will only be able to provide limited assistance/support. AllTalk draws on a variety of scripts and libraries that are not written or managed by myself, and they may fail, error or give strange results in custom built python environments.
 🔴 Please generate a diagnostics report and upload the "diagnostics.log" as this helps me understand your configuration.
 https://github.com/erew123/alltalk_tts/tree/main?#-how-to-make-a-diagnostics-report-file
 **Describe the bug**
 A clear and concise description of what the bug is.
 **To Reproduce**
 Steps to reproduce the behaviour:
 **Screenshots**
 If applicable, add screenshots to help explain your problem.
 **Text/logs**
 If applicable, copy/paste in your logs here from the console.
 **Desktop (please complete the following information):**
 AllTalk was updated: [approx. date]
 Custom Python environment: [yes/no give details if yes]
 Text-generation-webUI was updated: [approx. date]
 **Additional context**
 Add any other context about the problem here.
@@ -0,0 +1,53 @@
 # This workflow will install Python dependencies, run tests and lint with a single version of Python
 # For more information see: https://docs.github.com/en/actions/automating-builds-and-tests/building-and-testing-python
 name: Python CI
 on: [push, pull_request]
 permissions:
  contents: read
 jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - name: Set up Python 3.11.9
        uses: actions/setup-python@v3
        if: ${{ hashFiles('./test') != '' }}
        with:
          python-version: "3.11.9"
      - name: Install dependencies
        if: ${{ hashFiles('./test') != '' }}
        run: |
          python -m pip install --upgrade pip
          if [ -f system/requirements/requirements_unit_test.txt ]; then
            pip install -r system/requirements/requirements_unit_test.txt;
          fi
      - name: Test with pytest
        if: ${{ hashFiles('./test') != '' }}
        run: |
          pytest test --doctest-modules --junitxml=junit/test-results.xml --cov=com --cov-report=xml --cov-report=html
      - name: Surface failing tests
        if: always()
        uses: pmeier/pytest-results-action@v0.6.0
        with:
          # A list of JUnit XML files, directories containing the former, and wildcard
          # patterns to process.
          # See @actions/glob for supported patterns.
          path: junit/test-results.xml
          # (Optional) Add a summary of the results at the top of the report
          summary: true
          # (Optional) Select which results should be included in the report.
          # Follows the same syntax as `pytest -r`
          display-options: fEX
          # (Optional) Fail the workflow if no JUnit XML was found.
          fail-on-empty: false
          # (Optional) Title of the test results section in the workflow summary
          title: Test results
@@ -1,5 +1,176 @@
-voices/*.*
+voices/*
-models/*.*
+models/*
-outputs/*.*
+outputs/*
-config.json
+finetune/*
-models.json
+alltalk_environment/*
 system/config/at_github_sha.json
 system/tts_engines/tts_engines.json
 confignew.json
 diagnostics.log
 # Byte-compiled / optimized / DLL files
 __pycache__/
 *.py[cod]
 *$py.class
 # C extensions
 *.so
 # Distribution / packaging
 .Python
 build/
 develop-eggs/
 dist/
 downloads/
 eggs/
 .eggs/
 lib/
 lib64/
 parts/
 sdist/
 var/
 wheels/
 share/python-wheels/
 *.egg-info/
 .installed.cfg
 *.egg
 MANIFEST
 # PyInstaller
 #  Usually these files are written by a python script from a template
 #  before PyInstaller builds the exe, so as to inject date/other infos into it.
 *.manifest
 *.spec
 # Installer logs
 pip-log.txt
 pip-delete-this-directory.txt
 # Unit test / coverage reports
 htmlcov/
 .tox/
 .nox/
 .coverage
 .coverage.*
 .cache
 nosetests.xml
 coverage.xml
 *.cover
 *.py,cover
 .hypothesis/
 .pytest_cache/
 cover/
 # Translations
 *.mo
 *.pot
 # Django stuff:
 *.log
 local_settings.py
 db.sqlite3
 db.sqlite3-journal
 # Flask stuff:
 instance/
 .webassets-cache
 # Scrapy stuff:
 .scrapy
 # Sphinx documentation
 docs/_build/
 # PyBuilder
 .pybuilder/
 target/
 # Jupyter Notebook
 .ipynb_checkpoints
 # IPython
 profile_default/
 ipython_config.py
 # pyenv
 #   For a library or package, you might want to ignore these files since the code is
 #   intended to run in multiple environments; otherwise, check them in:
 # .python-version
 # pipenv
 #   According to pypa/pipenv#598, it is recommended to include Pipfile.lock in version control.
 #   However, in case of collaboration, if having platform-specific dependencies or dependencies
 #   having no cross-platform support, pipenv may install dependencies that don't work, or not
 #   install all needed dependencies.
 #Pipfile.lock
 # poetry
 #   Similar to Pipfile.lock, it is generally recommended to include poetry.lock in version control.
 #   This is especially recommended for binary packages to ensure reproducibility, and is more
 #   commonly ignored for libraries.
 #   https://python-poetry.org/docs/basic-usage/#commit-your-poetrylock-file-to-version-control
 #poetry.lock
 # pdm
 #   Similar to Pipfile.lock, it is generally recommended to include pdm.lock in version control.
 #pdm.lock
 #   pdm stores project-wide configurations in .pdm.toml, but it is recommended to not include it
 #   in version control.
 #   https://pdm.fming.dev/latest/usage/project/#working-with-version-control
 .pdm.toml
 .pdm-python
 .pdm-build/
 # PEP 582; used by e.g. github.com/David-OConnor/pyflow and github.com/pdm-project/pdm
 __pypackages__/
 # Celery stuff
 celerybeat-schedule
 celerybeat.pid
 # SageMath parsed files
 *.sage.py
 # Environments
 .env
 .venv
 env/
 venv/
 ENV/
 env.bak/
 venv.bak/
 # Spyder project settings
 .spyderproject
 .spyproject
 # Rope project settings
 .ropeproject
 # mkdocs documentation
 /site
 # mypy
 .mypy_cache/
 .dmypy.json
 dmypy.json
 # Pyre type checker
 .pyre/
 # pytype static type analyzer
 .pytype/
 # Cython debug symbols
 cython_debug/
 # PyCharm
 #  JetBrains specific template is maintained in a separate JetBrains.gitignore that can
 #  be found at https://github.com/github/gitignore/blob/main/Global/JetBrains.gitignore
 #  and can be added to the global gitignore or merged into this file.  For a more nuclear
 #  option (not recommended) you can uncomment the following to ignore the entire idea folder.
 .idea/*
 # Docker build
 docker/deepspeed/build
@@ -0,0 +1,9 @@
 {
    "files.associations": {
        "*.html": "jinja-html"
    },
    "emmet.includeLanguages": {
        "jinja-html": "html",
        "jinja2": "html"
    },
 }
@@ -0,0 +1,78 @@
 # Docker
 The Docker image currently works on Windows and Linux, optionally supporting NVIDIA GPUs.
 ## General Remarks
 - The resulting Docker image is 21 GB in size. Building might require even more disk space temporarily.
    - Another 15 GB is required for building DeepSpeed
 - Build time depends on your hardware and internet connection. Expect at least 20-30min to be normal for a full build.
  - This includes building the conda environment as well as DeepSpeed, which is the basis for the alltalk Docker image.
 - The Docker build for alltalk:
  - Downloads XTTS as default TTS engine
  - Enables RVC by default
  - Downloads all supported RVC models
  - Enables deepspeed by default
 - Starting the Docker image should only a few seconds due to all the steps that were already executed during build.
 ## Docker for Linux
 ### Ubuntu Specific Setup for GPUs
 1. Make sure the latest nvidia drivers are installed: `sudo ubuntu-drivers install`
 1. Install Docker your preferred way. One way to do it is to follow the official documentation [here](https://docs.docker.com/engine/install/ubuntu/#uninstall-old-versions).
    - Start by uninstalling the old versions
    - Follow the "apt" repository installation method
    - Check that everything is working with the "hello-world" container
 1. If, when launching the docker contain, you have an error message saying that the GPU cannot be used, you might have to install [Nvidia Docker Container Toolkit](https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/install-guide.html).
    - Install with the "apt" method
    - Run the docker configuration command
      ```sudo nvidia-ctk runtime configure --runtime=docker```
    - Restart docker
 ## Docker for Windows (WSL2)
 ### Windows Specific Setup for GPUs
 > Make sure your Nvidia drivers are up to date: https://www.nvidia.com/download/index.aspx
 1. Install WSL2 in PowerShell with `wsl --install` and restart
 2. Open PowerShell, type and enter ```ubuntu```.  It should now load you into wsl2
 3. Remove the original nvidia cache key: `sudo apt-key del 7fa2af80`
 4. Download CUDA toolkit keyring: `wget https://developer.download.nvidia.com/compute/cuda/repos/wsl-ubuntu/x86_64/cuda-keyring_1.1-1_all.deb`
 5. Install keyring: `sudo dpkg -i cuda-keyring_1.1-1_all.deb`
 6. Update package list: `sudo apt-get update`
 7. Install CUDA toolkit: `sudo apt-get -y install cuda-toolkit-12-6`
 8. Install Docker Desktop using WSL2 as the backend
 9. Restart
 10. If you wish to monitor the terminal remotely via SSH, follow [this guide](https://www.hanselman.com/blog/how-to-ssh-into-wsl2-on-windows-10-from-an-external-machine).
 11. Open PowerShell, type ```ubuntu```, [then follow below](#building-and-running-in-docker)
 ## Building and Running in Docker
 1. Open a terminal (or Ubuntu WSL) and go where you cloned the repo
 3. Build the image with `./docker-build.sh`
 4. Start the container with `./docker-start.sh`
 5. Visit `http://localhost:7851/` or remotely with `http://<ip>:7851`
 ## Arguments for building and starting docker
 There are various arguments to customize the build and start of the docker image.
 ### Arguments for `docker-build.sh`
 - `--tts_model` allows to choose the TTS model that is used by default. Valid values are `piper`, `vits`, `xtts`. Defaults to `xtts`.
  - Example: `docker-build.sh --tts_model piper`
 - `--tag` allows to choose the docker tag. Defaults to `latest`.
  - Example: `docker-build.sh --tag mytag`
 - `--clean` allows remove existing dependency build like conda environment or DeepSpeed.
    - Example: `docker-build.sh --clean`
 ### Arguments for `docker-start.sh`
 - `--config` lets you choose a config JSON file which can subset of `confignew.json`. This allows you to change only 
  few values and leave the rest as defined in the default `confignew.json` file.
  - Example: `docker-start.sh --config /my/config/file.json` with content `{"branding": "My Brand "}` will just change
    the branding in `confignew.json`.
 - `--voices` lets you add voices for the TTS engine in WAV format. You have to specify the folder containing all
  voice files.
  - Example: `docker-start.sh --voices /my/voices/dir`
 - `--rvc_voices` similar to voices, this option lets you pick the folder containing the RVC models.
  - Example: `docker-start.sh --rvc_voices /my/rvc/voices/dir`
 - `--no_ui` allows you to not expose port 7852 for the gradio interface. Note that you still have to set `launch_gradio`
  to `false` via JSON file passed to `--config`.
 - `--tag` allows to choose the docker tag of the image to run. Defaults to `latest`.
    - Example: `docker-start.sh --tag mytag`
 - Since the above commands only address the most important options, you might pass additional arbitrary docker commands
    to the `docker-start.sh`.
@@ -1,285 +0,0 @@
 // Copyright (c) Microsoft Corporation.
 // SPDX-License-Identifier: Apache-2.0
 // DeepSpeed Team
 #include <ATen/cuda/CUDAContext.h>
 #include <torch/extension.h>
 #include <cassert>
 #include <vector>
 #include "quantization.h"
 template <typename T>
 at::Tensor ds_quantize(at::Tensor& vals, int groups, int bits)
 {
    auto t_size = vals.sizes();
    int size = 1;
    for (auto dim : t_size) size *= dim;
    if ((((size / groups) - 1) / 4096 + 1) <= 256) {
        launch_fake_quantize_kernel(
            (T*)vals.data_ptr(), size, groups, bits, at::cuda::getCurrentCUDAStream());
    }
    return vals;
 }
 template <typename T>
 at::Tensor ds_sr_quantize(at::Tensor& vals, int groups, int bits)
 {
    auto t_size = vals.sizes();
    int size = 1;
    for (auto dim : t_size) size *= dim;
    if (((size / groups) / 4 / 1024) <= 256) {
        launch_sr_fake_quantize_kernel(
            (T*)vals.data_ptr(), size, groups, bits, at::cuda::getCurrentCUDAStream());
    }
    return vals;
 }
 template <typename T>
 at::Tensor ds_quantize_asym(at::Tensor& vals, int groups, int bits)
 {
    auto t_size = vals.sizes();
    int size = 1;
    for (auto dim : t_size) size *= dim;
    if ((((size / groups) - 1) / 4096 + 1) <= 256) {
        launch_fake_quantize_kernel_asym(
            (T*)vals.data_ptr(), size, groups, bits, at::cuda::getCurrentCUDAStream());
    }
    return vals;
 }
 template <typename T>
 at::Tensor ds_sr_quantize_asym(at::Tensor& vals, int groups, int bits)
 {
    auto t_size = vals.sizes();
    int size = 1;
    for (auto dim : t_size) size *= dim;
    if (((size / groups) / 4 / 1024) <= 256) {
        launch_sr_fake_quantize_kernel_asym(
            (T*)vals.data_ptr(), size, groups, bits, at::cuda::getCurrentCUDAStream());
    }
    return vals;
 }
 std::vector<at::Tensor> quantize_kernel(at::Tensor& input_vals,
                                        int groups,
                                        int numBits,
                                        quantize::Type quantType)
 {
    auto dtype = at::kFloat;
    auto params_options = at::TensorOptions()
                              .dtype(dtype)
                              .layout(at::kStrided)
                              .device(at::kCUDA)
                              .requires_grad(false);
    const int param_elems = (quantize::requires_offset(quantType)) ? 2 : 1;
    auto params = torch::empty({groups, param_elems}, params_options);
    auto output_options = at::TensorOptions()
                              .dtype(at::kChar)
                              .layout(at::kStrided)
                              .device(at::kCUDA)
                              .requires_grad(false);
    auto output_sizes = input_vals.sizes().vec();
    output_sizes[output_sizes.size() - 1] /= numBits == 8 ? 1 : 2;
    auto output = torch::empty(output_sizes, output_options);
    const int elems_per_group = at::numel(input_vals) / groups;
    launch_quant((int8_t*)output.data_ptr(),
                 (float*)params.data_ptr(),
                 (__half*)input_vals.data_ptr(),
                 groups,
                 elems_per_group,
                 numBits,
                 quantType,
                 at::cuda::getCurrentCUDAStream());
    return {output, params};
 }
 template <typename T>
 at::Tensor dequantize(at::Tensor& quantized_data,
                      at::Tensor& params,
                      int groups,
                      int num_bits,
                      quantize::Type quant_type)
 {
    auto dtype = (std::is_same<T, float>::value) ? torch::kFloat32 : torch::kFloat16;
    auto output_options = at::TensorOptions()
                              .dtype(dtype)
                              .layout(at::kStrided)
                              .device(at::kCUDA)
                              .requires_grad(false);
    auto output_sizes = quantized_data.sizes().vec();
    output_sizes[output_sizes.size() - 1] *= num_bits == 8 ? 1 : 2;
    auto output = torch::empty(output_sizes, output_options);
    const int total_elems = at::numel(output);
    const int elems_per_group = total_elems / groups;
    launch_dequantize_kernel((T*)output.data_ptr(),
                             (const int8_t*)quantized_data.data_ptr(),
                             (const float*)params.data_ptr(),
                             quant_type,
                             num_bits,
                             elems_per_group,
                             total_elems,
                             at::cuda::getCurrentCUDAStream());
    return output;
 }
 at::Tensor dequantize_int4_to_half_experimental(at::Tensor& data_in,
                                                at::Tensor& scale_buffer,
                                                at::Tensor& min_val_buffer,
                                                int num_group,
                                                int group_size)
 {
    auto output_options = at::TensorOptions().dtype(at::kHalf).device(at::kCUDA);
    auto output = torch::empty({num_group, group_size}, output_options);
    launch_dequantize_int4_to_half_experimental((uint8_t*)data_in.data_ptr(),
                                                (half*)output.data_ptr(),
                                                (half*)scale_buffer.data_ptr(),
                                                (half*)min_val_buffer.data_ptr(),
                                                num_group,
                                                group_size,
                                                at::cuda::getCurrentCUDAStream());
    return output;
 }
 std::vector<at::Tensor> ds_swizzle_quant(at::Tensor& input_vals,
                                         int groups,
                                         int num_bits,
                                         quantize::Type quant_type,
                                         int pipeline_size,
                                         int nodes,
                                         int devices_per_node)
 {
    auto scales_options = at::TensorOptions()
                              .dtype(at::kFloat)
                              .layout(at::kStrided)
                              .device(at::kCUDA)
                              .requires_grad(false);
    const int scales_elems = (quantize::requires_offset(quant_type)) ? 2 : 1;
    auto scales = torch::empty({groups, scales_elems}, scales_options);
    auto output_options = at::TensorOptions()
                              .dtype(at::kChar)
                              .layout(at::kStrided)
                              .device(at::kCUDA)
                              .requires_grad(false);
    const int quantization_scalar = 8 / num_bits;
    const int compressed_vals = at::numel(input_vals) / quantization_scalar;
    auto output = torch::empty({compressed_vals}, output_options);
    const int elems_per_group = at::numel(input_vals) / groups;
    launch_swizzled_quant((int8_t*)output.data_ptr(),
                          (float*)scales.data_ptr(),
                          (__half*)input_vals.data_ptr(),
                          num_bits,
                          quant_type,
                          groups,
                          elems_per_group,
                          pipeline_size,
                          nodes,
                          devices_per_node,
                          at::cuda::getCurrentCUDAStream());
    return {output, scales};
 }
 std::vector<at::Tensor> quantized_reduction(at::Tensor& input_vals,
                                            at::Tensor& input_scales,
                                            int in_groups,
                                            int out_groups,
                                            int num_bits,
                                            quantize::Type quant_type,
                                            int devices_per_node)
 {
    auto scales_options = at::TensorOptions()
                              .dtype(at::kFloat)
                              .layout(at::kStrided)
                              .device(at::kCUDA)
                              .requires_grad(false);
    const int scales_elems = (quantize::requires_offset(quant_type)) ? 2 : 1;
    auto scales = torch::empty({out_groups, scales_elems}, scales_options);
    auto output_options = at::TensorOptions()
                              .dtype(at::kChar)
                              .layout(at::kStrided)
                              .device(at::kCUDA)
                              .requires_grad(false);
     std::vector<int64_t> sz_vector(input_vals.sizes().begin(), input_vals.sizes().end());
     sz_vector[sz_vector.size() - 1] = sz_vector.back() / devices_per_node;  // num of GPU per nodes
     at::IntArrayRef sz(sz_vector);
     auto output = torch::empty(sz, output_options);
     const int elems_per_in_tensor = at::numel(input_vals) / devices_per_node;
     const int elems_per_in_group = elems_per_in_tensor / (in_groups / devices_per_node);
     const int elems_per_out_group = elems_per_in_tensor / out_groups;
 // old code:
 //    std::vector<long int> sz(input_vals.sizes().begin(), input_vals.sizes().end());
 //    sz[sz.size() - 1] = sz.back() / devices_per_node;  // num of GPU per nodes
 //    const int elems_per_in_tensor = at::numel(input_vals) / devices_per_node;
 //    auto output = torch::empty(sz, output_options);
 //    const int elems_per_in_group = elems_per_in_tensor / (in_groups / devices_per_node);
 //    const int elems_per_out_group = elems_per_in_tensor / out_groups;
    launch_dequant_reduce((int8_t*)output.data_ptr(),
                          (float*)scales.data_ptr(),
                          (const int8_t*)input_vals.data_ptr(),
                          (const float*)input_scales.data_ptr(),
                          devices_per_node,
                          num_bits,
                          quant_type,
                          out_groups,
                          elems_per_out_group,
                          elems_per_in_tensor,
                          in_groups / devices_per_node,
                          elems_per_in_group,
                          at::cuda::getCurrentCUDAStream());
    return {output, scales};
 }
 PYBIND11_MODULE(TORCH_EXTENSION_NAME, m)
 {
    m.def("ds_quantize_fp32", &ds_quantize<float>, "DeepSpeed Quantize with fp32 (CUDA)");
    m.def("ds_quantize_fp16", &ds_quantize<__half>, "DeepSpeed Quantize with fp16 (CUDA)");
    m.def("ds_sr_quantize_fp32", &ds_sr_quantize<float>, "DeepSpeed Quantize with fp32 (CUDA)");
    m.def("ds_sr_quantize_fp16", &ds_sr_quantize<__half>, "DeepSpeed Quantize with fp16 (CUDA)");
    m.def("ds_quantize_asym_fp32", &ds_quantize_asym<float>, "DeepSpeed Quantize with fp32 (CUDA)");
    m.def(
        "ds_quantize_asym_fp16", &ds_quantize_asym<__half>, "DeepSpeed Quantize with fp16 (CUDA)");
    m.def("ds_sr_quantize_asym_fp32",
          &ds_sr_quantize_asym<float>,
          "DeepSpeed Quantize with fp32 (CUDA)");
    m.def("ds_sr_quantize_asym_fp16",
          &ds_sr_quantize_asym<__half>,
          "DeepSpeed Quantize with fp16 (CUDA)");
    pybind11::enum_<quantize::Type>(m, "QuantizationType")
        .value("Symmetric", quantize::Type::Symmetric)
        .value("Asymmetric", quantize::Type::Asymmetric)
        .export_values();
    m.def("quantize", &quantize_kernel);
    m.def("dequantize", &dequantize<__half>);
    m.def("dequantize_fp32", &dequantize<float>);
    m.def("dequantize_int4_to_half_experimental",
          &dequantize_int4_to_half_experimental,
          "Dequantize int4 to half (experimental)");
    m.def("swizzle_quant", &ds_swizzle_quant);
    m.def("quantized_reduction", &quantized_reduction);
 }
@@ -1,160 +0,0 @@
 # Copyright (c) Microsoft Corporation.
 # SPDX-License-Identifier: Apache-2.0
 # DeepSpeed Team
 import os
 import torch
 import deepspeed
 import subprocess
 import argparse
 import psutil
 from .ops.op_builder.all_ops import ALL_OPS
 from .git_version_info import installed_ops, torch_info
 from deepspeed.accelerator import get_accelerator
 GREEN = '\033[92m'
 RED = '\033[91m'
 YELLOW = '\033[93m'
 END = '\033[0m'
 SUCCESS = f"{GREEN} [SUCCESS] {END}"
 OKAY = f"{GREEN}[OKAY]{END}"
 WARNING = f"{YELLOW}[WARNING]{END}"
 FAIL = f'{RED}[FAIL]{END}'
 INFO = '[INFO]'
 color_len = len(GREEN) + len(END)
 okay = f"{GREEN}[OKAY]{END}"
 warning = f"{YELLOW}[WARNING]{END}"
 def op_report(verbose=True):
    max_dots = 23
    max_dots2 = 11
    h = ["op name", "installed", "compatible"]
    print("-" * (max_dots + max_dots2 + len(h[0]) + len(h[1])))
    print("DeepSpeed C++/CUDA extension op report")
    print("-" * (max_dots + max_dots2 + len(h[0]) + len(h[1])))
    print("NOTE: Ops not installed will be just-in-time (JIT) compiled at\n"
          "      runtime if needed. Op compatibility means that your system\n"
          "      meet the required dependencies to JIT install the op.")
    print("-" * (max_dots + max_dots2 + len(h[0]) + len(h[1])))
    print("JIT compiled ops requires ninja")
    ninja_status = OKAY if ninja_installed() else FAIL
    print('ninja', "." * (max_dots - 5), ninja_status)
    print("-" * (max_dots + max_dots2 + len(h[0]) + len(h[1])))
    print(h[0], "." * (max_dots - len(h[0])), h[1], "." * (max_dots2 - len(h[1])), h[2])
    print("-" * (max_dots + max_dots2 + len(h[0]) + len(h[1])))
    installed = f"{GREEN}[YES]{END}"
    no = f"{YELLOW}[NO]{END}"
    for op_name, builder in ALL_OPS.items():
        dots = "." * (max_dots - len(op_name))
        is_compatible = OKAY if builder.is_compatible(verbose) else no
        is_installed = installed if installed_ops.get(op_name, False) else no
        dots2 = '.' * ((len(h[1]) + (max_dots2 - len(h[1]))) - (len(is_installed) - color_len))
        print(op_name, dots, is_installed, dots2, is_compatible)
    print("-" * (max_dots + max_dots2 + len(h[0]) + len(h[1])))
 def ninja_installed():
    try:
        import ninja  # noqa: F401 # type: ignore
    except ImportError:
        return False
    return True
 def nvcc_version():
    import torch.utils.cpp_extension
    cuda_home = torch.utils.cpp_extension.CUDA_HOME
    if cuda_home is None:
        return f"{RED} [FAIL] cannot find CUDA_HOME via torch.utils.cpp_extension.CUDA_HOME={torch.utils.cpp_extension.CUDA_HOME} {END}"
    try:
        output = subprocess.check_output([cuda_home + "/bin/nvcc", "-V"], universal_newlines=True)
    except FileNotFoundError:
        return f"{RED} [FAIL] nvcc missing {END}"
    output_split = output.split()
    release_idx = output_split.index("release")
    release = output_split[release_idx + 1].replace(',', '').split(".")
    return ".".join(release)
 def get_shm_size():
    try:
        temp_dir = os.getenv('TEMP') or os.getenv('TMP') or os.path.join(os.path.expanduser('~'), 'tmp')
        shm_stats = psutil.disk_usage(temp_dir)
        shm_size = shm_stats.total
        shm_hbytes = human_readable_size(shm_size)
        warn = []
        if shm_size < 512 * 1024**2:
            warn.append(
                f" {YELLOW} [WARNING] Shared memory size might be too small, consider increasing it. {END}"
            )
            # Add additional warnings specific to your use case if needed.
        return shm_hbytes, warn
    except Exception as e:
        return "UNKNOWN", [f"Error getting shared memory size: {e}"]
 def human_readable_size(size):
    units = ['B', 'KB', 'MB', 'GB', 'TB']
    i = 0
    while size >= 1024 and i < len(units) - 1:
        size /= 1024
        i += 1
    return f'{size:.2f} {units[i]}'
 def debug_report():
    max_dots = 33
    report = [("torch install path", torch.__path__), ("torch version", torch.__version__),
              ("deepspeed install path", deepspeed.__path__),
              ("deepspeed info", f"{deepspeed.__version__}, {deepspeed.__git_hash__}, {deepspeed.__git_branch__}")]
    if get_accelerator().device_name() == 'cuda':
        hip_version = getattr(torch.version, "hip", None)
        report.extend([("torch cuda version", torch.version.cuda), ("torch hip version", hip_version),
                       ("nvcc version", (None if hip_version else nvcc_version())),
                       ("deepspeed wheel compiled w.", f"torch {torch_info['version']}, " +
                        (f"hip {torch_info['hip_version']}" if hip_version else f"cuda {torch_info['cuda_version']}"))
                       ])
    else:
        report.extend([("deepspeed wheel compiled w.", f"torch {torch_info['version']} ")])
    report.append(("shared memory (/dev/shm) size", get_shm_size()))
    print("DeepSpeed general environment info:")
    for name, value in report:
        warns = []
        if isinstance(value, tuple):
            value, warns = value
        print(name, "." * (max_dots - len(name)), value)
        if warns:
            for warn in warns:
                print(warn)
 def parse_arguments():
    parser = argparse.ArgumentParser()
    parser.add_argument('--hide_operator_status',
                        action='store_true',
                        help='Suppress display of installation and compatibility statuses of DeepSpeed operators. ')
    parser.add_argument('--hide_errors_and_warnings', action='store_true', help='Suppress warning and error messages.')
    args = parser.parse_args()
    return args
 def main(hide_operator_status=False, hide_errors_and_warnings=False):
    if not hide_operator_status:
        op_report(verbose=not hide_errors_and_warnings)
    debug_report()
 def cli_main():
    args = parse_arguments()
    main(hide_operator_status=args.hide_operator_status, hide_errors_and_warnings=args.hide_errors_and_warnings)
 if __name__ == "__main__":
    main()
@@ -1,299 +0,0 @@
 // Copyright (c) Microsoft Corporation.
 // SPDX-License-Identifier: Apache-2.0
 // DeepSpeed Team
 #include <ATen/cuda/CUDAContext.h>
 #include <torch/extension.h>
 #include <cassert>
 #include <vector>
 #include "quantization.h"
 template <typename T>
 at::Tensor ds_quantize(at::Tensor& vals, int groups, int bits)
 {
    auto t_size = vals.sizes();
    int size = 1;
    for (auto dim : t_size) size *= dim;
    if ((((size / groups) - 1) / 4096 + 1) <= 256) {
        launch_fake_quantize_kernel(
            (T*)vals.data_ptr(), size, groups, bits, at::cuda::getCurrentCUDAStream());
    }
    return vals;
 }
 template <typename T>
 at::Tensor ds_sr_quantize(at::Tensor& vals, int groups, int bits)
 {
    auto t_size = vals.sizes();
    int size = 1;
    for (auto dim : t_size) size *= dim;
    if (((size / groups) / 4 / 1024) <= 256) {
        launch_sr_fake_quantize_kernel(
            (T*)vals.data_ptr(), size, groups, bits, at::cuda::getCurrentCUDAStream());
    }
    return vals;
 }
 template <typename T>
 at::Tensor ds_quantize_asym(at::Tensor& vals, int groups, int bits)
 {
    auto t_size = vals.sizes();
    int size = 1;
    for (auto dim : t_size) size *= dim;
    if ((((size / groups) - 1) / 4096 + 1) <= 256) {
        launch_fake_quantize_kernel_asym(
            (T*)vals.data_ptr(), size, groups, bits, at::cuda::getCurrentCUDAStream());
    }
    return vals;
 }
 template <typename T>
 at::Tensor ds_sr_quantize_asym(at::Tensor& vals, int groups, int bits)
 {
    auto t_size = vals.sizes();
    int size = 1;
    for (auto dim : t_size) size *= dim;
    if (((size / groups) / 4 / 1024) <= 256) {
        launch_sr_fake_quantize_kernel_asym(
            (T*)vals.data_ptr(), size, groups, bits, at::cuda::getCurrentCUDAStream());
    }
    return vals;
 }
 std::vector<at::Tensor> quantize_kernel(at::Tensor& input_vals,
                                        int groups,
                                        int numBits,
                                        quantize::Type quantType)
 {
    auto dtype = at::kFloat;
    auto params_options = at::TensorOptions()
                              .dtype(dtype)
                              .layout(at::kStrided)
                              .device(at::kCUDA)
                              .requires_grad(false);
    const int param_elems = (quantize::requires_offset(quantType)) ? 2 : 1;
    auto params = torch::empty({groups, param_elems}, params_options);
    auto output_options = at::TensorOptions()
                              .dtype(at::kChar)
                              .layout(at::kStrided)
                              .device(at::kCUDA)
                              .requires_grad(false);
    auto output_sizes = input_vals.sizes().vec();
    output_sizes[output_sizes.size() - 1] /= numBits == 8 ? 1 : 2;
    auto output = torch::empty(output_sizes, output_options);
    const int elems_per_group = at::numel(input_vals) / groups;
    launch_quant((int8_t*)output.data_ptr(),
                 (float*)params.data_ptr(),
                 (__half*)input_vals.data_ptr(),
                 groups,
                 elems_per_group,
                 numBits,
                 quantType,
                 at::cuda::getCurrentCUDAStream());
    return {output, params};
 }
 template <typename T>
 at::Tensor dequantize(at::Tensor& quantized_data,
                      at::Tensor& params,
                      int groups,
                      int num_bits,
                      quantize::Type quant_type)
 {
    auto dtype = (std::is_same<T, float>::value) ? torch::kFloat32 : torch::kFloat16;
    auto output_options = at::TensorOptions()
                              .dtype(dtype)
                              .layout(at::kStrided)
                              .device(at::kCUDA)
                              .requires_grad(false);
    auto output_sizes = quantized_data.sizes().vec();
    output_sizes[output_sizes.size() - 1] *= num_bits == 8 ? 1 : 2;
    auto output = torch::empty(output_sizes, output_options);
    const int total_elems = at::numel(output);
    const int elems_per_group = total_elems / groups;
    launch_dequantize_kernel((T*)output.data_ptr(),
                             (const int8_t*)quantized_data.data_ptr(),
                             (const float*)params.data_ptr(),
                             quant_type,
                             num_bits,
                             elems_per_group,
                             total_elems,
                             at::cuda::getCurrentCUDAStream());
    return output;
 }
 at::Tensor dequantize_int4_to_half_experimental(at::Tensor& data_in,
                                                at::Tensor& scale_buffer,
                                                at::Tensor& min_val_buffer,
                                                int num_group,
                                                int group_size)
 {
    auto output_options = at::TensorOptions().dtype(at::kHalf).device(at::kCUDA);
    auto output = torch::empty({num_group, group_size}, output_options);
    launch_dequantize_int4_to_half_experimental((uint8_t*)data_in.data_ptr(),
                                                (half*)output.data_ptr(),
                                                (half*)scale_buffer.data_ptr(),
                                                (half*)min_val_buffer.data_ptr(),
                                                num_group,
                                                group_size,
                                                at::cuda::getCurrentCUDAStream());
    return output;
 }
 at::Tensor dequantize_int8_to_half_experimental(at::Tensor& data_in,
                                                at::Tensor& scale_buffer,
                                                at::Tensor& min_val_buffer,
                                                int num_group,
                                                int group_size)
 {
    auto output_options = at::TensorOptions().dtype(at::kHalf).device(at::kCUDA);
    auto output = torch::empty({num_group, group_size}, output_options);
    launch_dequantize_int8_to_half_experimental((uint8_t*)data_in.data_ptr(),
                                                (half*)output.data_ptr(),
                                                (half*)scale_buffer.data_ptr(),
                                                (half*)min_val_buffer.data_ptr(),
                                                num_group,
                                                group_size,
                                                at::cuda::getCurrentCUDAStream());
    return output;
 }
 std::vector<at::Tensor> ds_swizzle_quant(at::Tensor& input_vals,
                                         int groups,
                                         int num_bits,
                                         quantize::Type quant_type,
                                         int pipeline_size,
                                         int nodes,
                                         int devices_per_node)
 {
    auto scales_options = at::TensorOptions()
                              .dtype(at::kFloat)
                              .layout(at::kStrided)
                              .device(at::kCUDA)
                              .requires_grad(false);
    const int scales_elems = (quantize::requires_offset(quant_type)) ? 2 : 1;
    auto scales = torch::empty({groups, scales_elems}, scales_options);
    auto output_options = at::TensorOptions()
                              .dtype(at::kChar)
                              .layout(at::kStrided)
                              .device(at::kCUDA)
                              .requires_grad(false);
    const int quantization_scalar = 8 / num_bits;
    const int compressed_vals = at::numel(input_vals) / quantization_scalar;
    auto output = torch::empty({compressed_vals}, output_options);
    const int elems_per_group = at::numel(input_vals) / groups;
    launch_swizzled_quant((int8_t*)output.data_ptr(),
                          (float*)scales.data_ptr(),
                          (__half*)input_vals.data_ptr(),
                          num_bits,
                          quant_type,
                          groups,
                          elems_per_group,
                          pipeline_size,
                          nodes,
                          devices_per_node,
                          at::cuda::getCurrentCUDAStream());
    return {output, scales};
 }
 std::vector<at::Tensor> quantized_reduction(at::Tensor& input_vals,
                                            at::Tensor& input_scales,
                                            int in_groups,
                                            int out_groups,
                                            int num_bits,
                                            quantize::Type quant_type,
                                            int devices_per_node)
 {
    auto scales_options = at::TensorOptions()
                              .dtype(at::kFloat)
                              .layout(at::kStrided)
                              .device(at::kCUDA)
                              .requires_grad(false);
    const int scales_elems = (quantize::requires_offset(quant_type)) ? 2 : 1;
    auto scales = torch::empty({out_groups, scales_elems}, scales_options);
    auto output_options = at::TensorOptions()
                              .dtype(at::kChar)
                              .layout(at::kStrided)
                              .device(at::kCUDA)
                              .requires_grad(false);
    std::vector<int64_t> sz_vector(input_vals.sizes().begin(), input_vals.sizes().end());
    sz_vector[sz_vector.size() - 1] = sz_vector.back() / devices_per_node;  // num of GPU per nodes
    at::IntArrayRef sz(sz_vector);
    auto output = torch::empty(sz, output_options);
    const int elems_per_in_tensor = at::numel(input_vals) / devices_per_node;
    const int elems_per_in_group = elems_per_in_tensor / (in_groups / devices_per_node);
    const int elems_per_out_group = elems_per_in_tensor / out_groups;
    launch_dequant_reduce((int8_t*)output.data_ptr(),
                          (float*)scales.data_ptr(),
                          (const int8_t*)input_vals.data_ptr(),
                          (const float*)input_scales.data_ptr(),
                          devices_per_node,
                          num_bits,
                          quant_type,
                          out_groups,
                          elems_per_out_group,
                          elems_per_in_tensor,
                          in_groups / devices_per_node,
                          elems_per_in_group,
                          at::cuda::getCurrentCUDAStream());
    return {output, scales};
 }
 PYBIND11_MODULE(TORCH_EXTENSION_NAME, m)
 {
    m.def("ds_quantize_fp32", &ds_quantize<float>, "DeepSpeed Quantize with fp32 (CUDA)");
    m.def("ds_quantize_fp16", &ds_quantize<__half>, "DeepSpeed Quantize with fp16 (CUDA)");
    m.def("ds_sr_quantize_fp32", &ds_sr_quantize<float>, "DeepSpeed Quantize with fp32 (CUDA)");
    m.def("ds_sr_quantize_fp16", &ds_sr_quantize<__half>, "DeepSpeed Quantize with fp16 (CUDA)");
    m.def("ds_quantize_asym_fp32", &ds_quantize_asym<float>, "DeepSpeed Quantize with fp32 (CUDA)");
    m.def(
        "ds_quantize_asym_fp16", &ds_quantize_asym<__half>, "DeepSpeed Quantize with fp16 (CUDA)");
    m.def("ds_sr_quantize_asym_fp32",
          &ds_sr_quantize_asym<float>,
          "DeepSpeed Quantize with fp32 (CUDA)");
    m.def("ds_sr_quantize_asym_fp16",
          &ds_sr_quantize_asym<__half>,
          "DeepSpeed Quantize with fp16 (CUDA)");
    pybind11::enum_<quantize::Type>(m, "QuantizationType")
        .value("Symmetric", quantize::Type::Symmetric)
        .value("Asymmetric", quantize::Type::Asymmetric)
        .export_values();
    m.def("quantize", &quantize_kernel);
    m.def("dequantize", &dequantize<__half>);
    m.def("dequantize_fp32", &dequantize<float>);
    m.def("dequantize_int4_to_half_experimental",
          &dequantize_int4_to_half_experimental,
          "Dequantize int4 to half (experimental)");
    m.def("dequantize_int8_to_half_experimental",
          &dequantize_int8_to_half_experimental,
          "Dequantize int8 to half (experimental)");
    m.def("swizzle_quant", &ds_swizzle_quant);
    m.def("quantized_reduction", &quantized_reduction);
 }
@@ -1,161 +0,0 @@
 # Copyright (c) Microsoft Corporation.
 # SPDX-License-Identifier: Apache-2.0
 # DeepSpeed Team
 import os
 import torch
 import deepspeed
 import subprocess
 import argparse
 import psutil
 from .ops.op_builder.all_ops import ALL_OPS
 from .git_version_info import installed_ops, torch_info
 from deepspeed.accelerator import get_accelerator
 GREEN = '\033[92m'
 RED = '\033[91m'
 YELLOW = '\033[93m'
 END = '\033[0m'
 SUCCESS = f"{GREEN} [SUCCESS] {END}"
 OKAY = f"{GREEN}[OKAY]{END}"
 WARNING = f"{YELLOW}[WARNING]{END}"
 FAIL = f'{RED}[FAIL]{END}'
 INFO = '[INFO]'
 color_len = len(GREEN) + len(END)
 okay = f"{GREEN}[OKAY]{END}"
 warning = f"{YELLOW}[WARNING]{END}"
 def op_report(verbose=True):
    max_dots = 23
    max_dots2 = 11
    h = ["op name", "installed", "compatible"]
    print("-" * (max_dots + max_dots2 + len(h[0]) + len(h[1])))
    print("DeepSpeed C++/CUDA extension op report")
    print("-" * (max_dots + max_dots2 + len(h[0]) + len(h[1])))
    print("NOTE: Ops not installed will be just-in-time (JIT) compiled at\n"
          "      runtime if needed. Op compatibility means that your system\n"
          "      meet the required dependencies to JIT install the op.")
    print("-" * (max_dots + max_dots2 + len(h[0]) + len(h[1])))
    print("JIT compiled ops requires ninja")
    ninja_status = OKAY if ninja_installed() else FAIL
    print('ninja', "." * (max_dots - 5), ninja_status)
    print("-" * (max_dots + max_dots2 + len(h[0]) + len(h[1])))
    print(h[0], "." * (max_dots - len(h[0])), h[1], "." * (max_dots2 - len(h[1])), h[2])
    print("-" * (max_dots + max_dots2 + len(h[0]) + len(h[1])))
    installed = f"{GREEN}[YES]{END}"
    no = f"{YELLOW}[NO]{END}"
    for op_name, builder in ALL_OPS.items():
        dots = "." * (max_dots - len(op_name))
        is_compatible = OKAY if builder.is_compatible(verbose) else no
        is_installed = installed if installed_ops.get(op_name, False) else no
        dots2 = '.' * ((len(h[1]) + (max_dots2 - len(h[1]))) - (len(is_installed) - color_len))
        print(op_name, dots, is_installed, dots2, is_compatible)
    print("-" * (max_dots + max_dots2 + len(h[0]) + len(h[1])))
 def ninja_installed():
    try:
        import ninja  # noqa: F401 # type: ignore
    except ImportError:
        return False
    return True
 def nvcc_version():
    import torch.utils.cpp_extension
    cuda_home = torch.utils.cpp_extension.CUDA_HOME
    if cuda_home is None:
        return f"{RED} [FAIL] cannot find CUDA_HOME via torch.utils.cpp_extension.CUDA_HOME={torch.utils.cpp_extension.CUDA_HOME} {END}"
    try:
        output = subprocess.check_output([cuda_home + "/bin/nvcc", "-V"], universal_newlines=True)
    except FileNotFoundError:
        return f"{RED} [FAIL] nvcc missing {END}"
    output_split = output.split()
    release_idx = output_split.index("release")
    release = output_split[release_idx + 1].replace(',', '').split(".")
    return ".".join(release)
 def get_shm_size():
    try:
        temp_dir = os.getenv('TEMP') or os.getenv('TMP') or os.path.join(os.path.expanduser('~'), 'tmp')
        shm_stats = psutil.disk_usage(temp_dir)
        shm_size = shm_stats.total
        shm_hbytes = human_readable_size(shm_size)
        warn = []
        if shm_size < 512 * 1024**2:
            warn.append(
                f" {YELLOW} [WARNING] Shared memory size might be too small, consider increasing it. {END}"
            )
            # Add additional warnings specific to your use case if needed.
        return shm_hbytes, warn
    except Exception as e:
        return "UNKNOWN", [f"Error getting shared memory size: {e}"]
 def human_readable_size(size):
    units = ['B', 'KB', 'MB', 'GB', 'TB']
    i = 0
    while size >= 1024 and i < len(units) - 1:
        size /= 1024
        i += 1
    return f'{size:.2f} {units[i]}'
 def debug_report():
    max_dots = 33
    report = [("torch install path", torch.__path__), ("torch version", torch.__version__),
              ("deepspeed install path", deepspeed.__path__),
              ("deepspeed info", f"{deepspeed.__version__}, {deepspeed.__git_hash__}, {deepspeed.__git_branch__}")]
    if get_accelerator().device_name() == 'cuda':
        hip_version = getattr(torch.version, "hip", None)
        report.extend([("torch cuda version", torch.version.cuda), ("torch hip version", hip_version),
                       ("nvcc version", (None if hip_version else nvcc_version())),
                       ("deepspeed wheel compiled w.", f"torch {torch_info['version']}, " +
                        (f"hip {torch_info['hip_version']}" if hip_version else f"cuda {torch_info['cuda_version']}"))
                       ])
    else:
        report.extend([("deepspeed wheel compiled w.", f"torch {torch_info['version']} ")])
    report.append(("shared memory (/dev/shm) size", get_shm_size()))
    print("DeepSpeed general environment info:")
    for name, value in report:
        warns = []
        if isinstance(value, tuple):
            value, warns = value
        print(name, "." * (max_dots - len(name)), value)
        if warns:
            for warn in warns:
                print(warn)
 def parse_arguments():
    parser = argparse.ArgumentParser()
    parser.add_argument('--hide_operator_status',
                        action='store_true',
                        help='Suppress display of installation and compatibility statuses of DeepSpeed operators. ')
    parser.add_argument('--hide_errors_and_warnings', action='store_true', help='Suppress warning and error messages.')
    args = parser.parse_args()
    return args
 def main(hide_operator_status=False, hide_errors_and_warnings=False):
    if not hide_operator_status:
        op_report(verbose=not hide_errors_and_warnings)
    debug_report()
 def cli_main():
    args = parse_arguments()
    main(hide_operator_status=args.hide_operator_status, hide_errors_and_warnings=args.hide_errors_and_warnings)
 if __name__ == "__main__":
    main()
@@ -1,18 +0,0 @@
 These are the modified files you would need for a MANUAL build of DeepSpeed 11.1 or 11.2.
 They were created by a mix of instructions from erew123 and S95Sedan.
 - You still have to download the DeepSpeed v11.1 or v11.2 installation files from Microsoft to use these files.
 https://github.com/microsoft/DeepSpeed/releases/tag/v0.11.2
 - These files were provided by S95Sedan here: 
 https://github.com/S95Sedan/Deepspeed-Windows
 - They are related to the manual build of DeepSpeed for AllTalk, which is linked here: 
 https://github.com/erew123/alltalk_tts/tree/main?tab=readme-ov-file#-option-2---a-bit-more-complicated
@@ -0,0 +1,124 @@
 ARG GITHUB_REPOSITORY
 FROM ${GITHUB_REPOSITORY}alltalk_environment:latest
 # Argument to choose the model: piper, vits, xtts
 ARG TTS_MODEL="xtts"
 ENV TTS_MODEL=$TTS_MODEL
 ARG DEEPSPEED_VERSION=0.16.2
 ENV DEEPSPEED_VERSION=$DEEPSPEED_VERSION
 ARG ALLTALK_DIR=/opt/alltalk
 ENV GRADIO_SERVER_NAME="0.0.0.0"
 WORKDIR ${ALLTALK_DIR}
 ##############################################################################
 # Install python dependencies (cannot use --no-deps because requirements are not complete)
 ##############################################################################
 COPY system/config system/config
 COPY system/requirements/requirements_standalone.txt system/requirements/requirements_standalone.txt
 COPY system/requirements/requirements_parler.txt system/requirements/requirements_parler.txt
 ENV PIP_CACHE_DIR=${ALLTALK_DIR}/pip_cache
 RUN <<EOR
    conda activate alltalk
    mkdir ${ALLTALK_DIR}/pip_cache
    pip install --no-cache-dir --cache-dir=${ALLTALK_DIR}/pip_cache -r system/requirements/requirements_standalone.txt
    pip install --no-cache-dir --cache-dir=${ALLTALK_DIR}/pip_cache --upgrade gradio==4.32.2
    # Parler:
    pip install --no-cache-dir --cache-dir=${ALLTALK_DIR}/pip_cache -r system/requirements/requirements_parler.txt
    conda clean --all --force-pkgs-dirs -y && pip cache purge
 EOR
 ##############################################################################
 # Install DeepSpeed
 ##############################################################################
 RUN mkdir -p /tmp/deepseped
 COPY docker/deepspeed/build/*.whl /tmp/deepspeed/
 RUN <<EOR
    DEEPSPEED_WHEEL=$(realpath /tmp/deepspeed/*.whl)
    conda activate alltalk
    echo "Using precompiled DeepSpeed wheel at ${DEEPSPEED_WHEEL}"
    CFLAGS="-I$CONDA_PREFIX/include/" LDFLAGS="-L$CONDA_PREFIX/lib/" \
      pip install --no-cache-dir ${DEEPSPEED_WHEEL}
    if [ $? -ne 0 ] ; then
      echo "Failed to install pip dependencies: $RESULT"
      exit 1
    fi
    rm ${DEEPSPEED_WHEEL}
    conda clean --all --force-pkgs-dirs -y && pip cache purge
 EOR
 ##############################################################################
 # Writing scripts to start alltalk:
 ##############################################################################
 RUN <<EOR
    cat << EOF > start_alltalk.sh
 #!/usr/bin/env bash
 source ~/.bashrc
 # Merging config from docker_confignew.json into confignew.json:
 jq -s '.[0] * .[1] * .[2]' confignew.json docker_default_config.json docker_confignew.json  > confignew.json.tmp
 mv confignew.json.tmp confignew.json
 conda activate alltalk
 python script.py
 EOF
    cat << EOF > start_finetune.sh
 #!/usr/bin/env bash
 source ~/.bashrc
 export TRAINER_TELEMETRY=0
 conda activate alltalk
 python finetune.py
 EOF
    cat << EOF > start_diagnostics.sh
 #!/usr/bin/env bash
 source ~/.bashrc
 conda activate alltalk
 python diagnostics.py
 EOF
    chmod +x start_alltalk.sh
    chmod +x start_finetune.sh
    chmod +x start_diagnostics.sh
 EOR
 COPY . .
 ##############################################################################
 # Create script to execute firstrun.py and run it:
 ##############################################################################
 RUN echo $'#!/usr/bin/env bash \n\
 source ~/.bashrc \n\
 conda activate alltalk \n\
 python ./system/config/firstrun.py $@' > ./start_firstrun.sh
 RUN chmod +x start_firstrun.sh
 RUN ./start_firstrun.sh --tts_model $TTS_MODEL
 RUN mkdir -p ${ALLTALK_DIR}/outputs
 RUN mkdir -p /root/.triton/autotune
 ##############################################################################
 # Enable deepspeed for all models:
 ##############################################################################
 RUN find . -name model_settings.json -exec sed -i -e 's/"deepspeed_enabled": false/"deepspeed_enabled": true/g' {} \;
 ##############################################################################
 # Download all RVC models:
 ##############################################################################
 RUN <<EOR
  jq -r '.[]' system/tts_engines/rvc_files.json > /tmp/rvc_files.txt
  xargs -n 1 curl --create-dirs --output-dir models/rvc_base -LO < /tmp/rvc_files.txt
  rm -f /tmp/rvc_files.txt
 EOR
 ##############################################################################
 # Start alltalk:
 ##############################################################################
 ENTRYPOINT ["sh", "-c", "./start_alltalk.sh"]
@@ -0,0 +1,661 @@
                    GNU AFFERO GENERAL PUBLIC LICENSE
                       Version 3, 19 November 2007
 Copyright (C) 2007 Free Software Foundation, Inc. <https://fsf.org/>
 Everyone is permitted to copy and distribute verbatim copies
 of this license document, but changing it is not allowed.
                            Preamble
  The GNU Affero General Public License is a free, copyleft license for
 software and other kinds of works, specifically designed to ensure
 cooperation with the community in the case of network server software.
  The licenses for most software and other practical works are designed
 to take away your freedom to share and change the works.  By contrast,
 our General Public Licenses are intended to guarantee your freedom to
 share and change all versions of a program--to make sure it remains free
 software for all its users.
  When we speak of free software, we are referring to freedom, not
 price.  Our General Public Licenses are designed to make sure that you
 have the freedom to distribute copies of free software (and charge for
 them if you wish), that you receive source code or can get it if you
 want it, that you can change the software or use pieces of it in new
 free programs, and that you know you can do these things.
  Developers that use our General Public Licenses protect your rights
 with two steps: (1) assert copyright on the software, and (2) offer
 you this License which gives you legal permission to copy, distribute
 and/or modify the software.
  A secondary benefit of defending all users' freedom is that
 improvements made in alternate versions of the program, if they
 receive widespread use, become available for other developers to
 incorporate.  Many developers of free software are heartened and
 encouraged by the resulting cooperation.  However, in the case of
 software used on network servers, this result may fail to come about.
 The GNU General Public License permits making a modified version and
 letting the public access it on a server without ever releasing its
 source code to the public.
  The GNU Affero General Public License is designed specifically to
 ensure that, in such cases, the modified source code becomes available
 to the community.  It requires the operator of a network server to
 provide the source code of the modified version running there to the
 users of that server.  Therefore, public use of a modified version, on
 a publicly accessible server, gives the public access to the source
 code of the modified version.
  An older license, called the Affero General Public License and
 published by Affero, was designed to accomplish similar goals.  This is
 a different license, not a version of the Affero GPL, but Affero has
 released a new version of the Affero GPL which permits relicensing under
 this license.
  The precise terms and conditions for copying, distribution and
 modification follow.
                       TERMS AND CONDITIONS
  0. Definitions.
  "This License" refers to version 3 of the GNU Affero General Public License.
  "Copyright" also means copyright-like laws that apply to other kinds of
 works, such as semiconductor masks.
  "The Program" refers to any copyrightable work licensed under this
 License.  Each licensee is addressed as "you".  "Licensees" and
 "recipients" may be individuals or organizations.
  To "modify" a work means to copy from or adapt all or part of the work
 in a fashion requiring copyright permission, other than the making of an
 exact copy.  The resulting work is called a "modified version" of the
 earlier work or a work "based on" the earlier work.
  A "covered work" means either the unmodified Program or a work based
 on the Program.
  To "propagate" a work means to do anything with it that, without
 permission, would make you directly or secondarily liable for
 infringement under applicable copyright law, except executing it on a
 computer or modifying a private copy.  Propagation includes copying,
 distribution (with or without modification), making available to the
 public, and in some countries other activities as well.
  To "convey" a work means any kind of propagation that enables other
 parties to make or receive copies.  Mere interaction with a user through
 a computer network, with no transfer of a copy, is not conveying.
  An interactive user interface displays "Appropriate Legal Notices"
 to the extent that it includes a convenient and prominently visible
 feature that (1) displays an appropriate copyright notice, and (2)
 tells the user that there is no warranty for the work (except to the
 extent that warranties are provided), that licensees may convey the
 work under this License, and how to view a copy of this License.  If
 the interface presents a list of user commands or options, such as a
 menu, a prominent item in the list meets this criterion.
  1. Source Code.
  The "source code" for a work means the preferred form of the work
 for making modifications to it.  "Object code" means any non-source
 form of a work.
  A "Standard Interface" means an interface that either is an official
 standard defined by a recognized standards body, or, in the case of
 interfaces specified for a particular programming language, one that
 is widely used among developers working in that language.
  The "System Libraries" of an executable work include anything, other
 than the work as a whole, that (a) is included in the normal form of
 packaging a Major Component, but which is not part of that Major
 Component, and (b) serves only to enable use of the work with that
 Major Component, or to implement a Standard Interface for which an
 implementation is available to the public in source code form.  A
 "Major Component", in this context, means a major essential component
 (kernel, window system, and so on) of the specific operating system
 (if any) on which the executable work runs, or a compiler used to
 produce the work, or an object code interpreter used to run it.
  The "Corresponding Source" for a work in object code form means all
 the source code needed to generate, install, and (for an executable
 work) run the object code and to modify the work, including scripts to
 control those activities.  However, it does not include the work's
 System Libraries, or general-purpose tools or generally available free
 programs which are used unmodified in performing those activities but
 which are not part of the work.  For example, Corresponding Source
 includes interface definition files associated with source files for
 the work, and the source code for shared libraries and dynamically
 linked subprograms that the work is specifically designed to require,
 such as by intimate data communication or control flow between those
 subprograms and other parts of the work.
  The Corresponding Source need not include anything that users
 can regenerate automatically from other parts of the Corresponding
 Source.
  The Corresponding Source for a work in source code form is that
 same work.
  2. Basic Permissions.
  All rights granted under this License are granted for the term of
 copyright on the Program, and are irrevocable provided the stated
 conditions are met.  This License explicitly affirms your unlimited
 permission to run the unmodified Program.  The output from running a
 covered work is covered by this License only if the output, given its
 content, constitutes a covered work.  This License acknowledges your
 rights of fair use or other equivalent, as provided by copyright law.
  You may make, run and propagate covered works that you do not
 convey, without conditions so long as your license otherwise remains
 in force.  You may convey covered works to others for the sole purpose
 of having them make modifications exclusively for you, or provide you
 with facilities for running those works, provided that you comply with
 the terms of this License in conveying all material for which you do
 not control copyright.  Those thus making or running the covered works
 for you must do so exclusively on your behalf, under your direction
 and control, on terms that prohibit them from making any copies of
 your copyrighted material outside their relationship with you.
  Conveying under any other circumstances is permitted solely under
 the conditions stated below.  Sublicensing is not allowed; section 10
 makes it unnecessary.
  3. Protecting Users' Legal Rights From Anti-Circumvention Law.
  No covered work shall be deemed part of an effective technological
 measure under any applicable law fulfilling obligations under article
 11 of the WIPO copyright treaty adopted on 20 December 1996, or
 similar laws prohibiting or restricting circumvention of such
 measures.
  When you convey a covered work, you waive any legal power to forbid
 circumvention of technological measures to the extent such circumvention
 is effected by exercising rights under this License with respect to
 the covered work, and you disclaim any intention to limit operation or
 modification of the work as a means of enforcing, against the work's
 users, your or third parties' legal rights to forbid circumvention of
 technological measures.
  4. Conveying Verbatim Copies.
  You may convey verbatim copies of the Program's source code as you
 receive it, in any medium, provided that you conspicuously and
 appropriately publish on each copy an appropriate copyright notice;
 keep intact all notices stating that this License and any
 non-permissive terms added in accord with section 7 apply to the code;
 keep intact all notices of the absence of any warranty; and give all
 recipients a copy of this License along with the Program.
  You may charge any price or no price for each copy that you convey,
 and you may offer support or warranty protection for a fee.
  5. Conveying Modified Source Versions.
  You may convey a work based on the Program, or the modifications to
 produce it from the Program, in the form of source code under the
 terms of section 4, provided that you also meet all of these conditions:
    a) The work must carry prominent notices stating that you modified
    it, and giving a relevant date.
    b) The work must carry prominent notices stating that it is
    released under this License and any conditions added under section
    7.  This requirement modifies the requirement in section 4 to
    "keep intact all notices".
    c) You must license the entire work, as a whole, under this
    License to anyone who comes into possession of a copy.  This
    License will therefore apply, along with any applicable section 7
    additional terms, to the whole of the work, and all its parts,
    regardless of how they are packaged.  This License gives no
    permission to license the work in any other way, but it does not
    invalidate such permission if you have separately received it.
    d) If the work has interactive user interfaces, each must display
    Appropriate Legal Notices; however, if the Program has interactive
    interfaces that do not display Appropriate Legal Notices, your
    work need not make them do so.
  A compilation of a covered work with other separate and independent
 works, which are not by their nature extensions of the covered work,
 and which are not combined with it such as to form a larger program,
 in or on a volume of a storage or distribution medium, is called an
 "aggregate" if the compilation and its resulting copyright are not
 used to limit the access or legal rights of the compilation's users
 beyond what the individual works permit.  Inclusion of a covered work
 in an aggregate does not cause this License to apply to the other
 parts of the aggregate.
  6. Conveying Non-Source Forms.
  You may convey a covered work in object code form under the terms
 of sections 4 and 5, provided that you also convey the
 machine-readable Corresponding Source under the terms of this License,
 in one of these ways:
    a) Convey the object code in, or embodied in, a physical product
    (including a physical distribution medium), accompanied by the
    Corresponding Source fixed on a durable physical medium
    customarily used for software interchange.
    b) Convey the object code in, or embodied in, a physical product
    (including a physical distribution medium), accompanied by a
    written offer, valid for at least three years and valid for as
    long as you offer spare parts or customer support for that product
    model, to give anyone who possesses the object code either (1) a
    copy of the Corresponding Source for all the software in the
    product that is covered by this License, on a durable physical
    medium customarily used for software interchange, for a price no
    more than your reasonable cost of physically performing this
    conveying of source, or (2) access to copy the
    Corresponding Source from a network server at no charge.
    c) Convey individual copies of the object code with a copy of the
    written offer to provide the Corresponding Source.  This
    alternative is allowed only occasionally and noncommercially, and
    only if you received the object code with such an offer, in accord
    with subsection 6b.
    d) Convey the object code by offering access from a designated
    place (gratis or for a charge), and offer equivalent access to the
    Corresponding Source in the same way through the same place at no
    further charge.  You need not require recipients to copy the
    Corresponding Source along with the object code.  If the place to
    copy the object code is a network server, the Corresponding Source
    may be on a different server (operated by you or a third party)
    that supports equivalent copying facilities, provided you maintain
    clear directions next to the object code saying where to find the
    Corresponding Source.  Regardless of what server hosts the
    Corresponding Source, you remain obligated to ensure that it is
    available for as long as needed to satisfy these requirements.
    e) Convey the object code using peer-to-peer transmission, provided
    you inform other peers where the object code and Corresponding
    Source of the work are being offered to the general public at no
    charge under subsection 6d.
  A separable portion of the object code, whose source code is excluded
 from the Corresponding Source as a System Library, need not be
 included in conveying the object code work.
  A "User Product" is either (1) a "consumer product", which means any
 tangible personal property which is normally used for personal, family,
 or household purposes, or (2) anything designed or sold for incorporation
 into a dwelling.  In determining whether a product is a consumer product,
 doubtful cases shall be resolved in favor of coverage.  For a particular
 product received by a particular user, "normally used" refers to a
 typical or common use of that class of product, regardless of the status
 of the particular user or of the way in which the particular user
 actually uses, or expects or is expected to use, the product.  A product
 is a consumer product regardless of whether the product has substantial
 commercial, industrial or non-consumer uses, unless such uses represent
 the only significant mode of use of the product.
  "Installation Information" for a User Product means any methods,
 procedures, authorization keys, or other information required to install
 and execute modified versions of a covered work in that User Product from
 a modified version of its Corresponding Source.  The information must
 suffice to ensure that the continued functioning of the modified object
 code is in no case prevented or interfered with solely because
 modification has been made.
  If you convey an object code work under this section in, or with, or
 specifically for use in, a User Product, and the conveying occurs as
 part of a transaction in which the right of possession and use of the
 User Product is transferred to the recipient in perpetuity or for a
 fixed term (regardless of how the transaction is characterized), the
 Corresponding Source conveyed under this section must be accompanied
 by the Installation Information.  But this requirement does not apply
 if neither you nor any third party retains the ability to install
 modified object code on the User Product (for example, the work has
 been installed in ROM).
  The requirement to provide Installation Information does not include a
 requirement to continue to provide support service, warranty, or updates
 for a work that has been modified or installed by the recipient, or for
 the User Product in which it has been modified or installed.  Access to a
 network may be denied when the modification itself materially and
 adversely affects the operation of the network or violates the rules and
 protocols for communication across the network.
  Corresponding Source conveyed, and Installation Information provided,
 in accord with this section must be in a format that is publicly
 documented (and with an implementation available to the public in
 source code form), and must require no special password or key for
 unpacking, reading or copying.
  7. Additional Terms.
  "Additional permissions" are terms that supplement the terms of this
 License by making exceptions from one or more of its conditions.
 Additional permissions that are applicable to the entire Program shall
 be treated as though they were included in this License, to the extent
 that they are valid under applicable law.  If additional permissions
 apply only to part of the Program, that part may be used separately
 under those permissions, but the entire Program remains governed by
 this License without regard to the additional permissions.
  When you convey a copy of a covered work, you may at your option
 remove any additional permissions from that copy, or from any part of
 it.  (Additional permissions may be written to require their own
 removal in certain cases when you modify the work.)  You may place
 additional permissions on material, added by you to a covered work,
 for which you have or can give appropriate copyright permission.
  Notwithstanding any other provision of this License, for material you
 add to a covered work, you may (if authorized by the copyright holders of
 that material) supplement the terms of this License with terms:
    a) Disclaiming warranty or limiting liability differently from the
    terms of sections 15 and 16 of this License; or
    b) Requiring preservation of specified reasonable legal notices or
    author attributions in that material or in the Appropriate Legal
    Notices displayed by works containing it; or
    c) Prohibiting misrepresentation of the origin of that material, or
    requiring that modified versions of such material be marked in
    reasonable ways as different from the original version; or
    d) Limiting the use for publicity purposes of names of licensors or
    authors of the material; or
    e) Declining to grant rights under trademark law for use of some
    trade names, trademarks, or service marks; or
    f) Requiring indemnification of licensors and authors of that
    material by anyone who conveys the material (or modified versions of
    it) with contractual assumptions of liability to the recipient, for
    any liability that these contractual assumptions directly impose on
    those licensors and authors.
  All other non-permissive additional terms are considered "further
 restrictions" within the meaning of section 10.  If the Program as you
 received it, or any part of it, contains a notice stating that it is
 governed by this License along with a term that is a further
 restriction, you may remove that term.  If a license document contains
 a further restriction but permits relicensing or conveying under this
 License, you may add to a covered work material governed by the terms
 of that license document, provided that the further restriction does
 not survive such relicensing or conveying.
  If you add terms to a covered work in accord with this section, you
 must place, in the relevant source files, a statement of the
 additional terms that apply to those files, or a notice indicating
 where to find the applicable terms.
  Additional terms, permissive or non-permissive, may be stated in the
 form of a separately written license, or stated as exceptions;
 the above requirements apply either way.
  8. Termination.
  You may not propagate or modify a covered work except as expressly
 provided under this License.  Any attempt otherwise to propagate or
 modify it is void, and will automatically terminate your rights under
 this License (including any patent licenses granted under the third
 paragraph of section 11).
  However, if you cease all violation of this License, then your
 license from a particular copyright holder is reinstated (a)
 provisionally, unless and until the copyright holder explicitly and
 finally terminates your license, and (b) permanently, if the copyright
 holder fails to notify you of the violation by some reasonable means
 prior to 60 days after the cessation.
  Moreover, your license from a particular copyright holder is
 reinstated permanently if the copyright holder notifies you of the
 violation by some reasonable means, this is the first time you have
 received notice of violation of this License (for any work) from that
 copyright holder, and you cure the violation prior to 30 days after
 your receipt of the notice.
  Termination of your rights under this section does not terminate the
 licenses of parties who have received copies or rights from you under
 this License.  If your rights have been terminated and not permanently
 reinstated, you do not qualify to receive new licenses for the same
 material under section 10.
  9. Acceptance Not Required for Having Copies.
  You are not required to accept this License in order to receive or
 run a copy of the Program.  Ancillary propagation of a covered work
 occurring solely as a consequence of using peer-to-peer transmission
 to receive a copy likewise does not require acceptance.  However,
 nothing other than this License grants you permission to propagate or
 modify any covered work.  These actions infringe copyright if you do
 not accept this License.  Therefore, by modifying or propagating a
 covered work, you indicate your acceptance of this License to do so.
  10. Automatic Licensing of Downstream Recipients.
  Each time you convey a covered work, the recipient automatically
 receives a license from the original licensors, to run, modify and
 propagate that work, subject to this License.  You are not responsible
 for enforcing compliance by third parties with this License.
  An "entity transaction" is a transaction transferring control of an
 organization, or substantially all assets of one, or subdividing an
 organization, or merging organizations.  If propagation of a covered
 work results from an entity transaction, each party to that
 transaction who receives a copy of the work also receives whatever
 licenses to the work the party's predecessor in interest had or could
 give under the previous paragraph, plus a right to possession of the
 Corresponding Source of the work from the predecessor in interest, if
 the predecessor has it or can get it with reasonable efforts.
  You may not impose any further restrictions on the exercise of the
 rights granted or affirmed under this License.  For example, you may
 not impose a license fee, royalty, or other charge for exercise of
 rights granted under this License, and you may not initiate litigation
 (including a cross-claim or counterclaim in a lawsuit) alleging that
 any patent claim is infringed by making, using, selling, offering for
 sale, or importing the Program or any portion of it.
  11. Patents.
  A "contributor" is a copyright holder who authorizes use under this
 License of the Program or a work on which the Program is based.  The
 work thus licensed is called the contributor's "contributor version".
  A contributor's "essential patent claims" are all patent claims
 owned or controlled by the contributor, whether already acquired or
 hereafter acquired, that would be infringed by some manner, permitted
 by this License, of making, using, or selling its contributor version,
 but do not include claims that would be infringed only as a
 consequence of further modification of the contributor version.  For
 purposes of this definition, "control" includes the right to grant
 patent sublicenses in a manner consistent with the requirements of
 this License.
  Each contributor grants you a non-exclusive, worldwide, royalty-free
 patent license under the contributor's essential patent claims, to
 make, use, sell, offer for sale, import and otherwise run, modify and
 propagate the contents of its contributor version.
  In the following three paragraphs, a "patent license" is any express
 agreement or commitment, however denominated, not to enforce a patent
 (such as an express permission to practice a patent or covenant not to
 sue for patent infringement).  To "grant" such a patent license to a
 party means to make such an agreement or commitment not to enforce a
 patent against the party.
  If you convey a covered work, knowingly relying on a patent license,
 and the Corresponding Source of the work is not available for anyone
 to copy, free of charge and under the terms of this License, through a
 publicly available network server or other readily accessible means,
 then you must either (1) cause the Corresponding Source to be so
 available, or (2) arrange to deprive yourself of the benefit of the
 patent license for this particular work, or (3) arrange, in a manner
 consistent with the requirements of this License, to extend the patent
 license to downstream recipients.  "Knowingly relying" means you have
 actual knowledge that, but for the patent license, your conveying the
 covered work in a country, or your recipient's use of the covered work
 in a country, would infringe one or more identifiable patents in that
 country that you have reason to believe are valid.
  If, pursuant to or in connection with a single transaction or
 arrangement, you convey, or propagate by procuring conveyance of, a
 covered work, and grant a patent license to some of the parties
 receiving the covered work authorizing them to use, propagate, modify
 or convey a specific copy of the covered work, then the patent license
 you grant is automatically extended to all recipients of the covered
 work and works based on it.
  A patent license is "discriminatory" if it does not include within
 the scope of its coverage, prohibits the exercise of, or is
 conditioned on the non-exercise of one or more of the rights that are
 specifically granted under this License.  You may not convey a covered
 work if you are a party to an arrangement with a third party that is
 in the business of distributing software, under which you make payment
 to the third party based on the extent of your activity of conveying
 the work, and under which the third party grants, to any of the
 parties who would receive the covered work from you, a discriminatory
 patent license (a) in connection with copies of the covered work
 conveyed by you (or copies made from those copies), or (b) primarily
 for and in connection with specific products or compilations that
 contain the covered work, unless you entered into that arrangement,
 or that patent license was granted, prior to 28 March 2007.
  Nothing in this License shall be construed as excluding or limiting
 any implied license or other defenses to infringement that may
 otherwise be available to you under applicable patent law.
  12. No Surrender of Others' Freedom.
  If conditions are imposed on you (whether by court order, agreement or
 otherwise) that contradict the conditions of this License, they do not
 excuse you from the conditions of this License.  If you cannot convey a
 covered work so as to satisfy simultaneously your obligations under this
 License and any other pertinent obligations, then as a consequence you may
 not convey it at all.  For example, if you agree to terms that obligate you
 to collect a royalty for further conveying from those to whom you convey
 the Program, the only way you could satisfy both those terms and this
 License would be to refrain entirely from conveying the Program.
  13. Remote Network Interaction; Use with the GNU General Public License.
  Notwithstanding any other provision of this License, if you modify the
 Program, your modified version must prominently offer all users
 interacting with it remotely through a computer network (if your version
 supports such interaction) an opportunity to receive the Corresponding
 Source of your version by providing access to the Corresponding Source
 from a network server at no charge, through some standard or customary
 means of facilitating copying of software.  This Corresponding Source
 shall include the Corresponding Source for any work covered by version 3
 of the GNU General Public License that is incorporated pursuant to the
 following paragraph.
  Notwithstanding any other provision of this License, you have
 permission to link or combine any covered work with a work licensed
 under version 3 of the GNU General Public License into a single
 combined work, and to convey the resulting work.  The terms of this
 License will continue to apply to the part which is the covered work,
 but the work with which it is combined will remain governed by version
 3 of the GNU General Public License.
  14. Revised Versions of this License.
  The Free Software Foundation may publish revised and/or new versions of
 the GNU Affero General Public License from time to time.  Such new versions
 will be similar in spirit to the present version, but may differ in detail to
 address new problems or concerns.
  Each version is given a distinguishing version number.  If the
 Program specifies that a certain numbered version of the GNU Affero General
 Public License "or any later version" applies to it, you have the
 option of following the terms and conditions either of that numbered
 version or of any later version published by the Free Software
 Foundation.  If the Program does not specify a version number of the
 GNU Affero General Public License, you may choose any version ever published
 by the Free Software Foundation.
  If the Program specifies that a proxy can decide which future
 versions of the GNU Affero General Public License can be used, that proxy's
 public statement of acceptance of a version permanently authorizes you
 to choose that version for the Program.
  Later license versions may give you additional or different
 permissions.  However, no additional obligations are imposed on any
 author or copyright holder as a result of your choosing to follow a
 later version.
  15. Disclaimer of Warranty.
  THERE IS NO WARRANTY FOR THE PROGRAM, TO THE EXTENT PERMITTED BY
 APPLICABLE LAW.  EXCEPT WHEN OTHERWISE STATED IN WRITING THE COPYRIGHT
 HOLDERS AND/OR OTHER PARTIES PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY
 OF ANY KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO,
 THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
 PURPOSE.  THE ENTIRE RISK AS TO THE QUALITY AND PERFORMANCE OF THE PROGRAM
 IS WITH YOU.  SHOULD THE PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF
 ALL NECESSARY SERVICING, REPAIR OR CORRECTION.
  16. Limitation of Liability.
  IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING
 WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MODIFIES AND/OR CONVEYS
 THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES, INCLUDING ANY
 GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING OUT OF THE
 USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED TO LOSS OF
 DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY YOU OR THIRD
 PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER PROGRAMS),
 EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE POSSIBILITY OF
 SUCH DAMAGES.
  17. Interpretation of Sections 15 and 16.
  If the disclaimer of warranty and limitation of liability provided
 above cannot be given local legal effect according to their terms,
 reviewing courts shall apply local law that most closely approximates
 an absolute waiver of all civil liability in connection with the
 Program, unless a warranty or assumption of liability accompanies a
 copy of the Program in return for a fee.
                     END OF TERMS AND CONDITIONS
            How to Apply These Terms to Your New Programs
  If you develop a new program, and you want it to be of the greatest
 possible use to the public, the best way to achieve this is to make it
 free software which everyone can redistribute and change under these terms.
  To do so, attach the following notices to the program.  It is safest
 to attach them to the start of each source file to most effectively
 state the exclusion of warranty; and each file should have at least
 the "copyright" line and a pointer to where the full notice is found.
    <one line to give the program's name and a brief idea of what it does.>
    Copyright (C) <year>  <name of author>
    This program is free software: you can redistribute it and/or modify
    it under the terms of the GNU Affero General Public License as published
    by the Free Software Foundation, either version 3 of the License, or
    (at your option) any later version.
    This program is distributed in the hope that it will be useful,
    but WITHOUT ANY WARRANTY; without even the implied warranty of
    MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
    GNU Affero General Public License for more details.
    You should have received a copy of the GNU Affero General Public License
    along with this program.  If not, see <https://www.gnu.org/licenses/>.
 Also add information on how to contact you by electronic and paper mail.
  If your software can interact with users remotely through a computer
 network, you should also make sure that it provides a way for users to
 get its source.  For example, if your program is a web application, its
 interface could display a "Source" link that leads users to an archive
 of the code.  There are many ways you could offer source, and different
 solutions will be better for different programs; see section 13 for the
 specific requirements.
  You should also get your employer (if you work as a programmer) or school,
 if any, to sign a "copyright disclaimer" for the program, if necessary.
 For more information on this, and how to apply and follow the GNU AGPL, see
 <https://www.gnu.org/licenses/>.
@@ -1,312 +1,170 @@
-# AllTalk TTS
+# AllTalk TTS v2
-AllTalk is an updated version of the Coqui_tts extension for Text Generation web UI. Features include:
+**Documentation/Wiki:** Refer to the built-in documentation or view the extensive [Wiki here](https://github.com/erew123/alltalk_tts/wiki).
- **Custom Start-up Settings:** Adjust your default start-up settings. [Screenshot](https://github.com/erew123/alltalk_tts#screenshots)
+**Known Errors:** See the [Known Errors page here](https://github.com/erew123/alltalk_tts/wiki/Error-Messages-List) for troubleshooting.
 - **Narrarator:** Use different voices for main character and narration. [Example Narration](https://vocaroo.com/18fYWVxiQpk1)
 - **Low VRAM mode:** Improve generation performance if your VRAM is filled by your LLM. [Screenshot](https://github.com/erew123/alltalk_tts#screenshots)
 - **DeepSpeed:** A 3-4x performance boost generating TTS. [DeepSpeed Windows/Linux Instructions](https://github.com/erew123/alltalk_tts?tab=readme-ov-file#deepspeed-installation-options) [Screenshot](https://github.com/erew123/alltalk_tts/assets/35898566/548619c8-5f1b-47d0-a73d-54d2fee3f3db)
 - **Local/Custom models:** Use any of the XTTSv2 models (API Local and XTTSv2 Local).
 - **Optional wav file maintenance:** Configurable deletion of old output wav files. [Screenshot](https://github.com/erew123/alltalk_tts#screenshots)
 - **Documentation:** Fully documented with a built in webpage. [Screenshot](https://github.com/erew123/alltalk_tts#screenshots)
 - **Console output** Clear command line output for any warnings or issues.
 - **Standalone/3rd Party support via JSON calls** Can be used with 3rd party applications via JSON calls.
-## Index
+**Issues & Support:** Report issues or bugs by opening a [support ticket here](https://github.com/erew123/alltalk_tts/issues).
- 🟩 [Installation](https://github.com/erew123/alltalk_tts?#-installation-on-text-generation-web-ui)
+**Feature Requests:** View current feature requests on the [Feature Requests page here](https://github.com/erew123/alltalk_tts/discussions/74).
 - 🟪 [Updating & problems with updating](https://github.com/erew123/alltalk_tts?#-updating)
 - 🟫 [Screenshots](https://github.com/erew123/alltalk_tts#-screenshots)
 - 🟨 [Help with problems](https://github.com/erew123/alltalk_tts?#-help-with-problems)
 - 🔵🟢🟡 [DeepSpeed Installation (Windows & Linux)](https://github.com/erew123/alltalk_tts?#-deepspeed-installation-options)
 - 🔴 [Future to-do list & Upcoming updates](https://github.com/erew123/alltalk_tts?#-future-to-do-list)
-#### Updates
+**Community Discussions:** Join discussions on AllTalk V2 in the [Discussion Board here](https://github.com/erew123/alltalk_tts/discussions/245). 
 The latest build (13 Dec 2023) has had the entire text filtering engine and narration engine rebuilt from scratch. It's highly complicated how its actually working, but the end result it a much clearer TTS output and much better control over the narrator option and how to handle text that isnt within quotes or asterisks. Its a highly recommened update, for the improved quality it gives to the TTS output, if nothing else.
-Should you want the older version of the narrator engine+text filtering, I will leave this older copy [here](https://github.com/erew123/alltalk_tts/releases/tag/v1-old-narrator)
+**Please note, my available time has become VERY limited due to unexpected family commitments [Please read here for details](https://github.com/erew123/alltalk_tts/issues/377). So please DYOR, look at the [Wiki here ](https://github.com/erew123/alltalk_tts/wiki) and refer to the TTS manufacturer (Links in the Gradio interface) for issues specific to THEIR TTS engines.**
-DeepSpeed **v11.2** can be installed within the **default text-generation-webui Python 3.11 environment**. Instructions [here](https://github.com/erew123/alltalk_tts#deepspeed-112-for-windows--python-311) (or scroll down). Please note, this is **not** an official Microsoft method (currently) so they will not support you with this style installation. Officially, only DeepSpeed v8.3 is installing on Python 3.9.x.
+I would also like to say a big thank you to anyone whom has contributed to this project either with PR's (list [here](https://github.com/erew123/alltalk_tts/graphs/contributors)) or in any discussion forums/issues tickets. Your input, guidance, help and thoughts are greatly appreciated.
-#### The one thing I cant easily work around
+### [💖 Sponsor this Project on Ko-fi](https://ko-fi.com/erew123)
 With a RP chat with your AI, **on your character card** `parameters menu` > `character tab` > `greeting` make sure that anything in there that is the **narrator is in asterisks** and anything **spoken is in double quotes**, then hit the `save` (disk) button. Greeting paragraphs/sentences are handled differently from how the AI sends text and so its difficut to account for them both.
-I could force a delimeter in at this stage, but I know it would/may affect things further down the line in the chat and I need a good think about that before just making a change. This issue **only** affects the greeting card/start of conversation and the "example" card that comes with text-generation-webui suffers this issue (if you want to try it for yourself). So you would put double quotes around like this (from the example card):
+---
-`"`Hey! I'm so excited to finally meet you. I've heard so many great things about you and I'm eager to pick your brain about computers. I'm sure you have a wealth of knowledge that I can learn from.`"`
+## AllTalk V2 Core Functionality
 - Comprehensive setup utilities for Windows & Linux
 - Multiple TTS engine support
    - Coqui XTTS TTS
    - Coqui VITS TTS
    - Piper TTS
    - Parler TTS
    - F5 TTS
    - Other TTS engines can be coded in
 - Retrieval-based Voice Conversion (RVC) pipeline
 - Easy integration of new TTS engines (some coding required)
 - Customizable settings for each TTS engine
 - In-app model downloads for each engine
 - Gradio web interface for easy management
 - Standalone application or integration with Text-generation-webui, SillyTavern, KoboldCPP, HomeAssistant
 - Narrator function for using different voices for characters and narration
 - Audio Transcoding to multiple formats (mp3, opus, etc.)
 - About 50 Gradio interface themes
 - Custom start-up settings
 - Clear console output for troubleshooting
 - Binds to all available IP addresses (0.0.0.0)
 - Fully customizable Global API settings
 - Transcription & Audio File Transcription
-## 🟩 Installation on Text generation web UI
+#### Documentation and Support
-This has been tested on the current Dec 2023 release of Text generation webUI. If you have not updated it for a while, you may wish to update Text generation webUI, [instructions here](https://github.com/oobabooga/text-generation-webui?tab=readme-ov-file#how-to-install)
+- Built-in documentation with web interface
 - GitHub Wiki documentation
-1) In a command prompt/terminal window you need to move into your Text generation webUI folder:
+#### Performance and Optimization (Depending on TTS engine used)
 - DeepSpeed integration for 2-3x performance boost (Depending on TTS engine)
 - Low VRAM mode for systems with limited GPU memory 
-`cd text-generation-webui`
+#### Voice Customization and Enhancement
 - Model Finetuning for improved voice reproduction
 - XTTS Multiple audio sample TTS generation for better voice reproduction
-2) Start the Text generation webUI Python environment for your OS:
+#### Bulk Operations and Management
 - Bulk TTS Generator/Editor for large-scale audio production
-`cmd_windows.bat`, `./cmd_linux.sh`, `cmd_macos.sh` or `cmd_wsl.bat`
+#### API and Integration
 - Comprehensive API Suite
 - OpenAI-compatible endpoint/API for broader software compatibility
 - JSON call support for third-party applications
-3) Move into your extensions folder:
+#### Experimental Features
 - Multi Engine Manager (MEM) for running multiple TTS instances simultaneously & queuing requests between them.
-`cd extensions`
+#### Screenshots
 Screenshots are available [here](https://github.com/erew123/alltalk_tts/discussions/237)
-4) Once there git clone this repository:
+---
-`git clone https://github.com/erew123/alltalk_tts`
+### 🟥 Platform-Specific Notes
-5) Move into the **alltalk_tts** folder:
+#### Windows
 - Requires Git, Microsoft C++ Build Tools, and Windows SDK.
 - DeepSpeed works with NVIDIA GPUs.
-`cd alltalk_tts`
+#### Linux
 - Requires specific packages based on your distribution (Debian-based or RPM-based).
 - DeepSpeed works with NVIDIA GPUs.
 - Limited experimental support for AMD GPUs (mainly for XTTS).
-6) Install the requirements:
+#### Mac (Theoretical)
 - Installation process is untested and theoretical.
 - No GPU acceleration for AllTalk TTS engines.
 - Some TTS engines may have limited or no support on Mac.
-*Nvidia graphics card machines* - `pip install -r requirements_nvidia.txt`
+For more details on Mac support limitations, please refer to the [Mac Support Disclaimer](https://github.com/erew123/alltalk_tts/wiki/Install-%E2%80%90-Manual-Installation-Guide).
-*Other machines (mac, amd etc)* - `pip install -r requirements_other.txt`
+#### GPU Support
 GPU support is provided by the developer of the individual TTS engine. If their TTS engine support's X GPU, then I can support X GPU, if it doesnt support X GPU, then I cannot support X GPU. Most of the engines will run on CPU, but some may be very slow on CPU.
-7) **(Optional DeepSpeed)** If you have an Nvidia Graphics card on a system running Linux or Windows and wish to use **DeepSpeed** please follow these instructions [here](https://github.com/erew123/alltalk_tts?#-deepspeed-installation-options). Though you may wish to check things are generally working with the below steps before installing DeepSpeed.
+- NVIDIA GPUs: Full support on Windows and Linux.
 - AMD GPUs: Limited experimental support on Linux (mainly for XTTS).
 - Intel ARC GPUs: No specific support currently.
 - Apple Silicon (M1/M2): No GPU acceleration for AllTalk TTS engines currently.
-8) You can now start move back to the main Text generation webUI folder `cd ..` (a few times), start Text generation webUI (`start_windows.bat`,`./start_linux.sh`, `start_macos.sh` or `start_wsl.bat`)  and load the AllTalk extension in the Text generation webUI **session** tab.
+---
-9) Please read the note below about start-up times and also the note above about ensuring your character cards are set up [correctly](https://github.com/erew123/alltalk_tts#the-one-thing-i-cant-easily-work-around)
+### 🟩 Quick Setup (Recommended for most users)
-10) Some extra voices downloadable [here](https://drive.google.com/file/d/1bYdZdr3L69kmzUN3vSiqZmLRD7-A3M47/view?usp=drive_link)
+For a fast and straightforward installation and recommended:
-**Note: It can take a while to start up.** Check the command prompt/terminal window if you want to know what its doing. After it says "Model Loaded" the Text generation webUI is usually available on its IP address a few seconds later, for you to connect to in your browser.
+- [Standalone Installation](https://github.com/erew123/alltalk_tts/wiki/Install-%E2%80%90-Standalone-Installation)
  - For users who want to run AllTalk TTS as a standalone application.
  - Available for Windows and Linux.
  - Most users will want a Standalone Installation and should use this choice.
-**Documentation:** Click on the link when inside Text generation webUI as shown in the screenshot [here](https://github.com/erew123/alltalk_tts#screenshots)
+- [Text-generation-webui Installation](https://github.com/erew123/alltalk_tts/wiki/Install-%E2%80%90-Text%E2%80%90generation%E2%80%90webui-Installation)
  - For users who want to integrate AllTalk TTS as a part of [Text-generation-webui](https://github.com/oobabooga/text-generation-webui).
  - Available for Windows and Linux.
  - Includes information on the optional/alternative TGWUI Remote Extension.
  - **CURRENTLY YOU SHOULD USE THE STANDALONE INSTALL METHOD AND TGWUI REMOTE EXTENSION**
-**Where to find voices** https://aiartes.com/voiceai or https://commons.wikimedia.org/ or interviews on youtube etc. Instructions on how to cut down and prepare a voice sample are within the built in documentation.
+These methods use scripts that automate most of the installation process, making it easier for users to get started quickly.
-### 🟩 Other installation notes
+### 🟩 Manual Installation
 On first startup, AllTalk will download the Coqui XTTSv2 2.0.2 model to its **models** folder (1.8GB space required). You can customse your model or use the TTS latest model within the interface (details in documentation).
-Once the extension is loaded, please find all documentation and settings on the link provided in the interface (as shown in the screenshot below).
+For users who prefer more control over the installation process or need to troubleshoot:
-To start AllTalk every Text generation webUI loads, edit the Text generation webUI `CMD_FLAGS.txt` file in the main `text-generation-webui` folder and add `--extensions alltalk_tts`.
+- [Manual Installation Guide](https://github.com/erew123/alltalk_tts/wiki/Install-%E2%80%90-Manual-Installation-Guide)
  - Detailed step-by-step instructions for manual installation.
  - Available for Windows, Linux, and Mac (theoretical).
  - Recommended for advanced users or those with specific setup requirements.
 ### 🟩 Google Colab Installation
 For users who want to run AllTalk TTS in a cloud environment:
 - [Google Colab Installation](https://github.com/erew123/alltalk_tts/wiki/Google-COLAB)
  - Instructions for setting up and running AllTalk TTS in Google Colab.
  - Ideal for users who want to try AllTalk TTS without installing it locally.
 ---
 ### 🟪 Updating
 This is pretty much a repeat of the installation process. 
-1) In a command prompt/terminal window you need to move into your Text generation webUI folder:
+As long as you did the `git clone` method to setup initially, you will be able to go into the folder and use `git pull` to download updates.
-`cd text-generation-webui`
+It is recommended you use `atsetup` to re-apply the settings file for any necessary package updates.
-2) Move into your extensions and alltalk_tts folder:
+---
-`cd extensions` then `cd alltalk_tts`
+### 🟨 Diagnostics Help with Issues/Start-up problems etc.
-3) At the command prompt/terminal, type:
+If you are having issues with starting AllTalk, it may well be because some of the 3rd Party packages versions have changed, or something is not right in your Python environment. 
-`git pull`
+- **Known Errors List:** [Wiki page is here](https://github.com/erew123/alltalk_tts/wiki/Error-Messages-List)
 - **Diagnostics tool Instructions:** [Wiki page is here](https://github.com/erew123/alltalk_tts/wiki/AllTalk-Diagnostics-Tool)
-4) Install the requirements:
+Whilst its impossible to constantly ensure that everything is going to work perfectly, after installation, you can use the diagnostics tool to:
-*Nvidia graphics card machines* - `pip install -r requirements_nvidia.txt`
+1) Generate a `diagnostics.log` file which contains information about your Python environment setup and performs various checks to ensure everything is installed.
 2) Identify possible issues by comparing **your** `diagnostics.log` file to the **AllTalk base** `basediagnostics.log` stored in the `alltalk_tts/system/config/` folder.
 3) Provide some semi-automated repair of your Python environment.
-*Other machines (mac, amd etc)* - `pip install -r requirements_other.txt`
+---
-#### 🟪 **git pull error** 
+### 🆘 Support Requests, Troubleshooting, BETA Discussions & Feature requests
-I did leave a mistake in the `/extensions/alltalk_tts/.gitignore` file at one point. If your `git pull` doesnt work, you can either follow the Problems Updating section below, or edit the `.gitignore` file and **replace its entire contents** with the below, save the file, then re-try the `git pull`
+**Documentation & Known Error codes**, please refer to the built in documentation or the [Wiki here ](https://github.com/erew123/alltalk_tts/wiki).
 ```
 voices/*.*
 models/*.*
 outputs/*.*
 config.json
 models.json
 ```
 ### 🟪 Problems Updating
-If you do experience any problems, the simplest method to resolve this will be:
+If you wish to code something yourself though, thats perfectly to do and youre welcome to discuss that with me if needed.
-1) re-name the existing `alltalk_tts` folder to something like `alltalk_tts.old`
+General discussions on the BETA should be [here in the discussion board](https://github.com/erew123/alltalk_tts/discussions/245)
-2) Start a console/terminal then:
+If you have a specifc technical problem, please open an issue ticket [here in the issues area](https://github.com/erew123/alltalk_tts/issues).
-`cd text-generation-webui` and start your python environment `cmd_windows.bat`, `./cmd_linux.sh`, `cmd_macos.sh` or `cmd_wsl.bat`
+**Please note, my available time has become VERY limited due to unexpected family commitments. So please DYOR, look at the [Wiki here ](https://github.com/erew123/alltalk_tts/wiki) and refer to the TTS manufacturer (Links in the Gradio interface) for issues specific to THEIR TTS engines.**
-3) Move into the extensions folder, same as if you were doing a fresh installation:
+---
 `cd extensions` then 
 `git clone https://github.com/erew123/alltalk_tts`
 This will download a fresh installation. 
 3) Move into the **alltalk_tts** folder:
 `cd alltalk_tts`
 4) Install the requirements:
 *Nvidia graphics card machines* - `pip install -r requirements_nvidia.txt`
 *Other machines (mac, amd etc)* - `pip install -r requirements_other.txt`
 5) Before starting it up, copy/merge the `models`, `voices` and `outputs` folders over from the `alltalk_tts.old` folder to the newly created `alltalk_tts` folder. This will keep your voices history and also stop it re-downloading the model again.
 You can now start text-generation-webui or AllTalk (standalone) and it should start up fine. You will need to re-set any saved configuration changes on the configuration page. 
 Assuming its all working fine and you are happy, you can delete the old alltalk_tts.old folder.
 ## 🟫 Screenshots
 |![image](https://github.com/erew123/alltalk_tts/assets/35898566/4ca9b4c7-60ce-4ac6-82e5-fd1989b84644) | ![image](https://github.com/erew123/alltalk_tts/assets/35898566/b0e13dba-c6b1-4ab7-845d-244ac1158330) |
 |:---:|:---:|
 |![image](https://github.com/erew123/alltalk_tts/assets/35898566/548619c8-5f1b-47d0-a73d-54d2fee3f3db) | ![image](https://github.com/erew123/alltalk_tts/assets/35898566/e35e987c-543a-486b-b4fb-ee6ebe6f59c6) |
 ## 🟨 Help with problems
 #### 🟨 I activated DeepSpeed in the settings page, but I didnt install DeepSpeed yet and now I have issues starting up
 You can either follow the [Problems Updating](https://github.com/erew123/alltalk_tts#-problems-updating) and fresh install your config. Or you can edit the `config.json` file within the `alltalk_tts` folder. You would look for '"deepspeed_activate": true,' and change the word true to false `"deepspeed_activate": false,' ,then save the file and try starting again.
 If you want to use DeepSpeed, you need an Nvidia Graphics card and to install DeepSpeed on your system. Instructions are [here](https://github.com/erew123/alltalk_tts?tab=readme-ov-file#deepspeed-installation-options).
 #### 🟨 I am having problems getting AllTalk to start after changing settings or making a custom setup/model setup.
 I would suggest following [Problems Updating](https://github.com/erew123/alltalk_tts#-problems-updating) and if you still have issues after that, you can raise an issue [here](https://github.com/erew123/alltalk_tts/issues)
 #### 🟨 I see some red "asyncio" messages
 As far as I am aware, these are to do with the chrome browser the gradio text-generation-webui in some way. I raised an issue about this on the text-generation-webui [here](https://github.com/oobabooga/text-generation-webui/issues/4788) where you can see that AllTalk is not loaded and the messages persist. Either way, this is more a warning than an actual issue, so shouldnt affect any functionality of either AllTalk or text-generation-webui, they are more just an annoyance.
 #### 🟨 I am having problems updating/some other issue where it wont start up/Im sure this is a bug
 Please see [Problems Updating](https://github.com/erew123/alltalk_tts#-problems-updating). If that doesnt help you can raise an issue [here](https://github.com/erew123/alltalk_tts/issues). It would be handy to have any log files from the console where your error is being shown. I can only losely support custom built Python environments and give general pointers.
 Also, is your text-generation-webui up to date? [instructions here](https://github.com/oobabooga/text-generation-webui?tab=readme-ov-file#how-to-install)
 ## 🔵🟢🟡 DeepSpeed Installation Options
 ### 🔵 For Linux
 ➡️DeepSpeed requires an Nvidia Graphics card!⬅️
 Covered in the online/buit-in documentation, but a nice easy install.
 ### 🟢🟡 For Windows & Python 3.11
 DeepSpeed v11.1 and v11.2 will work on the current text-generation-webui Python 3.11 environment! You have 2x options for how to setup DeepSpeed on Windows. A quick way (🟢Option 1) and a long way (🟡Option 2).
 Thanks to [@S95Sedan](https://github.com/S95Sedan) - They managed to get DeepSpeed 11.2 working on Windows via making some edits to the original Microsoft DeepSpeed v11.2 installation. The original post is [here](https://github.com/oobabooga/text-generation-webui/issues/4734#issuecomment-1843984142).
 #### 🟢 OPTION 1 - Quick and easy!
 #### Pre-Compiled Wheel Deepspeed v11.1 (for Windows and Python 3.11) 
 ➡️DeepSpeed requires an Nvidia Graphics card!⬅️
 [@S95Sedan](https://github.com/S95Sedan) has kindly provided a pre-compiled DeepSpeed v11.1 wheel file, which you can download and use. To use this, you will need to:
 **Note:** In my tests, with this method you will **not** need to install the Nvidia CUDA toolkit to make this work, but AllTalk may warn you when starting DeepSpeed that it doesnt see the CUDA Toolkit, however, it works fine for TTS purposes.
 1) Download the file [deepspeed-0.11.1+e9503fe-cp311-cp311-win_amd64.whl](https://drive.google.com/file/d/1PFsf6uSPY5Cb4o9VxiZ7DLv-j35L7Y41/view?usp=sharing) by going to the link and clicking the **download** icon at the top right of the screen and save the file it inside your **text-generation-webui** folder.
 2) Open a command prompt window, move into your **text-generation-webui folder**, you can now start the Python environment for text-generation-webui:
 `cmd_windows.bat`
 3) With the file that you saved in the **text-generation-webui folder** you now type the following:
 `pip install "deepspeed-0.11.1+e9503fe-cp311-cp311-win_amd64.whl"`
 4) This should install through cleanly and you should now have DeepSpeed v11.1 installed within the Python 3.11 environment of text-generation-webui.
 5) When you start up text-generation-webui, and AllTalk starts, you should see **[AllTalk Startup] DeepSpeed Detected**
 6) Within AllTalk, you will now have a checkbox for **Activate DeepSpeed** though remember you can only change **1x setting every 15 or so seconds**, so dont try to activate DeepSpeed **and** LowVRAM/Change your model simultantiously. Do one of those, wait 15-20 seconds until the change is confirmed in the terminal/command prompt, then you can change the other. When you are happy it works, you can set the default start-up settings in the settings page.
 #### 🟡 OPTION 2 - A bit more complicated!
 #### Manual Build DeepSpeed v11.2 (for Windows and Python 3.11) 
 ➡️DeepSpeed requires an Nvidia Graphics card!⬅️
 DeepSpeed Version 11.2 with CUDA 12.1 - Installation Instructions:
 1. Download the 11.2 release of [DeepSpeed](https://github.com/microsoft/DeepSpeed/releases/tag/v0.11.2) extract it to a folder. 
 2. Install Visual C++ build tools, such as [VS2019 C++ x64/x86](https://learn.microsoft.com/en-us/visualstudio/releases/2019/redistribution#vs2019-download) build tools.
 3. Download and install the [Nvidia Cuda Toolkit 12.1](https://developer.nvidia.com/cuda-12-1-0-download-archive)
 4. Edit your Windows environment variables to ensure that CUDA_HOME and CUDA_PATH are set to your Nvidia Cuda Toolkit path. (The folder above the bin folder that nvcc.exe is installed in). Examples are:<br>
 ```set CUDA_HOME=C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v12.1```<br>
 ```set CUDA_PATH=C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v12.1```<br>
 5. OPTIONAL If you do not have an python environment already created, you can install [Miniconda](https://docs.conda.io/projects/miniconda/en/latest/miniconda-install.html), then at a command prompt, create and activate your environment with:<br>
 ```conda create -n pythonenv python=3.11```<br>
 ```activate pythonenv```<br>
 6. Launch the Command Prompt cmd with Administrator privilege as it requires admin to allow creating symlink folders.
 7. Install PyTorch, 2.1.0 with CUDA 12.1 into your Python 3.11 environment e.g:<br>
 ```activate pythonenv``` (activate your python environment)<br>
 ```conda install pytorch==2.1.0 torchvision==0.16.0 torchaudio==2.1.0 pytorch-cuda=12.1 -c pytorch -c nvidia```
 8. In your python environment check that your CUDA_HOME and CUDA_PATH are still pointing to the correct location.<br>
 ```set``` (to list and check the windows environment variables. Refer to step 4 if not)
 9. Navigate to your deepspeed folder in the Command Prompt:<br>
 ```cd c:\deepspeed``` (wherever you extracted it to)
 10. Modify the following files:<br>
 **(These modified files are included in the git-pull of AllTalk, but if you want to modify them yourself, please follow the below)**
 deepspeed-0.11.2/build_win.bat** - at the top of the file, add:<br>
 ```set DS_BUILD_EVOFORMER_ATTN=0```
 deepspeed-0.11.2/csrc/quantization/pt_binding.cpp - lines 244-250 - change to:
 ```
    std::vector<int64_t> sz_vector(input_vals.sizes().begin(), input_vals.sizes().end());
    sz_vector[sz_vector.size() - 1] = sz_vector.back() / devices_per_node;  // num of GPU per nodes
    at::IntArrayRef sz(sz_vector);
    auto output = torch::empty(sz, output_options);
    const int elems_per_in_tensor = at::numel(input_vals) / devices_per_node;
    const int elems_per_in_group = elems_per_in_tensor / (in_groups / devices_per_node);
    const int elems_per_out_group = elems_per_in_tensor / out_groups;
 ```
 deepspeed-0.11.2/csrc/transformer/inference/csrc/pt_binding.cpp
 lines 541-542 - change to:
 ```
 									 {static_cast<unsigned>(hidden_dim * InferenceContext::Instance().GetMaxTokenLength()),
 									  static_cast<unsigned>(k * InferenceContext::Instance().GetMaxTokenLength()),
 ```
 lines 550-551 - change to:
 ```
 						 {static_cast<unsigned>(hidden_dim * InferenceContext::Instance().GetMaxTokenLength()),
 						  static_cast<unsigned>(k * InferenceContext::Instance().GetMaxTokenLength()),
 ```
 line 1581 - change to:
 ```
 		at::from_blob(intermediate_ptr, {input.size(0), input.size(1), static_cast<int64_t>(mlp_1_out_neurons)}, options);
 ```
 deepspeed-0.11.2/deepspeed/env_report.py
 line 10 - add:
 ```
 import psutil
 ```
 line 83 - 100 - change to:
 ```
 def get_shm_size():
    try:
        temp_dir = os.getenv('TEMP') or os.getenv('TMP') or os.path.join(os.path.expanduser('~'), 'tmp')
        shm_stats = psutil.disk_usage(temp_dir)
        shm_size = shm_stats.total
        shm_hbytes = human_readable_size(shm_size)
        warn = []
        if shm_size < 512 * 1024**2:
            warn.append(
                f" {YELLOW} [WARNING] Shared memory size might be too small, consider increasing it. {END}"
            )
            # Add additional warnings specific to your use case if needed.
        return shm_hbytes, warn
    except Exception as e:
        return "UNKNOWN", [f"Error getting shared memory size: {e}"]
 ```
 11. While still in your command line with python environment enabled run:<br>
 ```build_win.bat```
 12. Now cd dist to go into your dist folder and you can now pip install deepspeed-YOURFILENAME.whl (or whatever your WHL file is called).
 ### 🔴 Future to-do list
 - Complete & document the new/full standalone mode API.
 - Voice output within the command prompt/terminal (TBD).
 - Correct an issue on incorrect output folder path when running as a standalone app.
 - Correct a few spelling mistakes in the documnentation.
 - Possibly add some additional TTS engines (TBD).
 - Have a break!
@@ -0,0 +1,851 @@
@echo off
 cd /D "%~dp0"
 setlocal enabledelayedexpansion
@set "PATH=%PATH%;%SystemRoot%\system32"
 :: Generate the ESC character
 for /F %%a in ('echo prompt $E ^| cmd') do set "ESC=%%a"
 :: Standard Colors
 set "BLACK=%ESC%[30m"
 set "RED=%ESC%[31m"
 set "GREEN=%ESC%[32m"
 set "YELLOW=%ESC%[33m"
 set "BLUE=%ESC%[34m"
 set "MAGENTA=%ESC%[35m"
 set "CYAN=%ESC%[36m"
 set "WHITE=%ESC%[37m"
 set "L_BLACK=%ESC%[90m"
 set "L_RED=%ESC%[91m"
 set "L_GREEN=%ESC%[92m"
 set "L_YELLOW=%ESC%[93m"
 set "L_BLUE=%ESC%[94m"
 set "L_MAGENTA=%ESC%[95m"
 set "L_CYAN=%ESC%[96m"
 set "L_WHITE=%ESC%[97m"
 set "RESET=%ESC%[0m"
 echo "%CD%"| findstr /C:" " >nul &&  echo. && echo    %L_BLUE%ALLTALK WINDOWS SETUP UTILITY%RESET% && echo. &&  echo. &&  echo    You are trying to install AllTalk in a folder that has a space in the folder path e.g. && echo. && echo       C:\%L_RED%program files%RESET%\alltalk_tts && echo.	&& echo    This causes errors with Conda and Python scripts. Please follow this link for reference: &&  echo. && echo      %L_CYAN%https://docs.anaconda.com/free/working-with-conda/reference/faq/#installing-anaconda%RESET% && echo. && echo    Please use a folder path that has no spaces in it e.g. && echo. && echo       C:\myfiles\alltalk_tts\ && echo. && echo. && pause && goto :end
@rem Check for special characters in installation path
 set "SPCHARMESSAGE="WARNING: Special characters were detected in the installation path!" "         This can cause the installation to fail!""
 echo "%CD%"| findstr /R /C:"[!#\$%&()\*+,;<=>?@\[\]\^`{|}~]" >nul && (
 	call :PrintBigMessage %SPCHARMESSAGE%
 )
 set SPCHARMESSAGE=
@rem Check if curl is available
 curl --version >nul 2>&1
 if "%ERRORLEVEL%" NEQ "0" (
    echo curl is not available on this system. Please install curl then re-run the script https://curl.se/ 
 	echo or perform a manual installation of a Conda Python environment.
    goto end
 )
 :: Check for "-silent" install command-line argument
 if "%1"=="-silent" goto InstallCustomStandalone
 :MainMenu
 cls
 echo.
 echo    %L_BLUE%ALLTALK WINDOWS SETUP UTILITY%RESET%
 echo.
 echo    INSTALLATION TYPE
 echo    1) I am using AllTalk as part of %L_GREEN%Text-generation-webui%RESET%
 echo    2) I am using AllTalk as a %L_BLUE%Standalone Application%RESET%
 echo.
 echo    9) %L_RED%Exit/Quit%RESET%
 echo.
 echo.
 set /p UserOption="    Enter your choice: "
 if "%UserOption%"=="1" goto WebUIMenu
 if "%UserOption%"=="2" goto StandaloneMenu
 if "%UserOption%"=="9" goto End
 goto MainMenu
 :WebUIMenu
 cls
 echo.
 echo    %L_BLUE%TEXT-GENERATION-WEBUI SETUP%RESET%
 echo.
 echo    Please ensure you have started your Text-generation-webui Python 
 echo    environment. If you have NOT done this, please run %L_GREEN%cmd_windows.bat%RESET% 
 echo    in the %L_GREEN%text-generation-webui%RESET% folder and then re-run this script.
 echo.
 echo    BASE REQUIREMENTS
 echo    1) Apply/Re-Apply the requirements for %L_GREEN%Text-generation-webui%RESET%.
 echo.
 echo    OPTIONAL
 echo    2) Git Pull the latest AllTalk updates from Github
 echo.
 echo    DEEPSPEED FOR %L_YELLOW=%PyTorch 2.1.x%RESET%
 echo    4) Install DeepSpeed v11.2 for CUDA %L_GREEN%11.8%RESET% and Python-3.11.x and %L_YELLOW%PyTorch 2.1.x%RESET%.
 echo    5) Install DeepSpeed v11.2 for CUDA %L_GREEN%12.1%RESET% and Python-3.11.x and %L_YELLOW%PyTorch 2.1.x%RESET%.
 echo.
 echo    DEEPSPEED FOR %L_YELLOW=%PyTorch 2.2.x%RESET% (March 2024 builds of Text-gen-webui and later)
 echo    6) Install DeepSpeed v14.0 for CUDA %L_GREEN%12.1%RESET% and Python-3.11.x and %L_YELLOW%PyTorch 2.2.x%RESET%.
 echo    7) Install DeepSpeed v14.0 for CUDA %L_GREEN%11.8%RESET% and Python-3.11.x and %L_YELLOW%PyTorch 2.2.x%RESET%.
 echo.
 echo    U) Uninstall DeepSpeed.
 echo.
 echo.   OTHER
 echo    8) Generate a diagnostics file.
 echo.
 echo    9) %L_RED%Exit/Quit%RESET%
 echo.
 set /p WebUIOption="Enter your choice: "
 if "%WebUIOption%"=="1" goto InstallNvidiaTextGen
 if "%WebUIOption%"=="2" goto TGGitpull
 if "%WebUIOption%"=="4" goto InstallDeepSpeed118TextGen
 if "%WebUIOption%"=="5" goto InstallDeepSpeed121TextGen
 if "%WebUIOption%"=="6" goto InstallDeepSpeed121TextGenPytorch221
 if "%WebUIOption%"=="7" goto InstallDeepSpeed118TextGenPytorch221
 if "%WebUIOption%"=="U" goto UnInstallDeepSpeed
 if "%WebUIOption%"=="u" goto UnInstallDeepSpeed
 if "%WebUIOption%"=="8" goto GenerateDiagsTextGen
 if "%WebUIOption%"=="9" goto End
 goto WebUIMenu
 :StandaloneMenu
 cls
 echo.
 echo    %L_BLUE%ALLTALK STANDALONE APPLICATION SETUP%RESET%
 echo.
 echo    BASE REQUIREMENTS
 echo    1) Install AllTalk as a Standalone Application
 echo.
 echo    OPTIONAL
 echo    2) Git Pull the latest AllTalk updates from Github
 echo    3) Re-Apply/Update the requirements file
 echo    4) Delete AllTalk's custom Python environment
 echo    5) Purge the PIP cache
 echo.
 echo.   OTHER
 echo    8) Generate a diagnostics file
 echo.
 echo    9) %L_RED%Exit/Quit%RESET%
 echo.
 set /p StandaloneOption="Enter your choice: "
 if "%StandaloneOption%"=="1" goto InstallCustomStandalone
 if "%StandaloneOption%"=="2" goto STGitpull
 if "%StandaloneOption%"=="3" goto STReapplyrequirements
 if "%StandaloneOption%"=="4" goto STDeleteCustomStandalone
 if "%StandaloneOption%"=="5" goto STPurgepipcache
 if "%StandaloneOption%"=="8" goto GenerateDiagsStandalone
 if "%StandaloneOption%"=="9" goto EndStandalone
 goto StandaloneMenu
 :InstallNvidiaTextGen
 pip install -r system\requirements\requirements_textgen.txt
 pip install -r system\requirements\requirements_textgen2.txt
 pip install -r system\requirements\requirements_parler.txt
 echo ** Faiss **
 call "%CONDA_ROOT_PREFIX%\Scripts\conda.exe" install -y pytorch::faiss-cpu
 echo ** FFmpeg **
 call "%CONDA_ROOT_PREFIX%\Scripts\conda.exe" install -y -c conda-forge "ffmpeg=*=*gpl*"
 call "%CONDA_ROOT_PREFIX%\Scripts\conda.exe" install -y -c conda-forge "ffmpeg=*=h*_*" --no-deps
 if %ERRORLEVEL% neq 0 (
    echo.
    echo    There was an error installing the requirements.
    echo    Have you started your Text-gen-webui Python environment
    echo    with cmd_{yourOS} before running atsetup.bat?
    echo    Press any key to return to the menu.
    echo.
    pause
    goto WebUIMenu
 )
 Echo.
 echo.
 Echo    Requirements installed successfully.
 Echo. 
 pause
 goto WebUIMenu
 :TGGitpull
 git pull
 if %ERRORLEVEL% neq 0 (
    echo.
    echo    There was an error pulling from Github.
    echo    Press any key to return to the menu.
    echo.
    pause
    goto WebUIMenu
 )
 Echo.
 echo     AllTalk Updated from Github. Please re-apply
 echo     the latest requirements file. Option 1
 Echo. 
 pause
 goto WebUIMenu
 :InstallDeepSpeed118TextGen
 echo Downloading DeepSpeed...
 curl -LO https://github.com/erew123/alltalk_tts/releases/download/deepspeed/deepspeed-0.11.2+cuda118-cp311-cp311-win_amd64.whl
 if %ERRORLEVEL% neq 0 (
    echo.
    echo    Failed to download DeepSpeed wheel file.
    echo    Please check your internet connection or try again later.
    echo    Press any key to return to the menu.
    echo.
    pause
    goto WebUIMenu
 )
 echo DeepSpeed wheel file downloaded successfully.
 echo Installing DeepSpeed...
 pip install deepspeed-0.11.2+cuda118-cp311-cp311-win_amd64.whl
 if %ERRORLEVEL% neq 0 (
    echo. 
    echo    Failed to install DeepSpeed.
    echo    Please check if the wheel file is compatible with your system.
    echo    Press any key to return to the menu.
    echo.
    pause
    goto WebUIMenu
 )
 Echo.
 echo.
 Echo    DeepSpeed installed successfully.
 Echo. 
 del deepspeed-0.11.2+cuda118-cp311-cp311-win_amd64.whl
 pause
 goto WebUIMenu
 :InstallDeepSpeed121TextGen
 echo Downloading DeepSpeed...
 curl -LO https://github.com/erew123/alltalk_tts/releases/download/deepspeed/deepspeed-0.11.2+cuda121-cp311-cp311-win_amd64.whl
 if %ERRORLEVEL% neq 0 (
    echo.
    echo    Failed to download DeepSpeed wheel file.
    echo    Please check your internet connection or try again later.
    echo    Press any key to return to the menu.
    echo.
    pause
    goto WebUIMenu
 )
 echo    DeepSpeed wheel file downloaded successfully.
 echo    Installing DeepSpeed...
 pip install deepspeed-0.11.2+cuda121-cp311-cp311-win_amd64.whl
 if %ERRORLEVEL% neq 0 (
    echo. 
    echo    Failed to install DeepSpeed.
    echo    Please check if the wheel file is compatible with your system.
    echo    Press any key to return to the menu.
    echo.
    pause
    goto WebUIMenu
 )
 Echo.
 echo.
 Echo    DeepSpeed installed successfully.
 Echo. 
 del deepspeed-0.11.2+cuda121-cp311-cp311-win_amd64.whl
 pause
 goto WebUIMenu
 :InstallDeepSpeed121TextGenPytorch221
 echo Downloading DeepSpeed...
 curl -LO https://github.com/erew123/alltalk_tts/releases/download/DeepSpeed-14.0/deepspeed-0.14.0+ce78a63-cp311-cp311-win_amd64.whl
 if %ERRORLEVEL% neq 0 (
    echo.
    echo    Failed to download DeepSpeed wheel file.
    echo    Please check your internet connection or try again later.
    echo    Press any key to return to the menu.
    echo.
    pause
    goto WebUIMenu
 )
 echo    DeepSpeed wheel file downloaded successfully.
 echo    Installing DeepSpeed...
 pip install deepspeed-0.14.0+ce78a63-cp311-cp311-win_amd64.whl
 if %ERRORLEVEL% neq 0 (
    echo. 
    echo    Failed to install DeepSpeed.
    echo    Please check if the wheel file is compatible with your system.
    echo    Press any key to return to the menu.
    echo.
    pause
    goto WebUIMenu
 )
 Echo.
 echo.
 Echo    DeepSpeed installed successfully.
 Echo. 
 del deepspeed-0.14.0+ce78a63-cp311-cp311-win_amd64.whl
 pause
 goto WebUIMenu
 :InstallDeepSpeed118TextGenPytorch221
 echo Downloading DeepSpeed...
 curl -LO https://github.com/erew123/alltalk_tts/releases/download/DeepSpeed-14.0/deepspeed-0.14.0+cu118-cp311-cp311-win_amd64.whl
 if %ERRORLEVEL% neq 0 (
    echo.
    echo    Failed to download DeepSpeed wheel file.
    echo    Please check your internet connection or try again later.
    echo    Press any key to return to the menu.
    echo.
    pause
    goto WebUIMenu
 )
 echo    DeepSpeed wheel file downloaded successfully.
 echo    Installing DeepSpeed...
 pip install deepspeed-0.14.0+cu118-cp311-cp311-win_amd64.whl
 if %ERRORLEVEL% neq 0 (
    echo. 
    echo    Failed to install DeepSpeed.
    echo    Please check if the wheel file is compatible with your system.
    echo    Press any key to return to the menu.
    echo.
    pause
    goto WebUIMenu
 )
 Echo.
 echo.
 Echo    DeepSpeed installed successfully.
 Echo. 
 del deepspeed-0.14.0+cu118-cp311-cp311-win_amd64.whl
 pause
 goto WebUIMenu
 :UnInstallDeepSpeed
 pip uninstall deepspeed
 if %ERRORLEVEL% neq 0 (
    echo. 
    echo    There was an error uninstalling DeepSpeed.
    echo    Press any key to return to the menu.
    echo.
    pause
    goto WebUIMenu
 )
 echo.
 Echo    DeepSpeed uninstalled successfully.
 Echo. 
 pause
 goto WebUIMenu
 :GenerateDiagsTextGen
 Python diagnostics.py
 if %ERRORLEVEL% neq 0 (
    echo.
    echo    There was an error running diagnostics. Have you correctly started your 
    echo    Text-generation-webui Python environment with %L_GREEN%cmd_windows.bat%RESET%?
    echo.
    pause
    goto WebUIMenu
 )
 Echo.
 echo.
 Echo    Diagnostics.log generated. Please scroll up to look over the log.
 Echo. 
 pause
 goto WebUIMenu
 :InstallCustomStandalone
 set PATH=%PATH%;%SystemRoot%\system32
@rem Check if curl is available
 curl --version >nul 2>&1
 if "%ERRORLEVEL%" NEQ "0" (
    echo curl is not available on this system. Please install curl then re-run the script https://curl.se/ 
 	echo or perform a manual installation of a Conda Python environment.
    goto end
 )
 echo "%CD%"| findstr /C:" " >nul && echo This script relies on Miniconda which can not be silently installed under a path with spaces. Please correct your folder names and re-try installation. && goto exit
@rem Check for special characters in installation path
 set "SPCHARMESSAGE=WARNING: Special characters were detected in the installation path! This can cause the installation to fail!"
 echo "%CD%"| findstr /R /C:"[!#\$%&()\*+,;<=>?@\[\]\^`{|}~]" >nul && (
    call :PrintBigMessage %SPCHARMESSAGE%
 )
 set SPCHARMESSAGE=
@rem fix failed install when installing to a separate drive
 set TMP=%cd%\alltalk_environment
 set TEMP=%cd%\alltalk_environment
@rem deactivate existing conda envs as needed to avoid conflicts
 (call conda deactivate && call conda deactivate && call conda deactivate) 2>nul
@rem config
 set INSTALL_DIR=%cd%\alltalk_environment
 set CONDA_ROOT_PREFIX=%cd%\alltalk_environment\conda
 set INSTALL_ENV_DIR=%cd%\alltalk_environment\env
 set MINICONDA_DOWNLOAD_URL=https://repo.anaconda.com/miniconda/Miniconda3-py311_24.4.0-0-Windows-x86_64.exe
 set conda_exists=F
@rem figure out whether git and conda need to be installed
 call "%CONDA_ROOT_PREFIX%\_conda.exe" --version >nul 2>&1
 if "%ERRORLEVEL%" EQU "0" (
    set conda_exists=T
    call "%CONDA_ROOT_PREFIX%\condabin\conda.bat" activate "%INSTALL_ENV_DIR%"
    goto RunScript
 )
@rem download and install conda if not installed
 echo Downloading Miniconda from %MINICONDA_DOWNLOAD_URL% to %INSTALL_DIR%\miniconda_installer.exe
 mkdir "%INSTALL_DIR%"
 call curl -Lk "%MINICONDA_DOWNLOAD_URL%" > "%INSTALL_DIR%\miniconda_installer.exe" || ( echo. && echo Miniconda failed to download. && goto end )
 echo Installing Miniconda to %CONDA_ROOT_PREFIX%
 start /wait "" "%INSTALL_DIR%\miniconda_installer.exe" /InstallationType=JustMe /NoShortcuts=1 /AddToPath=0 /RegisterPython=0 /NoRegistry=1 /S /D=%CONDA_ROOT_PREFIX%
 echo Miniconda version:
 call "%CONDA_ROOT_PREFIX%\_conda.exe" --version || ( echo. && echo Miniconda not found. && goto end )
@rem create the installer env
 echo Packages to install: %PACKAGES_TO_INSTALL%
 call "%CONDA_ROOT_PREFIX%\_conda.exe" create --no-shortcuts -y -k --prefix "%INSTALL_ENV_DIR%" python=3.11.9 || ( echo. && echo Conda environment creation failed. && goto end )
@rem check if conda environment was actually created
 if not exist "%INSTALL_ENV_DIR%\python.exe" ( echo. && echo Conda environment is empty. && goto end )
@rem activate installer env
 call "%CONDA_ROOT_PREFIX%\condabin\conda.bat" activate "%INSTALL_ENV_DIR%" || ( echo. && echo Miniconda hook not found. && goto end )
 rem Install required packages
 :install_pytorch
 echo ** Installing PyTorch 2.2.1 **
 call "%CONDA_ROOT_PREFIX%\Scripts\conda.exe" install -y pytorch==2.2.1 torchvision==0.17.1 torchaudio==2.2.1 pytorch-cuda=12.1 -c pytorch -c nvidia
 echo.
 if errorlevel 1 (
    echo PyTorch installation failed, errorlevel was %errorlevel%. There should be a Conda
    echo error message above with a code and/or text explanation of the issue. Please note 
    echo the error code to help with diagnotics. 
    echo.
    echo Generally speaking though, errors could be caused by:
    echo.
    echo      1. Internet connection issues/unable to download PyTorch from Conda's website.
    echo      2. Disk space related issues - Check your disk space on this drive.
    echo      3. Incorrect or missing Conda environment - Restart installation.
    echo      4. Permissions issues - Check you have enough rights on this system.
    echo      5. Firewall or proxy settings blocking access to Conda's servers.
    echo      6. Antivirus or security software interference.
    echo      7. Other issues not mentioned above.
    echo.
    echo      Known errors are in the AllTalk Github Wiki > Error Messages Help/Support
    echo.
    choice /C YN /M "Do you want to retry the Pytorch installation?"
    if errorlevel 2 goto End
    if errorlevel 1 goto install_pytorch
 ) else (
    echo PyTorch installation was successful.
    echo.
 )
 :install_faiss
 echo ** Installing Faiss **
 call "%CONDA_ROOT_PREFIX%\Scripts\conda.exe" install -y pytorch::faiss-cpu
 echo.
 if errorlevel 1 (
    echo Faiss installation failed, errorlevel was %errorlevel%. There should be a Conda
    echo error message above with a code and/or text explanation of the issue. Please note 
    echo the error code to help with diagnotics. 
    echo.
    echo Generally speaking though, errors could be caused by:
    echo.
    echo      1. Internet connection issues/unable to download Faiss from Conda's website.
    echo      2. Disk space related issues - Check your disk space on this drive.
    echo      3. Incorrect or missing Conda environment - Restart installation.
    echo      4. Permissions issues - Check you have enough rights on this system.
    echo      5. Firewall or proxy settings blocking access to Conda's servers.
    echo      6. Antivirus or security software interference.
    echo      7. Your Conda policy is set to Strict see the Github Wiki for help with this.
    echo      8. Other issues not mentioned above.
    echo.
    echo      Known errors are in the AllTalk Github Wiki > Error Messages Help/Support
    echo.
    choice /C YN /M "Do you want to retry the Faiss installation?"
    if errorlevel 2 goto End
    if errorlevel 1 goto install_faiss
 ) else (
    echo Faiss installation was successful.
    echo.
 )
 :install_ffmpeg
 echo ** Installing FFmpeg **
 call "%CONDA_ROOT_PREFIX%\Scripts\conda.exe" install -y -c conda-forge "ffmpeg=*=*gpl*"
 call "%CONDA_ROOT_PREFIX%\Scripts\conda.exe" install -y -c conda-forge "ffmpeg=*=h*_*" --no-deps
 echo.
 if errorlevel 1 (
    echo FFmpeg installation failed, errorlevel was %errorlevel%. There should be a Conda
    echo error message above with a code and/or text explanation of the issue. Please note 
    echo the error code to help with diagnotics. 
    echo.
    echo Generally speaking though, errors could be caused by:
    echo.
    echo      1. Internet connection issues/unable to download FFmpeg from Conda's website.
    echo      2. Disk space related issues - Check your disk space on this drive.
    echo      3. Incorrect or missing Conda environment - Restart installation.
    echo      4. Permissions issues - Check you have enough rights on this system.
    echo      5. Firewall or proxy settings blocking access to Conda's servers.
    echo      6. Antivirus or security software interference.
    echo      7. Your Conda policy is set to Strict see the Github Wiki for help with this.
    echo      8. Other issues not mentioned above.
    echo.
    echo      Known errors are in the AllTalk Github Wiki > Error Messages Help/Support
    echo.
    choice /C YN /M "Do you want to retry the FFmpeg installation?"
    if errorlevel 2 goto End
    if errorlevel 1 goto install_ffmpeg
 ) else (
    echo FFmpeg installation was successful.
    echo.
 )
 echo ** Requirements file **
 pip install -r system\requirements\requirements_standalone.txt
 echo.
 :update_gradio
 echo ** Updating Gradio **
 pip install --upgrade gradio==4.44.1
 echo.
 if errorlevel 1 (
    echo Gradio update failed, errorlevel was %errorlevel%. There should be an error
    echo message above with a code and/or text explanation of the issue. Please note 
    echo the error code to help with diagnotics. 
    echo.
    echo Generally speaking though, errors could be caused by:
    echo.
    echo      1. Internet connection issues/unable to download Gradio from PyPI.
    echo      2. Disk space related issues - Check your disk space on this drive.
    echo      3. Incorrect or missing Python environment - Restart installation.
    echo      4. Permissions issues - Check you have enough rights on this system.
    echo      5. Firewall or proxy settings blocking access to PyPI servers.
    echo      6. Antivirus or security software interference.
    echo      7. Other issues not mentioned above.
    echo.
    echo      Known errors are in the AllTalk Github Wiki > Error Messages Help/Support
    echo.
    choice /C YN /M "Do you want to retry the Gradio update?"
    if errorlevel 2 goto End
    if errorlevel 1 goto update_gradio
 ) else (
    echo Gradio update was successful.
    echo.
 )
 :install_deepspeed
 echo ** Downloading DeepSpeed **
 curl -LO https://github.com/erew123/alltalk_tts/releases/download/DeepSpeed-14.0/deepspeed-0.14.0+ce78a63-cp311-cp311-win_amd64.whl
 echo.
 if errorlevel 1 (
    echo DeepSpeed download failed, errorlevel was %errorlevel%. There should be an error
    echo message above with a code and/or text explanation of the issue. Please note 
    echo the error code to help with diagnotics. 
    echo.
    echo Generally speaking though, errors could be caused by:
    echo.
    echo      1. Internet connection issues/unable to reach GitHub.
    echo      2. Disk space related issues - Check your disk space on this drive.
    echo      3. Permissions issues - Check you have enough rights on this system.
    echo      4. Firewall or proxy settings blocking access to GitHub.
    echo      5. Antivirus or security software interference.
    echo      6. The file may have been moved or deleted from the GitHub repository.
    echo      7. Other issues not mentioned above.
    echo.
    echo      Known errors are in the AllTalk Github Wiki > Error Messages Help/Support
    echo.
    choice /C YN /M "Do you want to retry the DeepSpeed download?"
    if errorlevel 2 goto End
    if errorlevel 1 goto install_deepspeed
 ) else (
    echo DeepSpeed download was successful.
    echo.
 )
 echo ** Installing DeepSpeed **
 pip install deepspeed-0.14.0+ce78a63-cp311-cp311-win_amd64.whl
 echo.
 if errorlevel 1 (
    echo DeepSpeed installation failed, errorlevel was %errorlevel%. There should be an error
    echo message above with a code and/or text explanation of the issue. Please note 
    echo the error code to help with diagnotics. General issues may be:
    echo.
    echo Generally speaking though, errors could be caused by:
    echo.
    echo      1. The DeepSpeed wheel file was not downloaded successfully in the previous step.
    echo      2. Microsoft C++ development tools for Python are not installed correctly.
    echo      3. Disk space related issues - Check your disk space on this drive.
    echo      4. Incorrect or missing Python environment - Restart installation.
    echo      5. Permissions issues - Check you have enough rights on this system.
    echo      6. Antivirus or security software interference.
    echo      7. Other issues not mentioned above.
    echo.
    echo      Known errors are in the AllTalk Github Wiki > Error Messages Help/Support
    echo.
    echo Please ensure you have followed the instructions to install the Microsoft C++ development
    echo tools for Python, which is detailed on the AllTalk GitHub page.
    echo.
    choice /C YN /M "Do you want to retry the DeepSpeed installation?"
    if errorlevel 2 goto End
    if errorlevel 1 goto install_deepspeed
 ) else (
    echo DeepSpeed installation was successful.
    del deepspeed-0.14.0+ce78a63-cp311-cp311-win_amd64.whl
    echo.
 )
 echo ** Installing Parler **
 pip install -r system\requirements\requirements_parler.txt
 echo Clean Environment
 call "%CONDA_ROOT_PREFIX%\Scripts\conda.exe" clean --all --force-pkgs-dirs -y
 echo.
 echo.
 echo Installation process completed.
@rem Create start_environment.bat to run AllTalk environment
 echo @echo off > start_environment.bat
 echo cd /D "%~dp0" >> start_environment.bat
 echo set CONDA_ROOT_PREFIX=%cd%\alltalk_environment\conda >> start_environment.bat
 echo set INSTALL_ENV_DIR=%cd%\alltalk_environment\env >> start_environment.bat
 echo call "%CONDA_ROOT_PREFIX%\condabin\conda.bat" activate "%INSTALL_ENV_DIR%" >> start_environment.bat
@rem Create start_alltalk.bat to run AllTalk
 echo @echo off > start_alltalk.bat
 echo cd /D "%~dp0" >> start_alltalk.bat
 echo set CONDA_ROOT_PREFIX=%cd%\alltalk_environment\conda >> start_alltalk.bat
 echo set INSTALL_ENV_DIR=%cd%\alltalk_environment\env >> start_alltalk.bat
 echo call "%CONDA_ROOT_PREFIX%\condabin\conda.bat" activate "%INSTALL_ENV_DIR%" >> start_alltalk.bat
 echo call python script.py >> start_alltalk.bat
@rem Create start_finetune.bat to run AllTalk
 echo @echo off > start_finetune.bat
 echo cd /D "%~dp0" >> start_finetune.bat
 echo set CONDA_ROOT_PREFIX=%cd%\alltalk_environment\conda >> start_finetune.bat
 echo set INSTALL_ENV_DIR=%cd%\alltalk_environment\env >> start_finetune.bat
 echo call "%CONDA_ROOT_PREFIX%\condabin\conda.bat" activate "%INSTALL_ENV_DIR%" >> start_finetune.bat
 echo call python finetune.py >> start_finetune.bat
 echo @echo off > start_diagnostics.bat
 echo cd /D "%~dp0" >> start_diagnostics.bat
 echo set CONDA_ROOT_PREFIX=%cd%\alltalk_environment\conda >> start_diagnostics.bat
 echo set INSTALL_ENV_DIR=%cd%\alltalk_environment\env >> start_diagnostics.bat
 echo call "%CONDA_ROOT_PREFIX%\condabin\conda.bat" activate "%INSTALL_ENV_DIR%" >> start_diagnostics.bat
 echo call python diagnostics.py >> start_diagnostics.bat
 Echo.
 Echo    Run %L_YELLOW%start_alltalk.bat%RESET% to start AllTalk.
 Echo    Run %L_YELLOW%start_diagnostics.bat%RESET% to start the diagnostics.
 Echo    Run %L_YELLOW%start_finetune.bat%RESET% to start Finetuning.
 Echo    Run %L_YELLOW%start_environment.bat%RESET% to start the AllTalk Python environment.
 Echo. 
 Echo    Documentation is built into the Gradio interface. Please explore the documentation for
 Echo    tips, troubleshooing and explanations as most common questions are answered there.
 Echo.
 if "%1"=="-silent" goto End
 pause
 goto StandaloneMenu
 :STDeleteCustomStandalone
@rem Check if the alltalk_environment directory exists
 if not exist "%cd%\alltalk_environment\" (
    echo.
    echo    %L_GREEN%alltalk_environment%RESET% directory does not exist. No need to delete.
    echo.
    pause
    goto StandaloneMenu
 )
@rem Check if a Conda environment is active
 if not defined CONDA_PREFIX goto NoCondaEnvDeleteCustomStandalone
@rem Deactivate Conda environment if it's active
 Echo    Exiting the Conda Environment. Please run %L_GREEN%atsetup.bat%RESET% again and delete the environment.
 call conda deactivate
 :NoCondaEnvDeleteCustomStandalone
 echo Deleting "alltalk_environment". Please wait.
 rd /s /q "alltalk_environment"
 del start_alltalk.bat
 del start_environment.bat
 if %ERRORLEVEL% neq 0 (
    echo.
    echo    Failed to delete alltalk_environment folder.
    echo    Please make sure it is not in use and try again.
    echo    Worst case, manually delete the folder called 'alltalk_environment'
    echo.
    pause
    goto StandaloneMenu
 )
 echo.
 Echo.
 echo    Environment %L_GREEN%alltalk_environment%RESET% deleted. Please set up the environment again.
 Echo.
 pause
 goto StandaloneMenu
 :GenerateDiagsStandalone
 cd /D "%~dp0"
 set CONDA_ROOT_PREFIX=%cd%\alltalk_environment\conda
 set INSTALL_ENV_DIR=%cd%\alltalk_environment\env
@rem Check if the Conda environment exists
 if not exist "%INSTALL_ENV_DIR%\python.exe" (
    echo.
    echo    The Conda environment at "%INSTALL_ENV_DIR%" does not exist.
    echo    Please install the environment before proceeding.
    echo. 
    pause
    goto StandaloneMenu
 )
@rem Attempt to activate the Conda environment
 call "%CONDA_ROOT_PREFIX%\condabin\conda.bat" activate "%INSTALL_ENV_DIR%"
 if errorlevel 1 (
    echo. 
    echo    Failed to activate the Conda environment.
    echo    Please check your installation and try again.
    echo.
    pause
    goto StandaloneMenu
 )
@rem Run diagnostics
 Python diagnostics.py
 if %ERRORLEVEL% neq 0 (
    echo.
    echo    There was an error running diagnostics.
    echo    Press any key to return to the menu.
    echo.
    pause
    goto StandaloneMenu
 )
 Echo.
 Echo.
 Echo    Diagnostics.log generated. Please scroll up to look over the log.
 Echo.
 pause
 goto StandaloneMenu
 :STReapplyrequirements
 cd /D "%~dp0"
 set CONDA_ROOT_PREFIX=%cd%\alltalk_environment\conda
 set INSTALL_ENV_DIR=%cd%\alltalk_environment\env
@rem Check if the Conda environment exists
 if not exist "%INSTALL_ENV_DIR%\python.exe" (
    echo.
    echo    The Conda environment at "%INSTALL_ENV_DIR%" does not exist.
    echo    Please install the environment before proceeding.
    echo. 
    pause
    goto StandaloneMenu
 )
@rem Attempt to activate the Conda environment
 call "%CONDA_ROOT_PREFIX%\condabin\conda.bat" activate "%INSTALL_ENV_DIR%"
 if errorlevel 1 (
    echo. 
    echo    Failed to activate the Conda environment.
    echo    Please check your installation and try again.
    echo.
    pause
    goto StandaloneMenu
 )
@rem Run Reapply requirements
 echo.
 echo  Re-installing requirements. If you have problems after this, it may indicate 
 echo  that the custom Python environment is damaged in some way. As such it may be
 echo  best to delete the custom Python environment and re-run the entire setup
 echo  routine again to rebuild AllTalk's Python environment. Please press a key
 echo  to continue re-applying the Requirements for a standalone installation.
 echo.
 pause
 goto install_faiss
 :STPurgepipcache
 cd /D "%~dp0"
 set CONDA_ROOT_PREFIX=%cd%\alltalk_environment\conda
 set INSTALL_ENV_DIR=%cd%\alltalk_environment\env
@rem Check if the Conda environment exists
 if not exist "%INSTALL_ENV_DIR%\python.exe" (
    echo.
    echo    The Conda environment at "%INSTALL_ENV_DIR%" does not exist.
    echo    Please install the environment before proceeding.
    echo. 
    pause
    goto StandaloneMenu
 )
@rem Attempt to activate the Conda environment
 call "%CONDA_ROOT_PREFIX%\condabin\conda.bat" activate "%INSTALL_ENV_DIR%"
 if errorlevel 1 (
    echo. 
    echo    Failed to activate the Conda environment.
    echo    Please check your installation and try again.
    echo.
    pause
    goto StandaloneMenu
 )
@rem Clear the PIP cache
 echo.
 echo     Purging the PIP cache of downloaded files.
 echo.
 pip cache purge
 if %ERRORLEVEL% neq 0 (
    echo.
    echo    There was an error.
    echo    Press any key to return to the menu.
    echo.
    pause
    goto StandaloneMenu
 )
 Echo.
 Echo    The PIP cache has been purged.
 Echo.
 pause
 goto StandaloneMenu
 :STGitpull
 cd /D "%~dp0"
 set CONDA_ROOT_PREFIX=%cd%\alltalk_environment\conda
 set INSTALL_ENV_DIR=%cd%\alltalk_environment\env
@rem Check if the Conda environment exists
 if not exist "%INSTALL_ENV_DIR%\python.exe" (
    echo.
    echo    The Conda environment at "%INSTALL_ENV_DIR%" does not exist.
    echo    Please install the environment before proceeding.
    echo. 
    pause
    goto StandaloneMenu
 )
@rem Attempt to activate the Conda environment
 call "%CONDA_ROOT_PREFIX%\condabin\conda.bat" activate "%INSTALL_ENV_DIR%"
 if errorlevel 1 (
    echo. 
    echo    Failed to activate the Conda environment.
    echo    Please check your installation and try again.
    echo.
    pause
    goto StandaloneMenu
 )
@rem Pull from Github
 echo.
 echo     Pulling the latest updates. Please re-apply
 echo     the latest requirements file. Option 3
 echo.
 git pull
 if %ERRORLEVEL% neq 0 (
    echo.
    echo    There was an error pulling from Github.
    echo    Press any key to return to the menu.
    echo.
    pause
    goto StandaloneMenu
 )
 Echo.
 echo     AllTalk Updated from Github. Please re-apply
 echo     the latest requirements file. Option 3
 Echo.
 pause
 goto StandaloneMenu
 :EndStandalone
 echo Exiting AllTalk Setup Utility...
 echo.
 Echo    Remember, after installation you can....
 Echo.
 Echo    Run %L_YELLOW%start_alltalk.bat%RESET% to start AllTalk.
 Echo    Run %L_YELLOW%start_finetune.bat%RESET% to start Finetuning.
 Echo    Run %L_YELLOW%start_environment.bat%RESET% to start the AllTalk Python environment.
 Echo    Run %L_YELLOW%start_diagnostics.bat%RESET% to start generate a diagnostics file.
 Echo.
 Echo    Documentation is built into the Gradio interface. Please explore the documentation for
 Echo    tips, troubleshooing and explanations as most common questions are answered there.
 exit /b
 :End
 echo Exiting AllTalk Setup Utility...
 Echo.
 Echo    Documentation is built into the Gradio interface and the AllTalk Wiki page on Github.
 Echo    Please explore the documentation for tips, troubleshooing and explanations on the most
 Echo    common questions.
 exit /b
@@ -0,0 +1,558 @@
 #!/bin/bash
 # ANSI color codes
 RED='\033[1;31m'
 GREEN='\033[1;32m'
 YELLOW='\033[1;33m'
 BLUE='\033[1;34m'
 CYAN='\033[1;36m'
 MAGENTA='\033[1;35m'
 WHITE='\033[1;37m'
 L_RED='\033[1;91m'
 L_GREEN='\033[1;92m'
 L_YELLOW='\033[1;93m'
 L_BLUE='\033[1;94m'
 L_CYAN='\033[1;96m'
 L_MAGENTA='\033[1;95m'
 NC='\033[0m' # No Color
 # Navigate to the script's directory
 cd "$(dirname "$0")"
 # Function to check if curl is installed
 check_curl() {
    if ! command -v curl &> /dev/null; then
        echo "curl is not available on this system. Please install curl then re-run the script https://curl.se/"
        echo "or perform a manual installation of a Conda Python environment."
        exit 1
    fi
 }
 # Check if the current directory path contains a space
 containsSpace=false
 currentPath=$(pwd)
 if echo "$currentPath" | grep -q ' '; then
    containsSpace=true
 fi
 if [ "$containsSpace" = true ]; then
    echo
    echo -e "    ${L_BLUE}ALLTALK LINUX SETUP UTILITY${NC}"
    echo
    echo
    echo -e "    You are trying to install AllTalk in a folder that has a space in the"
    echo -e "    folder path e.g."
    echo 
    echo -e "       /home/${L_RED}program files${NC}/alltalk_tts"
    echo 
    echo -e "    This causes errors with Conda and Python scripts. Please follow this"
    echo -e "    link for reference:"
    echo 
    echo -e "      ${L_CYAN}https://docs.anaconda.com/free/working-with-conda/reference/faq/#installing-anaconda${NC}"
    echo 
    echo -e "    Please use a folder path that has no spaces in it e.g." 
    echo 
    echo -e "       /home/myfiles/alltalk_tts/"
    echo 
    echo
    read -p "Press Enter to continue..." 
    exit 1
 else
    # Continue with the main menu
    echo "Continue with the main menu."
 fi
 # Main Menu
 main_menu() {
    while true; do
        clear
        echo
        echo -e "    ${L_BLUE}ALLTALK LINUX SETUP UTILITY${NC}"
        echo
        echo "    INSTALLATION TYPE"
        echo -e "    1) I am using AllTalk as part of ${L_GREEN}Text-generation-webui${NC}"
        echo -e "    2) I am using AllTalk as a ${L_GREEN}Standalone Application${NC}"
        echo
        echo -e "    9)${L_RED} Exit/Quit${NC}"
        echo
        read -p "    Enter your choice: " user_option
        case $user_option in
            1) webui_menu ;;
            2) standalone_menu ;;
            9) exit 0 ;;
            *) echo "Invalid option"; sleep 2 ;;
        esac
    done
 }
 # Text-generation-webui Menu
 webui_menu() {
    while true; do
        clear
        echo
        echo -e "    ${L_BLUE}TEXT-GENERATION-WEBUI SETUP${NC}"
        echo
        echo -e "    Please ensure you have started your Text-generation-webui Python"
        echo -e "    environment. If you have NOT done this, please run ${L_GREEN}/cmd_linux.sh${NC}"
        echo -e "    in the ${L_GREEN}text-generation-webui${NC} folder and then re-run this script."
        echo
        echo "    BASE REQUIREMENTS"
        echo -e "    1) Apply/Re-Apply the requirements for an ${L_GREEN}Text-generation-webui${NC}"
        echo
        echo "    OPTIONAL"
        echo "    2) Git Pull the latest AllTalk updates from Github"
        echo
        echo "    DEEPSPEED"
        echo "    4) Install DeepSpeed."
        echo "    5) Uninstall DeepSpeed."
        echo
        echo "    OTHER"
        echo "    6) Generate a diagnostics file."
        echo
        echo -e "    9)${L_RED} Exit/Quit${NC}"
        echo
        read -p "    Enter your choice: " webui_option
        case $webui_option in
            1) install_nvidia_textgen ;;
            2) tg_gitpull ;;
            4) install_deepspeed ;;
            5) uninstall_deepspeed ;;
            6) generate_diagnostics_textgen ;;
            9) exit 0 ;;
            *) echo "Invalid option"; sleep 2 ;;
        esac
    done
 }
 install_nvidia_textgen() {
    local requirements_file="system/requirements/requirements_textgen.txt"
    echo "    Installing Requirements from $requirements_file..."
    if ! pip install -r "$requirements_file"; then
        echo
        echo "    There was an error pulling from Github."
        echo "    Please check the output for details."
        read -p "    Press any key to return to the menu. " -n 1
        echo
        return
    fi
    echo
    echo "    Requirements installed successfully."
    echo "    Installing additional requirements..."
    if ! pip install -r system/requirements/requirements_textgen2.txt; then
        echo
        echo "    There was an error installing additional requirements."
        echo "    Please check the output for details."
        read -p "    Press any key to return to the menu. " -n 1
        echo
        return
    fi
    if ! conda install -y pytorch::faiss-cpu; then
        echo
        echo "    There was an error installing faiss-cpu."
        echo "    Please check the output for details."
        read -p "    Press any key to return to the menu. " -n 1
        echo
        return
    fi
    if ! conda install -y conda-forge::ffmpeg; then
        echo
        echo "    There was an error installing ffmpeg."
        echo "    Please check the output for details."
        read -p "    Press any key to return to the menu. " -n 1
        echo
        return
    fi
    echo
    echo "    Additional requirements installed successfully."
    echo
    echo -e "    To install ${L_YELLOW}DeepSpeed${NC} on Linux, there are additional"
    echo -e "    steps required. Please see the Github or documentation on DeepSeed."
    echo
    read -p "    Press any key to continue. " -n 1
    echo
 }
 tg_gitpull() {
    echo
    if ! git pull; then
        echo
        echo "    There was an error installing the requirements."
        echo "    Please check the output for details."
        echo
        read -p "    Press any key to return to the menu. " -n 1
        echo
        return
    fi
    echo
    echo "    AllTalk Updated from Github. Please re-apply"
    echo "    the latest requirements file. (Option 1)"
    echo
    read -p "    Press any key to continue. " -n 1
    echo
 }
 # Function to install DeepSpeed
 install_deepspeed() {
    clear
    echo
    echo -e "    ${L_BLUE}DEEPSPEED INSTALLATION REQUIREMENTS${NC}"
    echo
    echo -e "    - Please go see this link as the instructions are there.${NC}"
    echo -e "    The instructions there are making it as simple as posisble.${NC}"
    echo -e "      ${L_GREEN}https://github.com/erew123/alltalk_tts/releases/tag/DeepSpeed-14.2-Linux${NC}"
    echo
    read -p "    Have you completed all the above steps? (y/n): " confirm
    if [ "$confirm" != "y" ]; then
        echo -e "    ${RED}DeepSpeed installation cannot proceed without completing the prerequisites.${NC}"
        return
    fi
    echo -e "\n    ${GREEN}Proceeding with DeepSpeed installation...${NC}"
    if [ $? -ne 0 ]; then
        echo -e "    ${RED}There was an error installing DeepSpeed.${NC}"
        return
    fi
    echo -e "    ${GREEN}DeepSpeed installed successfully.${NC}"
    read -p "    Press any key to continue. " -n 1
    echo
 }
 uninstall_deepspeed() {
    echo "Uninstalling DeepSpeed..."
    pip uninstall -y deepspeed
    if [ $? -ne 0 ]; then
        echo
        echo "    There was an error uninstalling DeepSpeed."
        echo
        echo "    Press any key to return to the menu."
        read -n 1
        return
    fi
    echo
    echo "    DeepSpeed uninstalled successfully."
    echo
    echo "    Press any key to continue."
    read -n 1
 }
 generate_diagnostics_textgen() {
    # Run diagnostics
    if ! python diagnostics.py; then
        echo
        echo "    There was an error running diagnostics."
        echo
        read -p "    Press any key to return to the menu. " -n 1
        echo
        return
    fi
    echo
    echo "    Diagnostics log file generated successfully."
    echo "    Please see diagnostics.log"
    echo
    read -p "    Press any key to continue. " -n 1
    echo
 }
 # Standalone Menu
 standalone_menu() {
    while true; do
        clear
        echo
        echo -e "    ${L_BLUE}ALLTALK STANDALONE APPLICATION SETUP${NC}"
        echo
        echo "    BASE REQUIREMENTS"
        echo "    1) Install AllTalk as a Standalone Application"
        echo
        echo "    OPTIONAL"
        echo "    2) Git Pull the latest AllTalk updates from Github"
        echo "    3) Re-Apply/Update the requirements file"
        echo "    4) Delete AllTalk's custom Python environment"
        echo "    5) Purge the PIP cache"
        echo
        echo "    OTHER"        
        echo "    8) Generate a diagnostics file"
        echo
        echo -e "    9)${L_RED} Exit/Quit${NC}"
        echo
        read -p "    Enter your choice: " standalone_option
        case $standalone_option in
            1) install_custom_standalone ;;
            2) gitpull_standalone ;;
            3) reapply_standalone ;;
            4) delete_custom_standalone ;;
            5) pippurge_standalone ;;
            8) generate_diagnostics_standalone ;;
            9) exit 0 ;;
            *) echo "Invalid option"; sleep 2 ;;
        esac
    done
 }
 install_custom_standalone() {
    cd "$(dirname "${BASH_SOURCE[0]}")"
    if [[ "$(pwd)" =~ " " ]]; then
        echo "This script relies on Miniconda which can not be silently installed under a path with spaces."
        exit
    fi
    # Deactivate existing conda envs as needed to avoid conflicts
    { conda deactivate && conda deactivate && conda deactivate; } 2> /dev/null
    OS_ARCH=$(uname -m)
    case "${OS_ARCH}" in
        x86_64*)    OS_ARCH="x86_64" ;;
        arm64* | aarch64*) OS_ARCH="aarch64" ;;
        *)          echo "Unknown system architecture: $OS_ARCH! This script runs only on x86_64 or arm64" && exit ;;
    esac
    # Config
    INSTALL_DIR="$(pwd)/alltalk_environment"
    CONDA_ROOT_PREFIX="${INSTALL_DIR}/conda"
    INSTALL_ENV_DIR="${INSTALL_DIR}/env"
    MINICONDA_DOWNLOAD_URL="https://repo.anaconda.com/miniconda/Miniconda3-py311_24.4.0-0-Linux-${OS_ARCH}.sh"
    if [ ! -x "${CONDA_ROOT_PREFIX}/bin/conda" ]; then
        echo "Downloading Miniconda from $MINICONDA_DOWNLOAD_URL to ${INSTALL_DIR}/miniconda_installer.sh"
        mkdir -p "${INSTALL_DIR}"
        curl -L "${MINICONDA_DOWNLOAD_URL}" -o "${INSTALL_DIR}/miniconda_installer.sh"
        chmod +x "${INSTALL_DIR}/miniconda_installer.sh"
        bash "${INSTALL_DIR}/miniconda_installer.sh" -b -p "${CONDA_ROOT_PREFIX}"
        echo "Miniconda installed."
    fi
    if [ ! -d "${INSTALL_ENV_DIR}" ]; then
        "${CONDA_ROOT_PREFIX}/bin/conda" create -y --prefix "${INSTALL_ENV_DIR}" -c conda-forge python=3.11.9
        echo "Conda environment created at ${INSTALL_ENV_DIR}."
    fi
    # Activate the environment and install requirements
    source "${CONDA_ROOT_PREFIX}/etc/profile.d/conda.sh"
    conda activate "${INSTALL_ENV_DIR}"
    # pip install torch==2.2.1+cu121 torchaudio==2.2.1+cu121 --upgrade --force-reinstall --extra-index-url https://download.pytorch.org/whl/cu121 --no-cache-dir
    conda install -y pytorch=2.2.1 torchvision torchaudio pytorch-cuda=12.1 -c pytorch -c nvidia
    conda install -y nvidia/label/cuda-12.1.0::cuda-toolkit=12.1
    conda install -y pytorch::faiss-cpu
    conda install -y -c conda-forge "ffmpeg=*=*gpl*"
    conda install -y -c conda-forge "ffmpeg=*=h*_*" --no-deps
    echo
    echo
    echo
    echo "    Fix Nvidia's broken symlinks in the /env/lib folder"
    echo
    # Define the environment path
    env_path="${INSTALL_ENV_DIR}/lib"
    echo "    Installing additional requirements."
    echo
    pip install -r system/requirements/requirements_standalone.txt
    curl -LO https://github.com/erew123/alltalk_tts/releases/download/DeepSpeed-14.0/deepspeed-0.14.2+cu121torch2.2-cp311-cp311-manylinux_2_24_x86_64.whl
    pip install --upgrade gradio==4.44.1
    echo Installing DeepSpeed...
    pip install deepspeed-0.14.2+cu121torch2.2-cp311-cp311-manylinux_2_24_x86_64.whl
    rm deepspeed-0.14.2+cu121torch2.2-cp311-cp311-manylinux_2_24_x86_64.whl
    pip install -r system/requirements/requirements_parler.txt
    conda clean --all --force-pkgs-dirs -y
    # Create start_environment.sh to run AllTalk
    cat << EOF > start_environment.sh
 #!/bin/bash
 cd "$(dirname "${BASH_SOURCE[0]}")"
 if [[ "$(pwd)" =~ " " ]]; then echo This script relies on Miniconda which can not be silently installed under a path with spaces. && exit; fi
 # deactivate existing conda envs as needed to avoid conflicts
 { conda deactivate && conda deactivate && conda deactivate; } 2> /dev/null
 # config
 CONDA_ROOT_PREFIX="$(pwd)/alltalk_environment/conda"
 INSTALL_ENV_DIR="$(pwd)/alltalk_environment/env"
 # environment isolation
 export PYTHONNOUSERSITE=1
 unset PYTHONPATH
 unset PYTHONHOME
 export CUDA_PATH="$INSTALL_ENV_DIR"
 export CUDA_HOME="$CUDA_PATH"
 # activate env
 bash --init-file <(echo "source \"$CONDA_ROOT_PREFIX/etc/profile.d/conda.sh\" && conda activate \"$INSTALL_ENV_DIR\"")
 EOF
    cat << EOF > start_alltalk.sh
 #!/bin/bash
 source "${CONDA_ROOT_PREFIX}/etc/profile.d/conda.sh"
 conda activate "${INSTALL_ENV_DIR}"
 python script.py
 EOF
    cat << EOF > start_finetune.sh
 #!/bin/bash
 export TRAINER_TELEMETRY=0
 source "${CONDA_ROOT_PREFIX}/etc/profile.d/conda.sh"
 conda activate "${INSTALL_ENV_DIR}"
 python finetune.py
 EOF
    cat << EOF > start_diagnostics.sh
 #!/bin/bash
 source "${CONDA_ROOT_PREFIX}/etc/profile.d/conda.sh"
 conda activate "${INSTALL_ENV_DIR}"
 python diagnostics.py
 EOF
    chmod +x start_alltalk.sh
    chmod +x start_environment.sh
    chmod +x start_finetune.sh
    chmod +x start_diagnostics.sh
    echo
    echo
    echo -e "    Run ${L_YELLOW}./start_alltalk.sh${NC} to start AllTalk."
    echo -e "    Run ${L_YELLOW}./start_diagnostics.sh${NC} to start Diagnostics."
    echo -e "    Run ${L_YELLOW}./start_finetune.sh${NC} to start Finetuning."
    echo -e "    Run ${L_YELLOW}./start_environment.sh${NC} to start the AllTalk Python environment."
    echo
    echo -e "    To install ${L_YELLOW}DeepSpeed${NC} on Linux, there are additional"
    echo -e "    steps required. Please see the Github or documentation on DeepSeed."
    echo
    read -p "    Press any key to continue. " -n 1
 }
 delete_custom_standalone() {
    local env_dir="$PWD/alltalk_environment"
    # Check if the alltalk_environment directory exists
    if [ ! -d "$env_dir" ]; then
        echo "    \"$env_dir\" directory does not exist. No need to delete."
        read -p "    Press any key to continue. " -n 1
        echo
        return
    fi
    # Check if a Conda environment is active and deactivate it
    if [ -n "$CONDA_PREFIX" ]; then
        echo "    Exiting the Conda environment. You may need to start ./atstart.sh again"
        conda deactivate
    fi
    echo "Deleting \"$env_dir\". Please wait."
    rm -rf "$env_dir"
    if [ -d "$env_dir" ]; then
        echo "    Failed to delete \"$env_dir\" folder."
        echo "    Please make sure it is not in use and try again."
    else
        echo "    Environment \"$env_dir\" deleted successfully."
    fi
    read -p "    Press any key to continue. " -n 1
    echo
 }
 generate_diagnostics_standalone() {
    local env_dir="$PWD/alltalk_environment"
    local conda_root_prefix="${env_dir}/conda"
    local install_env_dir="${env_dir}/env"
    if [ ! -d "${install_env_dir}" ]; then
        echo
        echo "    The Conda environment at '${install_env_dir}' does not exist."
        echo "    Please install the environment before proceeding."
        echo
        read -p "    Press any key to return to the menu. " -n 1
        echo
        return
    fi
    source "${conda_root_prefix}/etc/profile.d/conda.sh"
    conda activate "${install_env_dir}"
    if ! python diagnostics.py; then
        echo
        echo "    There was an error running diagnostics."
        echo
        read -p "    Press any key to return to the menu. " -n 1
        echo
        return
    fi
    echo
    echo "    Diagnostics completed successfully."
    read -p "    Press any key to continue. " -n 1
    echo
 }
 gitpull_standalone() {
    local env_dir="$PWD/alltalk_environment"
    local conda_root_prefix="${env_dir}/conda"
    local install_env_dir="${env_dir}/env"
    if [ ! -d "${install_env_dir}" ]; then
        echo
        echo "    The Conda environment at '${install_env_dir}' does not exist."
        echo "    Please install the environment before proceeding."
        echo
        read -p "    Press any key to return to the menu. " -n 1
        echo
        return
    fi
    source "${conda_root_prefix}/etc/profile.d/conda.sh"
    conda activate "${install_env_dir}"
    if ! git pull; then
        echo
        echo "    There was an error pulling from Github."
        echo
        read -p "    Press any key to return to the menu. " -n 1
        echo
        return
    fi
    echo
    echo "    AllTalk Updated from Github. Please re-apply."
    echo "    the latest requirements file. (Option 3)"
    echo
    read -p "    Press any key to continue. " -n 1
    echo
 }
 pippurge_standalone() {
    if ! pip cache purge; then
        echo
        echo "    There was an error."
        echo
        read -p "    Press any key to return to the menu. " -n 1
        echo
        return
    fi
    echo
    echo "    The PIP cache has been purged."
    echo
    read -p "    Press any key to continue. " -n 1
    echo
 }
 reapply_standalone() {
    local env_dir="$PWD/alltalk_environment"
    local conda_root_prefix="${env_dir}/conda"
    local install_env_dir="${env_dir}/env"
    if [ ! -d "${install_env_dir}" ]; then
        echo
        echo "    The Conda environment at '${install_env_dir}' does not exist."
        echo "    Please install the environment before proceeding."
        echo
        read -p "    Press any key to return to the menu. " -n 1
        echo
        return
    fi
    source "${conda_root_prefix}/etc/profile.d/conda.sh"
    conda activate "${install_env_dir}"
    echo
    echo "    Downloading and installing PyTorch. This step can take a long time"
    echo "    depending on your internet connection and hard drive speed. Please"
    echo "    be patient."
    pip install torch>=2.2.1+cu121 torchaudio>=2.2.1+cu121 --upgrade --force-reinstall --extra-index-url https://download.pytorch.org/whl/cu121
    echo
    echo "    Installing additional requirements."
    echo
    pip install -r system/requirements/requirements_standalone.txt
    pip install -r system/requirements/requirements_parler.txt
    echo
    echo "    Requirements have been re-applied/updated."
    echo
    read -p "    Press any key to continue. " -n 1
    echo
 }
 # Start the main menu
 main_menu
@@ -1 +0,0 @@
 {"activate": true, "autoplay": true, "branding": "AllTalk ", "narrator_enabled": false, "deepspeed_activate": false, "delete_output_wavs": "Disabled", "ip_address": "127.0.0.1", "language": "English", "low_vram": false, "local_temperature": "0.7", "local_repetition_penalty": "10.0", "model_loaded": true, "model_name": "tts_models/multilingual/multi-dataset/xtts_v2", "narrator_voice": "female_01.wav", "output_folder_wav": "extensions/alltalk_tts/outputs/", "port_number": "7851", "remove_trailing_dots": false, "show_text": true, "tts_method_api_local": false, "tts_method_api_tts": false, "tts_method_xtts_local": true, "voice": "female_01.wav"}
@@ -0,0 +1,371 @@
 import os
 import time
 import shutil
 from pathlib import Path
 from filelock import FileLock
 from abc import ABC, abstractmethod
 from typing import Callable, MutableSequence
 from pydantic import BaseModel, ConfigDict, AliasGenerator, AliasChoices, Field
 class AlltalkConfigTheme(BaseModel):
    # Map 'class' to 'clazz' and vice versa
    model_config = ConfigDict(
        alias_generator=AliasGenerator(
            validation_alias=lambda field_name:
                {
                    "clazz": AliasChoices("clazz", "class"),
                }.get(field_name, None),
            serialization_alias=lambda field_name: "class" if field_name == "clazz" else field_name,
        )
    )
    file: str | None = None
    clazz: str = "gradio/base"
 class AlltalkConfigRvcSettings(BaseModel):
    rvc_enabled: bool = False
    rvc_char_model_file: str = "Disabled"
    rvc_narr_model_file: str = "Disabled"
    split_audio: bool = True
    autotune: bool = False
    pitch: int = 0
    filter_radius: int = 3
    index_rate: float = 0.75
    rms_mix_rate: int = 1
    protect: float = 0.5
    hop_length: int = 130
    f0method: str = "fcpe"
    embedder_model: str = "hubert"
    training_data_size: int = 45000
 class AlltalkConfigTgwUi(BaseModel):
    tgwui_activate_tts: bool = True
    tgwui_autoplay_tts: bool = True
    tgwui_narrator_enabled: str = "false"
    tgwui_non_quoted_text_is: str = "character"
    tgwui_deepspeed_enabled: bool = False
    tgwui_language: str = "English"
    tgwui_lowvram_enabled: bool = False
    tgwui_pitch_set: int = 0
    tgwui_temperature_set: float = 0.75
    tgwui_repetitionpenalty_set: int = 10
    tgwui_generationspeed_set: int = 1
    tgwui_narrator_voice: str = "female_01.wav"
    tgwui_show_text: bool = True
    tgwui_character_voice: str = "female_01.wav"
    tgwui_rvc_char_voice: str = "Disabled"
    tgwui_rvc_char_pitch: int = 0
    tgwui_rvc_narr_voice: str = "Disabled"
    tgwui_rvc_narr_pitch: int = 0
 class AlltalkConfigApiDef(BaseModel):
    api_port_number: int = 7851
    api_allowed_filter: str = "[^a-zA-Z0-9\\s.,;:!?\\-\\'\"$\\u0400-\\u04FF\\u00C0-\\u017F\\u0150\\u0151\\u0170\\u0171\\u011E\\u011F\\u0130\\u0131\\u0900-\\u097F\\u2018\\u2019\\u201C\\u201D\\u3001\\u3002\\u3040-\\u309F\\u30A0-\\u30FF\\u4E00-\\u9FFF\\u3400-\\u4DBF\\uF900-\\uFAFF\\u0600-\\u06FF\\u0750-\\u077F\\uFB50-\\uFDFF\\uFE70-\\uFEFF\\uAC00-\\uD7A3\\u1100-\\u11FF\\u3130-\\u318F\\uFF01\\uFF0c\\uFF1A\\uFF1B\\uFF1F]"
    api_length_stripping: int = 3
    api_max_characters: int = 2000
    api_use_legacy_api: bool = False
    api_legacy_ip_address: str = "127.0.0.1"
    api_text_filtering: str = "standard"
    api_narrator_enabled: str = "false"
    api_text_not_inside: str = "character"
    api_language: str = "en"
    api_output_file_name: str = "myoutputfile"
    api_output_file_timestamp: bool = True
    api_autoplay: bool = False
    api_autoplay_volume: float = 0.5
 class AlltalkConfigDebug(BaseModel):
    debug_transcode: bool = False
    debug_tts: bool = False
    debug_openai: bool = False
    debug_concat: bool = False
    debug_tts_variables: bool = False
    debug_rvc: bool = False
    debug_func: bool = False
    debug_api: bool = False
    debug_fullttstext: bool = False
    debug_narrator: bool = False
    debug_gradio_IP: bool = False
    debug_transcribe: bool = False
    debug_proxy: bool = False
 class AlltalkConfigGradioPages(BaseModel):
    Generate_Help_page: bool = True
    Voice2RVC_page: bool = True
    TTS_Generator_page: bool = True
    TTS_Engines_Settings_page: bool = True
    alltalk_documentation_page: bool = True
    api_documentation_page: bool = True
 class AlltalkAvailableEngine(BaseModel):
    name: str = ""
    selected_model: str = ""
 class AlltalkConfigProxyEndpoint(BaseModel):
    enabled: bool = False
    external_port: int = 0
    external_ip: str = "0.0.0.0"
    cert_name: str = ""
 class AlltalkConfigProxySettings(BaseModel):
    proxy_enabled: bool = False
    start_on_startup: bool = False
    gradio_endpoint: AlltalkConfigProxyEndpoint = AlltalkConfigProxyEndpoint(external_port=444)
    api_endpoint: AlltalkConfigProxyEndpoint = AlltalkConfigProxyEndpoint(external_port=443)
    cert_validation: bool = True
    logging_enabled: bool = True
    log_level: str = "INFO"
 class AbstractJsonConfig(ABC):
    def __init__(self, config_path: Path | str, file_check_interval: int):
        super().__init__()
        self.__delegate = None
        self.__config_path = Path(config_path) if type(config_path) is str else config_path
        self.__last_read_time = 0  # Track when we last read the file
        self.__file_check_interval = file_check_interval
    def get_config_path(self):
        return self.__config_path
    def reload(self):
        self._load_config()
        return self
    def _reload_on_change(self):
        # Check if config file has been modified and reload if needed
        if time.time() - self.__last_read_time >= self.__file_check_interval:
            try:
                most_recent_modification = self.get_config_path().stat().st_mtime
                if most_recent_modification > self.__last_read_time:
                    self.reload()
            except Exception as e:
                print(f"Error checking config file: {e}")
    def _load_config(self):
        self.__last_read_time = self.get_config_path().stat().st_mtime
        def __load():
            with open(self.get_config_path(), "r") as configfile:
                json_string = configfile.read()
            # The delegate is the actual config loaded:
            self.__delegate = self._handle_loaded_config(json_string)
        self.__with_lock_and_backup(self.get_config_path(), False, __load)
    def _save_file(self, path: Path | None | str, default=None, indent=4):
        file_path = (Path(path) if type(path) is str else path) if path is not None else self.get_config_path()
        def __save():
            with open(file_path, 'w') as file:
                file.write(self.__delegate.model_dump_json(indent=indent, by_alias=True))
        self.__with_lock_and_backup(file_path, True, __save)
    def __with_lock_and_backup(self, path: Path, backup: bool, callable: Callable[[], None]):
        lock_path = path.with_suffix('.lock')
        backup_path = None
        try:
            with FileLock(lock_path):
                # Create backup:
                if path.exists() and backup:
                    backup_path = path.with_suffix('.backup')
                    shutil.copy(path, backup_path)
                try:
                    callable()
                except Exception as e:
                    if backup_path and backup_path.exists():
                        shutil.copy(backup_path, path)
                    raise Exception(f"Failed to save config: {e}")
        finally:
            # Cleanup lock and backup files:
            if lock_path.exists():
                lock_path.unlink()
            if backup and backup_path and backup_path.exists():
                backup_path.unlink()
    @abstractmethod
    def _handle_loaded_config(self, json_string: str):
        pass
    def to_dict(self):
        return self.__delegate.model_dump(by_alias=True)
    # Delegation to the loaded config data
    @property
    def _xtra(self): return [o for o in dir(self.__delegate) if not o.startswith('_')]
    def __getattr__(self, key):
        if key in self._xtra: return getattr(self.__delegate, key)
        raise AttributeError(key)
    def __getattribute__(self, key):
        if key.startswith("_"):
            # Private and protected attributes always are on self:
            return super().__getattribute__(key)
        if key in self._xtra:
            # Delegate attribute
            return getattr(self.__delegate, key)
        # Get own attribute:
        return super().__getattribute__(key)
    def __setattr__(self, key, value):
        if key.startswith("_"):
            # Private and protected attributes always are on self:
            super().__setattr__(key, value)
        elif key in dir(self.__delegate):
            # Delegate attribute
            setattr(self.__delegate, key, value)
        else:
            # Set own attribute:
            super().__setattr__(key, value)
    def __dir__(self):
        def custom_dir(c, add): return dir(type(c)) + list(c.__dict__.keys()) + add
        return custom_dir(self, self._xtra)
 class AlltalkNewEnginesConfigFields:
    engines_available: MutableSequence[AlltalkAvailableEngine] = Field(default_factory=list)
 class AlltalkNewEnginesConfigModel(BaseModel, AlltalkNewEnginesConfigFields):
    def get_engine_names_available(self):
        return [engine.name for engine in self.engines_available]
    def get_engines_matching(self, condition: Callable[[AlltalkAvailableEngine], bool]):
        return [x for x in self.engines_available if condition(x)]
 class AlltalkNewEnginesConfig(AbstractJsonConfig, AlltalkNewEnginesConfigFields):
    __instance = None
    __this_dir = Path(__file__).parent.resolve()
    def __init__(self, config_path: Path | str = os.path.join(__this_dir, "system", "tts_engines", "new_engines.json")):
        super().__init__(config_path, 5)
        self._load_config()
    @staticmethod
    def get_instance():
        if AlltalkNewEnginesConfig.__instance is None:
            AlltalkNewEnginesConfig.__instance = AlltalkNewEnginesConfig()
        AlltalkNewEnginesConfig.__instance._reload_on_change()
        return AlltalkNewEnginesConfig.__instance
    def _handle_loaded_config(self, json_string: str):
        return AlltalkNewEnginesConfigModel.model_validate_json(json_string)
 class AlltalkTTSEnginesConfigFields:
    engines_available: MutableSequence[AlltalkAvailableEngine] = Field(default_factory=list)
    engine_loaded: str = ""
    selected_model: str = ""
 class AlltalkTTSEnginesConfigModel(BaseModel, AlltalkTTSEnginesConfigFields):
    def get_engine_names_available(self):
        return [engine.name for engine in self.engines_available]
    def is_valid_engine(self, engine_name):
        return engine_name in self.get_engine_names_available()
    def change_engine(self, requested_engine):
        if requested_engine == self.engine_loaded:
            return self
        for engine in self.engines_available:
            if engine.name == requested_engine:
                self.engine_loaded = requested_engine
                self.selected_model = engine.selected_model
                return self
        return self
 class AlltalkTTSEnginesConfig(AbstractJsonConfig, AlltalkTTSEnginesConfigFields):
    __instance = None
    __this_dir = Path(__file__).parent.resolve()
    def __init__(self, config_path: Path | str = os.path.join(__this_dir, "system", "tts_engines", "tts_engines.json")):
        super().__init__(config_path, 5)
        self._load_config()
    @staticmethod
    def get_instance(force_reload = False):
        if AlltalkTTSEnginesConfig.__instance is None:
            force_reload = False
            AlltalkTTSEnginesConfig.__instance = AlltalkTTSEnginesConfig()
        if force_reload:
            AlltalkTTSEnginesConfig.__instance.reload()
        else:
            AlltalkTTSEnginesConfig.__instance._reload_on_change()
        return AlltalkTTSEnginesConfig.__instance
    def _handle_loaded_config(self, json_string: str):
        cfg = AlltalkTTSEnginesConfigModel.model_validate_json(json_string)
        cfg.engines_available = self.__handle_loaded_config_engines(cfg.engines_available)
        return cfg
    def __handle_loaded_config_engines(self, available_engines):
        available_engine_names = [engine.name for engine in available_engines]
        # Getting the engines that are not already part of the available engines:
        new_engines_config = AlltalkNewEnginesConfig.get_instance()
        new_engines = new_engines_config.get_engines_matching(lambda eng: eng.name not in available_engine_names)
        # Merge engines:
        return available_engines + new_engines
    def save(self, path: Path | str | None = None):
        self._save_file(path)
 class AlltalkConfigFields:
    branding: str = "AllTalk "
    delete_output_wavs: str = "Disabled"
    gradio_interface: bool = True
    output_folder: str = "outputs"
    gradio_port_number: int = 7852
    firstrun_model: bool = True
    firstrun_splash: bool = True
    launch_gradio: bool = True
    transcode_audio_format: str = "Disabled"
    theme: AlltalkConfigTheme = AlltalkConfigTheme()
    rvc_settings: AlltalkConfigRvcSettings = AlltalkConfigRvcSettings()
    tgwui: AlltalkConfigTgwUi = AlltalkConfigTgwUi()
    api_def: AlltalkConfigApiDef = AlltalkConfigApiDef()
    debugging: AlltalkConfigDebug = AlltalkConfigDebug()
    gradio_pages: AlltalkConfigGradioPages = AlltalkConfigGradioPages()
    proxy_settings: AlltalkConfigProxySettings = AlltalkConfigProxySettings()
 class AlltalkConfigModel(BaseModel, AlltalkConfigFields):
    __this_dir = Path(__file__).parent.resolve()
    def get_output_directory(self):
        return self.__this_dir / self.output_folder
 class AlltalkConfig(AbstractJsonConfig, AlltalkConfigFields):
    __instance = None
    __this_dir = Path(__file__).parent.resolve()
    def __init__(self, config_path: Path | str = __this_dir / "confignew.json"):
        super().__init__(config_path, 5)
        self._load_config()
    @staticmethod
    def default_config_path():
        return AlltalkConfig.__this_dir / "confignew.json"
    @staticmethod
    def get_instance(force_reload = False):
        if AlltalkConfig.__instance is None:
            force_reload = False
            AlltalkConfig.__instance = AlltalkConfig()
        if force_reload:
            AlltalkConfig.__instance.reload()
        else:
            AlltalkConfig.__instance._reload_on_change()
        return AlltalkConfig.__instance
    def save(self, path: Path | str | None = None):
        self._save_file(path)
    def _handle_loaded_config(self, json_string: str):
        model = AlltalkConfigModel.model_validate_json(json_string)
        model.get_output_directory().mkdir(parents=True, exist_ok=True)
        return model
@@ -0,0 +1,109 @@
 {
    "branding": "AllTalk ",
    "delete_output_wavs": "Disabled",
    "gradio_interface": true,
    "output_folder": "outputs",
    "gradio_port_number": 7852,
    "firstrun_model": true,
    "firstrun_splash": true,
    "launch_gradio": true,
    "transcode_audio_format": "Disabled",
    "theme": {
        "file": null,
        "class": "gradio/base"
    },
    "rvc_settings": {
        "rvc_enabled": false,
        "rvc_char_model_file": "Disabled",
        "rvc_narr_model_file": "Disabled",
        "split_audio": true,
        "autotune": false,
        "pitch": 0,
        "filter_radius": 3,
        "index_rate": 0.75,
        "rms_mix_rate": 1,
        "protect": 0.5,
        "hop_length": 130,
        "f0method": "fcpe",
        "embedder_model": "hubert",
        "training_data_size": 45000
    },
    "tgwui": {
        "tgwui_activate_tts": true,
        "tgwui_autoplay_tts": true,
        "tgwui_narrator_enabled": "false",
        "tgwui_non_quoted_text_is": "character",
        "tgwui_deepspeed_enabled": false,
        "tgwui_language": "English",
        "tgwui_lowvram_enabled": false,
        "tgwui_pitch_set": 0,
        "tgwui_temperature_set": 0.75,
        "tgwui_repetitionpenalty_set": 10,
        "tgwui_generationspeed_set": 1,
        "tgwui_narrator_voice": "female_01.wav",
        "tgwui_show_text": true,
        "tgwui_character_voice": "female_01.wav",
        "tgwui_rvc_char_voice": "Disabled",
        "tgwui_rvc_char_pitch": 0,
        "tgwui_rvc_narr_voice": "Disabled",
        "tgwui_rvc_narr_pitch": 0
    },
    "api_def": {
        "api_port_number": 7851,
        "api_allowed_filter": "[^a-zA-Z0-9\\s.,;:!?\\-\\'\"$\\u0400-\\u04FF\\u00C0-\\u017F\\u0150\\u0151\\u0170\\u0171\\u011E\\u011F\\u0130\\u0131\\u0900-\\u097F\\u2018\\u2019\\u201C\\u201D\\u3001\\u3002\\u3040-\\u309F\\u30A0-\\u30FF\\u4E00-\\u9FFF\\u3400-\\u4DBF\\uF900-\\uFAFF\\u0600-\\u06FF\\u0750-\\u077F\\uFB50-\\uFDFF\\uFE70-\\uFEFF\\uAC00-\\uD7A3\\u1100-\\u11FF\\u3130-\\u318F\\uFF01\\uFF0c\\uFF1A\\uFF1B\\uFF1F]",
        "api_length_stripping": 3,
        "api_max_characters": 2000,
        "api_use_legacy_api": false,
        "api_legacy_ip_address": "127.0.0.1",
        "api_text_filtering": "standard",
        "api_narrator_enabled": "false",
        "api_text_not_inside": "character",
        "api_language": "en",
        "api_output_file_name": "myoutputfile",
        "api_output_file_timestamp": true,
        "api_autoplay": false,
        "api_autoplay_volume": 0.5
    },
    "debugging": {
        "debug_transcode": false,
        "debug_tts": false,
        "debug_openai": false,
        "debug_concat": false,
        "debug_tts_variables": false,
        "debug_rvc": false,
        "debug_func": false,
        "debug_api": false,
        "debug_fullttstext": false,
        "debug_narrator": false,
        "debug_gradio_IP": false,
        "debug_transcribe": false,
        "debug_proxy": false
    },
    "gradio_pages": {
        "Generate_Help_page": true,
        "Voice2RVC_page": true,
        "TTS_Generator_page": true,
        "TTS_Engines_Settings_page": true,
        "alltalk_documentation_page": true,
        "api_documentation_page": true
    },
    "proxy_settings": {
        "proxy_enabled": false,
        "start_on_startup": false,
        "gradio_endpoint": {
            "enabled": false,
            "external_port": 444,
            "external_ip": "0.0.0.0",
            "cert_name": ""
        },
        "api_endpoint": {
            "enabled": false,
            "external_port": 443,
            "external_ip": "0.0.0.0",
            "cert_name": ""
        },
        "cert_validation": true,
        "logging_enabled": true,
        "log_level": "INFO"
    }
 }
@@ -0,0 +1,98 @@
 #!/usr/bin/env bash
 SCRIPT_DIR=$( cd -- "$( dirname -- "${BASH_SOURCE[0]}" )" &> /dev/null && pwd )
 cd $SCRIPT_DIR
 . ${SCRIPT_DIR=}/docker/variables.sh
 TTS_MODEL=xtts
 DOCKER_TAG=latest
 CLEAN=false
 # Create required build directories if they don't exist
 mkdir -p ${SCRIPT_DIR=}/docker/deepspeed/build
 echo "Created build directories"
 # Parse arguments
 while [ "$#" -gt 0 ]; do
  case "$1" in
    --cuda-version)
      CUDA_VERSION="$2"
      shift
      ;;
    --python-version)
      PYTHON_VERSION="$2"
      shift
      ;;
    --tts_model)
      TTS_MODEL="$2"
      shift
      ;;
    --deepspeed-version)
      DEEPSPEED_VERSION="$2"
      shift
      ;;
    --tag)
      DOCKER_TAG="$2"
      shift
      ;;
    --github-repository)
      if [ -n "${GITHUB_REPOSITORY}" ] && ! [[ $GITHUB_REPOSITORY =~ ^--.* ]]; then
        GITHUB_REPOSITORY="$2"
        shift
      fi
      ;;
    --clean)
      CLEAN=true
      ;;
    *)
      printf '%s\n' "Invalid argument ($1)"
      exit 1
      ;;
  esac
  shift
 done
 if [ "$CLEAN" = true ]; then
  rm -rf ${SCRIPT_DIR=}/docker/deepspeed/build
  # Recreate directories after clean
  mkdir -p ${SCRIPT_DIR=}/docker/deepspeed/build
  echo "Cleaned and recreated build directories"  
 fi
 echo "Building base environment image"
 $SCRIPT_DIR/docker/base/build-base-env.sh \
  --cuda-version ${CUDA_VERSION} \
  --python-version ${PYTHON_VERSION} \
  --github-repository ${GITHUB_REPOSITORY} \
  --tag ${DOCKER_TAG}
 if [ $? -ne 0 ]; then
  echo "Failed to build base environment image"
  exit 1
 fi
 echo "Building DeepSpeed"
 $SCRIPT_DIR/docker/deepspeed/build-deepspeed.sh \
  --python-version ${PYTHON_VERSION} \
  --github-repository ${GITHUB_REPOSITORY} \
  --tag ${DOCKER_TAG}
 if [ $? -ne 0 ]; then
  echo "Failed to build DeepSpeed"
  exit 1
 fi
 echo "Starting docker build process using TTS model '${TTS_MODEL}' and docker tag '${DOCKER_TAG}'"
 echo "Building for CUDA $CUDA_VERSION using python ${PYTHON_VERSION}"
 docker buildx \
  build \
  --progress=plain \
  --build-arg TTS_MODEL=$TTS_MODEL \
  --build-arg ALLTALK_DIR=$ALLTALK_DIR \
  --build-arg DEEPSPEED_VERSION=$DEEPSPEED_VERSION \
  -t ${GITHUB_REPOSITORY}alltalk_beta:${DOCKER_TAG} \
  .
 echo "Docker build process finished. Use docker-start.sh to start the container."
@@ -0,0 +1,74 @@
 #!/usr/bin/env bash
 SCRIPT_DIR=$( cd -- "$( dirname -- "${BASH_SOURCE[0]}" )" &> /dev/null && pwd )
 . ${SCRIPT_DIR=}/docker/variables.sh
 WITH_UI=true
 DOCKER_TAG=latest
 declare -a ADDITIONAL_ARGS=()
 # Parse arguments
 while [ "$#" -gt 0 ]; do
  case "$1" in
    --config)
      CONFIG="$2"
      shift
      ;;
    --voices)
      VOICES="$2"
      shift
      ;;
    --rvc_voices)
      RVC_VOICES="$2"
      shift
      ;;
    --no_ui)
      WITH_UI=false
      ;;
    --tag)
      DOCKER_TAG="$2"
      shift
      ;;
    --github-repository)
      if [ -n "${GITHUB_REPOSITORY}" ] && ! [[ $GITHUB_REPOSITORY =~ ^--.* ]]; then
        GITHUB_REPOSITORY="$2"
        shift
      fi
      ;;
    *)
      # Allow to pass arbitrary arguments to docker as well to be flexible:
      ADDITIONAL_ARGS+=( $1 )
      ;;
  esac
  shift
 done
 # Compose docker arguments based on user input to the script:
 declare -a DOCKER_ARGS=()
 if [[ -n $CONFIG ]]; then
  # Mount the config file to docker_confignew.json:
  DOCKER_ARGS+=( -v ${CONFIG}:${ALLTALK_DIR}/docker_confignew.json )
 fi
 if [[ -n $VOICES ]]; then
  DOCKER_ARGS+=( -v ${VOICES}:${ALLTALK_DIR}/voices )
 fi
 if [[ -n $RVC_VOICES ]]; then
  DOCKER_ARGS+=( -v ${RVC_VOICES}:${ALLTALK_DIR}/models/rvc_voices )
 fi
 if [ "$WITH_UI" = true ] ; then
    DOCKER_ARGS+=( -p 7852:7852 )
 fi
 docker run \
  --rm \
  -it \
  -p 7851:7851 \
  --gpus=all \
  --name alltalk \
 "${DOCKER_ARGS[@]}" \
 "${ADDITIONAL_ARGS[@]}" \
  ${GITHUB_REPOSITORY}alltalk_beta:${DOCKER_TAG} &> /dev/stdout
@@ -0,0 +1,80 @@
 FROM continuumio/miniconda3:24.7.1-0 AS build
 ARG CUDA_VERSION=12.6.0
 ENV CUDA_VERSION=$CUDA_VERSION
 ARG PYTHON_VERSION=3.11.11
 ENV PYTHON_VERSION=$PYTHON_VERSION
 SHELL ["/bin/bash", "-l", "-c"]
 ENV SHELL=/bin/bash
 ENV HOST=0.0.0.0
 ENV DEBIAN_FRONTEND=noninteractive
 ENV CUDA_DOCKER_ARCH=all
 ENV NVIDIA_VISIBLE_DEVICES=all
 ENV CONDA_AUTO_UPDATE_CONDA="false"
 ENV ALLTALK_DIR=/opt/alltalk
 ##############################################################################
 # Installation/Basic Utilities
 ##############################################################################
 RUN <<EOR
    apt-get update
    apt-get upgrade -y
    apt-get install --no-install-recommends -y \
      espeak-ng \
      curl \
      wget \
      jq \
      vim
    apt-get clean && rm -rf /var/lib/apt/lists/*
 EOR
 ##############################################################################
 # Create a conda environment and install dependencies:
 ##############################################################################
 RUN <<EOR
    CUDA_SHORT_VERSION=${CUDA_VERSION%.*}
    conda create -y -n "alltalk" -c conda-forge python=${PYTHON_VERSION}
    conda activate alltalk
    conda install -y \
      gcc_linux-64 \
      gxx_linux-64 \
      libaio \
      faiss-gpu=1.9.0 \
      cuda=${CUDA_VERSION} cuda-tools=${CUDA_VERSION} cuda-toolkit=${CUDA_VERSION} cuda-version=${CUDA_SHORT_VERSION} \
      cuda-command-line-tools=${CUDA_VERSION} cuda-compiler=${CUDA_VERSION} cuda-runtime=${CUDA_VERSION} \
      cuda-libraries=${CUDA_VERSION} cuda-libraries-dev=${CUDA_VERSION} cuda-libraries-static=${CUDA_VERSION} \
      cuda-cudart=${CUDA_SHORT_VERSION} cuda-cudart_linux-64=${CUDA_SHORT_VERSION} \
      cudnn=9.3 \
      py-cpuinfo=9 \
      conda-forge::ffmpeg=7.1.0 \
      conda-forge::portaudio=19.7.0 \
      -c pytorch \
      -c nvidia \
      -c anaconda
    if [ $? -ne 0 ] ; then
      echo "Failed to install conda dependencies: $RESULT"
      exit 1
    fi
    conda clean -a && pip cache purge
 EOR
 RUN <<EOR
  CUDA_SHORT_VERSION=${CUDA_VERSION%.*}
  CUDA_SHORT_VERSION_NO_DOT=${CUDA_SHORT_VERSION//./}
  conda activate alltalk
  mkdir -p ${ALLTALK_DIR}/pip_cache
  pip install --no-cache-dir --cache-dir=${ALLTALK_DIR}/pip_cache \
      torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu${CUDA_SHORT_VERSION_NO_DOT}
  if [ $? -ne 0 ] ; then
    echo "Failed to install pip dependencies: $RESULT"
    exit 1
  fi
 EOR
@@ -0,0 +1,47 @@
 #!/usr/bin/env bash
 SCRIPT_DIR=$( cd -- "$( dirname -- "${BASH_SOURCE[0]}" )" &> /dev/null && pwd )
 cd $SCRIPT_DIR
 . ${SCRIPT_DIR=}/../variables.sh
 DOCKER_TAG=latest
 # Parse arguments
 while [ "$#" -gt 0 ]; do
  case "$1" in
    --cuda-version)
      CUDA_VERSION="$2"
      shift
      ;;
    --python-version)
      PYTHON_VERSION="$2"
      shift
      ;;
    --tag)
      DOCKER_TAG="$2"
      shift
      ;;
    --github-repository)
      if [ -n "${GITHUB_REPOSITORY}" ] && ! [[ $GITHUB_REPOSITORY =~ ^--.* ]]; then
        GITHUB_REPOSITORY="$2"
        shift
      fi
      ;;
    *)
      echo "Unknown argument '$1'"
      exit 1
      ;;
  esac
  shift
 done
 echo "Building conda environment using python ${PYTHON_VERSION} with CUDA ${CUDA_VERSION}"
 docker buildx \
  build \
  --progress=plain \
  --build-arg CUDA_VERSION=$CUDA_VERSION \
  --build-arg PYTHON_VERSION=$PYTHON_VERSION \
  -t ${GITHUB_REPOSITORY}alltalk_environment:${DOCKER_TAG} \
  .
@@ -0,0 +1,65 @@
 ARG GITHUB_REPOSITORY
 FROM ${GITHUB_REPOSITORY}alltalk_environment:latest
 ARG DEEPSPEED_VERSION=0.16.3
 ENV DEEPSPEED_VERSION=$DEEPSPEED_VERSION
 ##############################################################################
 # Directories:
 ##############################################################################
 ENV STAGE_DIR=/tmp
 RUN mkdir -p ${STAGE_DIR}
 ##############################################################################
 # Install python dependencies
 ##############################################################################
 ENV PIP_CACHE_DIR=/tmp/pip_cache
 RUN <<EOR
  conda activate alltalk
  pip install \
      deepspeed-kernels
 EOR
 ##############################################################################
 # Clone DeepSpeed sources
 ##############################################################################
 RUN <<EOR
  git clone https://github.com/microsoft/DeepSpeed.git ${STAGE_DIR}/DeepSpeed
  cd ${STAGE_DIR}/DeepSpeed
  git checkout .
  git checkout "tags/v${DEEPSPEED_VERSION}" -b "v${DEEPSPEED_VERSION}"
  # Patching some files due to bug https://github.com/microsoft/DeepSpeed/issues/6709
  START_LINE=`awk '/#ifndef BF16_AVAILABLE/{ print NR; exit }' csrc/transformer/inference/csrc/gelu.cu`
  TO_LINE=$((START_LINE + 2))
  sed -i "${START_LINE},${TO_LINE}d" ./csrc/transformer/inference/csrc/gelu.cu
  START_LINE=`awk '/#ifndef BF16_AVAILABLE/{ print NR; exit }' csrc/transformer/inference/csrc/transform.cu`
  TO_LINE=$((START_LINE + 2))
  sed -i "${START_LINE},${TO_LINE}d" ./csrc/transformer/inference/csrc/transform.cu
 EOR
 ##############################################################################
 # Create DeepSpeed build file
 ##############################################################################
 RUN <<EOR
  cat << EOF > build_deepspeed.sh
 #!/usr/bin/env bash
 source ~/.bashrc
 mkdir -p /deepspeed
 conda activate alltalk
 cd ${STAGE_DIR}/DeepSpeed
 DS_BUILD_OPS=1 python setup.py build_ext -j8 bdist_wheel
 mv ${STAGE_DIR}/DeepSpeed/dist/*.whl /deepspeed/
 EOF
 EOR
 RUN chmod +x /build_deepspeed.sh
 ##############################################################################
 # Build DeepSpeed:
 ##############################################################################
 ENTRYPOINT ["/build_deepspeed.sh"]
@@ -0,0 +1,65 @@
 #!/usr/bin/env bash
 SCRIPT_DIR=$( cd -- "$( dirname -- "${BASH_SOURCE[0]}" )" &> /dev/null && pwd )
 cd $SCRIPT_DIR
 . ${SCRIPT_DIR=}/../variables.sh
 DOCKER_TAG=latest
 # Parse arguments
 while [ "$#" -gt 0 ]; do
  case "$1" in
    --python-version)
      PYTHON_VERSION="$2"
      shift
      ;;
    --deepspeed-version)
      DEEPSPEED_VERSION="$2"
      shift
      ;;
    --tag)
      DOCKER_TAG="$2"
      shift
      ;;
    --github-repository)
      if [ -n "${GITHUB_REPOSITORY}" ] && ! [[ $GITHUB_REPOSITORY =~ ^--.* ]]; then
        GITHUB_REPOSITORY="$2"
        shift
      fi
      ;;
    *)
      echo "Unknown argument '$1'"
      exit 1
      ;;
  esac
  shift
 done
 PYTHON_SHORT_VERSION=${PYTHON_VERSION%.*}
 PYTHON_VERSION_NO_DOT=${PYTHON_SHORT_VERSION//./}
 if [[ -n $(find build -name "deepspeed-${DEEPSPEED_VERSION}*-cp${PYTHON_VERSION_NO_DOT}-cp${PYTHON_VERSION_NO_DOT}-*.whl") ]]
 then
   echo "DeepSpeed was already built - skipping..."
   exit 0
 fi
 echo "Building DeepSpeed $DEEPSPEED_VERSION using python ${PYTHON_VERSION}"
 rm -rf build # make sure to properly clean up
 mkdir -p build
 docker buildx \
  build \
  --build-arg GITHUB_REPOSITORY=$GITHUB_REPOSITORY \
  --build-arg DEEPSPEED_VERSION=$DEEPSPEED_VERSION \
  -t ${GITHUB_REPOSITORY}alltalk_deepspeed:${DOCKER_TAG} \
  .
 docker run \
  --rm \
  -it \
  --gpus=all \
  --name deepspeed \
  -v $SCRIPT_DIR/build:/deepspeed \
  ${GITHUB_REPOSITORY}alltalk_deepspeed:${DOCKER_TAG}
@@ -0,0 +1,7 @@
 #!/usr/bin/env bash
 export CUDA_VERSION=12.6.0
 export PYTHON_VERSION=3.11.11
 export DEEPSPEED_VERSION=0.16.2
 export ALLTALK_DIR=/opt/alltalk
 export GITHUB_REPOSITORY=
@@ -0,0 +1 @@
 {}
@@ -0,0 +1,7 @@
 {
  "delete_output_wavs": "1",
  "rvc_settings": {
    "rvc_enabled": true,
    "f0method": "rmvpe"
  }
 }
@@ -0,0 +1,329 @@
 {
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "H-WKVp4Uukcl"
      },
      "source": [
        "#Install Server Requirements\n",
        "This should be run on a minimum of a T4 runtime, though it will run on a CPU only session, however long TTS generations may time out/error.\n",
        "\n",
        "This is a **work in progress**. Known issues:\n",
        "\n",
        "- The 1st TTS generation has a brief stutter.\n",
        "- RVC is not yet working.\n",
        "- Transcoding/ffmpeg isnt working.\n",
        "- Things yet to do on selecting your first model and other configuration setups.\n",
        "\n",
        "If you enable DeepSpeed for XTTS models, DeepSpeed has to compile on the 1st TTS generation which can take about 90 seconds. After that it should be fine."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 9,
      "metadata": {
        "cellView": "form",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "W6ls-RHTQeZv",
        "outputId": "25c62fb3-26ef-4493-8647-1d8e49bf5f01"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "************************************\n",
            "** Server requirements installed ***\n",
            "*** Please proceed to next step ****\n",
            "************************************\n",
            "\n"
          ]
        }
      ],
      "source": [
        "#@markdown Click the `Play button` to the left of this message to install the requirements.<br><br>\n",
        "#@markdown **OPTIONAL** Mounting your Google Drive allows you to drag and drop samples/models via the `drive/Mydrive` path. This allows you<br>\n",
        "#@markdown to store or use specific audio samples or finetuned models.<br>\n",
        "#@markdown **Audio samples** need to be placed in `alltalk_tts/voices`<br>\n",
        "#@markdown **XTTS models** need to be placed in `alltalk_tts/models/xtts/{yourmodelfolderhere}`<br>\n",
        "mount_gdrive = False #@param{type:\"boolean\"}\n",
        "\n",
        "if mount_gdrive:\n",
        "  from google.colab import drive\n",
        "  drive.mount('/content/drive')\n",
        "from IPython.display import clear_output\n",
        "print(\"*******************************************************************\")\n",
        "print(\"** Installing server requirements. This will take 5-10 minutes ****\")\n",
        "print(\"*******************************************************************\")\n",
        "!apt install libasound2-dev portaudio19-dev libportaudio2 libportaudiocpp0 libaio-dev espeak-ng > '/dev/null' 2>&1\n",
        "clear_output()\n",
        "print(\"************************\")\n",
        "print(\"*** Cloning AllTalk ****\")\n",
        "print(\"************************\\n\")\n",
        "!git clone -b alltalkbeta https://github.com/erew123/alltalk_tts\n",
        "clear_output()\n",
        "print(\"\\n********************************\")\n",
        "print(\"*** Installing Requirements ****\")\n",
        "print(\"********************************\\n\")\n",
        "!python -m pip install --upgrade \"pip<24.1\"\n",
        "!pip install --quiet -r /content/alltalk_tts/system/requirements/requirements_colab.txt\n",
        "clear_output()\n",
        "print(\"\\n*****************************\")\n",
        "print(\"*** Installing DeepSpeed ****\")\n",
        "print(\"*****************************\\n\")\n",
        "!pip install deepspeed\n",
        "!pip install orjson\n",
        "!pip install faiss-cpu\n",
        "!pip install fairseq\n",
        "clear_output()\n",
        "print(\"\\n******************************\")\n",
        "print(\"*** Installing Cloudflare ****\")\n",
        "print(\"******************************\\n\")\n",
        "# Install cloudflare\n",
        "!wget -q https://github.com/cloudflare/cloudflared/releases/latest/download/cloudflared-linux-amd64.deb > '/dev/null' 2>&1\n",
        "!apt install ./cloudflared-linux-amd64.deb aria2 > '/dev/null' 2>&1\n",
        "!rm cloudflared-linux-amd64.deb > '/dev/null' 2>&1\n",
        "!python -m spacy download en_core_web_md\n",
        "clear_output()\n",
        "print(\"\\n***************************\")\n",
        "print(\"*** Installing Cutlass ****\")\n",
        "print(\"***************************\\n\")\n",
        "# Clone the CUTLASS repository\n",
        "!git clone https://github.com/NVIDIA/cutlass.git\n",
        "!export CUTLASSPATH=/content/cutlass\n",
        "!sudo curl -L https://github.com/BtbN/FFmpeg-Builds/releases/download/latest/ffmpeg-master-latest-linux64-gpl.tar.xz -o /usr/local/bin/ffmpeg.tar.xz\n",
        "clear_output()\n",
        "%cd /usr/local/bin/\n",
        "clear_output()\n",
        "!7z e /usr/local/bin/ffmpeg.tar.xz -y\n",
        "clear_output()\n",
        "!7z e /usr/local/bin/ffmpeg.tar -y\n",
        "clear_output()\n",
        "!sudo chmod a+rx /usr/local/bin/ffmpeg\n",
        "clear_output()\n",
        "!pip uninstall jax -y\n",
        "clear_output()\n",
        "print(\"************************************\")\n",
        "print(\"** Server requirements installed ***\")\n",
        "print(\"*** Please proceed to next step ****\")\n",
        "print(\"************************************\\n\")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "2nk26r66RQ23"
      },
      "source": [
        "\n",
        "# Start AllTalk TTS Server\n",
        "\n",
        "This will start the AllTalk API and Gradio Web interface. From here you can download models, generate TTS and use external applications via the API address.\n",
        "\n",
        "The AllTalk API address is what you would use in Kobold, SillyTavern, TGWUI's Remote extension etc if you want to generate TTS with those applications."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "cellView": "form",
        "id": "9PCQhSoiWvhL"
      },
      "outputs": [],
      "source": [
        "#@markdown Click the `Play button` to the left of this message to start AllTalk API and Gradio Interface<br>\n",
        "import re\n",
        "import time\n",
        "import json\n",
        "import threading\n",
        "import time\n",
        "\n",
        "def keep_alive():\n",
        "    while True:\n",
        "        time.sleep(60)  # Run every 60 seconds (adjust as needed)\n",
        "\n",
        "keep_alive_thread = threading.Thread(target=keep_alive)\n",
        "keep_alive_thread.start()\n",
        "\n",
        "Tunnel = \"cloudflare\"\n",
        "host = \"127.0.0.1\"\n",
        "ports = [7851, 7852]\n",
        "tunnel_urls = []\n",
        "\n",
        "# Starting tunnels for each port.\n",
        "for port in ports:\n",
        "    if Tunnel == \"cloudflare\":\n",
        "        !nohup cloudflared tunnel --url http://{host}:{port} > lt_{port}.log 2>&1 &\n",
        "    else:\n",
        "        !nohup npx lt -p {port} > lt_{port}.log 2>&1 &\n",
        "\n",
        "# Wait a couple of seconds for the tunnels to initialize.\n",
        "time.sleep(10)\n",
        "\n",
        "# Extract URLs for each tunnel.\n",
        "for port in ports:\n",
        "    log_file = f'lt_{port}.log'\n",
        "    with open(log_file, 'r') as testwritefile:\n",
        "        log_content = testwritefile.read()\n",
        "\n",
        "        # Use regular expressions to find the URL.\n",
        "        if Tunnel == \"cloudflare\":\n",
        "            url_match = re.search(r'(https://[-a-z0-9]+\\.trycloudflare\\.com)', log_content)\n",
        "        else:\n",
        "            url_match = re.search(r'your url is: (https?://\\S+)', log_content)\n",
        "\n",
        "        if url_match:\n",
        "            tunnel_url = url_match.group(1)\n",
        "            tunnel_urls.append(tunnel_url)\n",
        "        else:\n",
        "            print(f\"URL for port {port} not found.\")\n",
        "\n",
        "# Save the tunnel URLs to a JSON file.\n",
        "try:\n",
        "    # Try to open the JSON file for reading.\n",
        "    with open('/content/alltalk_tts/googlecolab.json', 'r') as f:\n",
        "        data = json.load(f)\n",
        "except FileNotFoundError:\n",
        "    # If the file doesn't exist, create an empty dictionary.\n",
        "    data = {}\n",
        "\n",
        "data['google_ip_address'] = tunnel_urls\n",
        "\n",
        "# Write the modified data (or newly created data) back to the file.\n",
        "with open('/content/alltalk_tts/googlecolab.json', 'w') as f:\n",
        "    json.dump(data, f)\n",
        "\n",
        "host = \"0.0.0.0\"\n",
        "\n",
        "if Tunnel == \"localtunnel\":\n",
        "    print(\"Before you copy the link above click on it and copy that ip:\")\n",
        "    !curl ipv4.icanhazip.com\n",
        "\n",
        "# Start API server.\n",
        "!python /content/alltalk_tts/script.py\n"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "# Start XTTS model Finetuning\n",
        "\n",
        "Starts the Finetuning application for XTTS models.\n",
        "\n",
        "You can either run the `Start AllTalk` to download the base XTTS model(s) for finetuning. Or you can use the folder icon on the left hand side of the screen and upload an XTTS model that you want to finetune. If you are manually uploading a model, you would place your model files in `models/xtts/{yourmodelfolderhere}` and you will need all the models files in that folder `config.json, dvae.pth, mel_stats.pth, model.pth, speakers_xtts.pth, vocab.json`. Without 1x model available, Finetuning will not start.\n",
        "\n",
        "Likewise you can download your finetuned model from there, OR copy it to your Google Drive after finetuning, for later use in AllTalk. To access the Finetuning Gradio interface, connect to the `Google Colab Finetuning url` when Finetuning has started."
      ],
      "metadata": {
        "id": "gDzNPMwM7qvs"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "#@markdown Click the `Play button` to the left of this message to start Finetuning<br>\n",
        "import re\n",
        "import time\n",
        "import json\n",
        "import threading\n",
        "import time\n",
        "\n",
        "def keep_alive():\n",
        "    while True:\n",
        "        time.sleep(60)  # Run every 60 seconds (adjust as needed)\n",
        "\n",
        "keep_alive_thread = threading.Thread(target=keep_alive)\n",
        "keep_alive_thread.start()\n",
        "\n",
        "Tunnel = \"cloudflare\"\n",
        "host = \"127.0.0.1\"\n",
        "ports = [7052]\n",
        "tunnel_urls = []\n",
        "\n",
        "# Starting tunnels for each port.\n",
        "for port in ports:\n",
        "    if Tunnel == \"cloudflare\":\n",
        "        !nohup cloudflared tunnel --url http://{host}:{port} > lt_{port}.log 2>&1 &\n",
        "    else:\n",
        "        !nohup npx lt -p {port} > lt_{port}.log 2>&1 &\n",
        "\n",
        "# Wait a couple of seconds for the tunnels to initialize.\n",
        "time.sleep(10)\n",
        "\n",
        "# Extract URLs for each tunnel.\n",
        "for port in ports:\n",
        "    log_file = f'lt_{port}.log'\n",
        "    with open(log_file, 'r') as testwritefile:\n",
        "        log_content = testwritefile.read()\n",
        "\n",
        "        # Use regular expressions to find the URL.\n",
        "        if Tunnel == \"cloudflare\":\n",
        "            url_match = re.search(r'(https://[-a-z0-9]+\\.trycloudflare\\.com)', log_content)\n",
        "        else:\n",
        "            url_match = re.search(r'your url is: (https?://\\S+)', log_content)\n",
        "\n",
        "        if url_match:\n",
        "            tunnel_url = url_match.group(1)\n",
        "            tunnel_urls.append(tunnel_url)\n",
        "            print(f\"Google Colab Finetuning url: {tunnel_url}\\n\")\n",
        "            print(f\"********************************************************************\")\n",
        "            print(f\"**** Use the above URL to connect to Finetuning on Google Colab ****\")\n",
        "            print(f\"********************************************************************\")\n",
        "            print(f\"************* Now starting the Finetuning Application **************\")\n",
        "            print(f\"********************************************************************\\n\")\n",
        "        else:\n",
        "            print(f\"URL for port {port} not found.\")\n",
        "\n",
        "# Save the tunnel URLs to a JSON file.\n",
        "try:\n",
        "    # Try to open the JSON file for reading.\n",
        "    with open('/content/alltalk_tts/googlecolab.json', 'r') as f:\n",
        "        data = json.load(f)\n",
        "except FileNotFoundError:\n",
        "    # If the file doesn't exist, create an empty dictionary.\n",
        "    data = {}\n",
        "\n",
        "data['google_ip_address'] = tunnel_urls\n",
        "\n",
        "# Write the modified data (or newly created data) back to the file.\n",
        "with open('/content/alltalk_tts/googlecolab.json', 'w') as f:\n",
        "    json.dump(data, f)\n",
        "\n",
        "host = \"0.0.0.0\"\n",
        "\n",
        "if Tunnel == \"localtunnel\":\n",
        "    print(\"Before you copy the link above click on it and copy that ip:\")\n",
        "    !curl ipv4.icanhazip.com\n",
        "\n",
        "# Start API server.\n",
        "!python /content/alltalk_tts/finetune.py\n"
      ],
      "metadata": {
        "cellView": "form",
        "id": "Dewvq5s38Sfd"
      },
      "execution_count": null,
      "outputs": []
    }
  ],
  "metadata": {
    "accelerator": "GPU",
    "colab": {
      "gpuType": "T4",
      "provenance": []
    },
    "kernelspec": {
      "display_name": "Python 3",
      "name": "python3"
    },
    "language_info": {
      "name": "python"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
 }
@@ -1,11 +0,0 @@
 {
    "base_path": "models",
    "model_path": "xttsv2_2.0.2",
    "files_to_download": {
        "LICENSE.txt": "https://huggingface.co/coqui/XTTS-v2/resolve/v2.0.2/LICENSE.txt?download=true",
        "README.md": "https://huggingface.co/coqui/XTTS-v2/resolve/v2.0.2/README.md?download=true",
        "config.json": "https://huggingface.co/coqui/XTTS-v2/resolve/v2.0.2/config.json?download=true",
        "model.pth": "https://huggingface.co/coqui/XTTS-v2/resolve/v2.0.2/model.pth?download=true",
        "vocab.json": "https://huggingface.co/coqui/XTTS-v2/resolve/v2.0.2/vocab.json?download=true"
    }
 }
@@ -1,145 +0,0 @@
 import os
 from pathlib import Path
 import requests
 from tqdm import tqdm
 import importlib.metadata as metadata
 import json
 from packaging import version
 ##############################################################
 #### LOAD PARAMS FROM CONFIG.JSON - REQUIRED FOR BRANDING ####
 ##############################################################
 # STARTUP VARIABLE - Create "this_dir" variable as the current script directory
 this_dir = Path(__file__).parent.resolve()
 # load standard config file in and get settings
 def load_config(file_path):
    with open(file_path, "r") as config_file:
        config = json.load(config_file)
    return config
 config_file_path = this_dir / "config.json"
 params = load_config(config_file_path)
 # Define the path to the modeldownload config file file
 modeldownload_config_file_path = this_dir / "modeldownload.json"
 # Check if the JSON file exists
 if modeldownload_config_file_path.exists():
    with open(modeldownload_config_file_path, "r") as config_file:
        settings = json.load(config_file)
    # Extract settings from the loaded JSON
    base_path = Path(settings.get("base_path", ""))
    model_path = Path(settings.get("model_path", ""))
    files_to_download = settings.get("files_to_download", {})
 else:
    # Default settings if the JSON file doesn't exist or is empty
    print(
        f"[{params['branding']}Startup] \033[91mWarning\033[0m modeldownload.json is missing so please re-download it and save it in the coquii_tts main folder."
    )
    print(
        f"[{params['branding']}Startup] \033[91mWarning\033[0m API Local and XTTSv2 Local will error unless this is corrected."
    )
 # Read the version specifier from requirements_nvidia.txt
 with open(this_dir / "requirements_nvidia.txt", "r") as req_file:
    requirements = req_file.readlines()
 tts_version_required = None
 for req in requirements:
    if req.startswith("TTS>="):
        tts_version_required = req.strip().split(">=")[1]
        break
 if tts_version_required is None:
    raise ValueError(
        f"[{params['branding']}Startup] \033[91mWarning\033[0m Could not find TTS version specifier in requirements_nvidia.txt"
    )
 def create_directory_if_not_exists(directory):
    if not directory.exists():
        directory.mkdir(parents=True)
 def download_file(url, destination):
    response = requests.get(url, stream=True)
    total_size_in_bytes = int(response.headers.get("content-length", 0))
    block_size = 1024  # 1 Kibibyte
    progress_bar = tqdm(total=total_size_in_bytes, unit="iB", unit_scale=True)
    with open(destination, "wb") as file:
        for data in response.iter_content(block_size):
            progress_bar.update(len(data))
            file.write(data)
    progress_bar.close()
 def check_tts_version():
    try:
        tts_version = metadata.version("tts")
        print(
            f"[{params['branding']}Startup] TTS version installed: \033[93m"
            + tts_version
            + "\033[0m"
        )
        if version.parse(tts_version) < version.parse(tts_version_required):
            print(
                f"[{params['branding']}Startup] \033[91mWarning\033[0m TTS version is too old. Please upgrade to version \033[93m"
                + tts_version_required
                + "\033[0m or later.\033[0m"
            )
            print(
                f"[{params['branding']}Startup] \033[91mWarning\033[0m At your terminal/command prompt \033[94mpip install --upgrade tts\033[0m"
            )
        else:
            print(f"[{params['branding']}Startup] TTS version is up to date.")
    except metadata.PackageNotFoundError:
        print(
            f"[{params['branding']}Startup] \033[91mWarning\033[0m TTS is not installed."
        )
 # Check and create directories
 if str(base_path) == "models":
    create_directory_if_not_exists(this_dir / base_path / model_path)
 else:
    create_directory_if_not_exists(base_path / model_path)
    print(
        f"[{params['branding']}Startup] \033[94mInfo\033[0m Custom path set in \033[93mmodeldownload.json\033[0m. Using the following settings:"
    )
    print(
        f"[{params['branding']}Startup] \033[94mInfo\033[0m Base folder Path:\033[93m",
        base_path,
        "\033[0m",
    )
    print(
        f"[{params['branding']}Startup] \033[94mInfo\033[0m Model folder Path:\033[93m",
        model_path,
        "\033[0m",
    )
    print(
        f"[{params['branding']}Startup] \033[94mInfo\033[0m Full Path:\033[93m",
        base_path / model_path,
        "\033[0m",
    )
 # Download files if they don't exist
 print("[" + params["branding"] + "Startup] Checking Model is Downloaded.")
 for filename, url in files_to_download.items():
    if str(base_path) == "models":
        destination = this_dir / base_path / model_path / filename
    else:
        destination = Path(base_path) / model_path / filename
    if not destination.exists():
        print(f"[{params['branding']}Startup] Downloading " + filename + "...")
        download_file(url, destination)
 check_tts_version()
@@ -0,0 +1,2 @@
 [pytest]
 pythonpath = .
@@ -1,12 +0,0 @@
 numpy>=1.22.0
 soundfile>=0.12.1
 uvicorn>=0.24.0.post1
 TTS>=0.21.3
 torch>=2.1.0+cu118
 torchaudio>=2.1.0+cu118
 fastapi>=0.104.1
 Jinja2>=3.1.2
 requests>=2.31.0
 tqdm>=4.66.1
 importlib-metadata>=4.8.1
 packaging>=23.2
@@ -1,12 +0,0 @@
 numpy>=1.22.0
 soundfile>=0.12.1
 uvicorn>=0.24.0.post1
 TTS>=0.21.3
 torch>=2.1.0
 torchaudio>=2.1.0
 fastapi>=0.104.1
 Jinja2>=3.1.2
 requests>=2.31.0
 tqdm>=4.66.1
 importlib-metadata>=4.8.1
 packaging>=23.2
@@ -1,8 +0,0 @@
 .SDAP .hires_opts input[type="number"] {
    width: 6em !important;
 }
 /* silero_tts preview */
 .form:has(> #silero_preview_text) {
    min-width: 75%
 }
@@ -0,0 +1,20 @@
 BASIC SETUP
 -----------
 Place the `alltalk.js` in your `\SillyTavern\public\scripts\extensions\tts\` folder, replacing the existing file.
 CHANGING THE DEFAULT IP ADDRESS
 -------------------------------
 If you change the default IP/URL in the SillyTavern interface, you need to press Enter for it to save the setting. Just typing in an address/URL will not change the setting.
 SIMPLE TROUBLESHOOTING
 ----------------------
 The settings for AllTalk are stored, saved and managed by SillyTavern and not the AllTalk extension. You can manually edit OR delete these settings if necessary by editing the `settings.json` file that SillyTavern stores. I would suggest taking a copy of the file before making any changes.
 Older versions of SillyTavern store `settings.json` file in `\SillyTavern\public\`
 Newer version of SillyTavern store `settings.json` file in `\SillyTavern\data\default-user`
 Within the `settings.json` file you will find a section called `"AllTalk": {` and it is closed out a few lines down with `},`
 You can either manually edit any of the settings in there OR you can delete the entire section from the start of `"AllTalk":` to the end of `},` and when you refresh the SillyTavern web page OR restart SillyTavern, new settings for AllTalk will be created/saved.
@@ -0,0 +1 @@
 Place the `alltalk.js` in your `\SillyTavern\public\scripts\extensions\tts\` folder, replacing the existing file.
@@ -0,0 +1,834 @@
 import { doExtrasFetch, getApiUrl, modules } from '../../extensions.js';
 import { saveTtsProviderSettings } from './index.js';
 export { AllTalkTtsProvider };
 class AllTalkTtsProvider {
    //########//
    // Config //
    //########//
    settings = {};
    constructor() {
        // Initialize with default settings if they are not already set
        this.settings = {
            provider_endpoint: this.settings.provider_endpoint || 'http://localhost:7851',
            language: this.settings.language || 'en',
            voiceMap: this.settings.voiceMap || {},
            at_generation_method: this.settings.at_generation_method || 'standard_generation',
            narrator_enabled: this.settings.narrator_enabled || 'false',
            at_narrator_text_not_inside: this.settings.at_narrator_text_not_inside || 'narrator',
            narrator_voice_gen: this.settings.narrator_voice_gen || 'female_01.wav',
            finetuned_model: this.settings.finetuned_model || 'false'
        };
        // Separate property for dynamically updated settings from the server
        this.dynamicSettings = {
            modelsAvailable: [],
            currentModel: '',
            deepspeed_available: false,
            deepSpeedEnabled: false,
            lowVramEnabled: false,
        };
    }
    ready = false;
    voices = [];
    separator = '. ';
    audioElement = document.createElement('audio');
    languageLabels = {
        'Arabic': 'ar',
        'Brazilian Portuguese': 'pt',
        'Chinese': 'zh-cn',
        'Czech': 'cs',
        'Dutch': 'nl',
        'English': 'en',
        'French': 'fr',
        'German': 'de',
        'Italian': 'it',
        'Polish': 'pl',
        'Russian': 'ru',
        'Spanish': 'es',
        'Turkish': 'tr',
        'Japanese': 'ja',
        'Korean': 'ko',
        'Hungarian': 'hu',
        'Hindi': 'hi',
    };
    get settingsHtml() {
        let html = `<div class="at-settings-separator">AllTalk Settings</div>`;
        html += `<div class="at-settings-row">
        <div class="at-settings-option">
            <label for="at_generation_method">AllTalk TTS Generation Method</label>
                <select id="at_generation_method">
                <option value="standard_generation">Standard Audio Generation (AT Narrator - Optional)</option>
                <option value="streaming_enabled">Streaming Audio Generation (AT Narrator - Disabled)</option>
        </select>
        </div>
        </div>`;
        html += `<div class="at-settings-row">
        <div class="at-settings-option">
            <label for="at_narrator_enabled">AT Narrator</label>
                <select id="at_narrator_enabled">
                <option value="true">Enabled</option>
                <option value="false">Disabled</option>
        </select>
        </div>
        <div class="at-settings-option">
            <label for="at_narrator_text_not_inside">Text Not Inside * or " is</label>
                <select id="at_narrator_text_not_inside">
                <option value="narrator">Narrator</option>
                <option value="character">Character</option>
        </select>
        </div>
    </div>`;
        html += `<div class="at-settings-row">
        <div class="at-settings-option">
            <label for="narrator_voice">Narrator Voice</label>
            <select id="narrator_voice">`;
        if (this.voices) {
            for (let voice of this.voices) {
                html += `<option value="${voice.voice_id}">${voice.name}</option>`;
            }
        }
        html += `</select>
        </div>
        <div class="at-settings-option">
            <label for="language_options">Language</label>
            <select id="language_options">`;
        for (let language in this.languageLabels) {
            html += `<option value="${this.languageLabels[language]}" ${this.languageLabels[language] === this.settings?.language ? 'selected="selected"' : ''}>${language}</option>`;
        }
        html += `</select>
        </div>
    </div>`;
        html += `<div class="at-model-endpoint-row">
    <div class="at-model-option">
            <label for="switch_model">Switch Model</label>
            <select id="switch_model">
            <option value="api_tts">API TTS</option>
            <option value="api_local">API Local</option>
            <option value="xttsv2_local">XTTSv2 Local</option>
        </select>
        </div>
        <div class="at-endpoint-option">
            <label for="at_server">AllTalk Endpoint:</label>
            <input id="at_server" type="text" class="text_pole" maxlength="80" value="${this.settings.provider_endpoint}"/>
        </div>
   </div>`;
        html += `<div class="at-model-endpoint-row">
    <div class="at-settings-option">
        <label for="low_vram">Low VRAM</label>
        <input id="low_vram" type="checkbox"/>
    </div>
    <div class="at-settings-option">
        <label for="deepspeed">DeepSpeed</label>
        <input id="deepspeed" type="checkbox"/>
    </div>
    <div class="at-settings-option status-option">
        <span>Status: <span id="status_info">Ready</span></span>
    </div>
    <div class="at-settings-option empty-option">
        <!-- This div remains empty for spacing -->
    </div>
 </div>`;
        html += `<div class="at-website-row">
        <div class="at-website-option">
        <span>AllTalk <a target="_blank" href="${this.settings.provider_endpoint}">Config & Docs</a>.</span>
    </div>
    <div class="at-website-option">
        <span>AllTalk <a target="_blank" href="https://github.com/erew123/alltalk_tts/">Website</a>.</span>
    </div>
 </div>`;
        html += `<div class="at-website-row">
 <div class="at-website-option">
 <span><strong>Text-generation-webui</strong> users - Uncheck <strong>Enable TTS</strong> in Text-generation-webui.</span>
 </div>
 </div>`;
        return html;
    }
    //#################//
    // Startup ST & AT //
    //#################//
    async loadSettings(settings) {
        updateStatus('Offline');
        if (Object.keys(settings).length === 0) {
            console.info('Using default AllTalk TTS Provider settings');
        } else {
            // Populate settings with provided values, ignoring server-provided settings
            for (const key in settings) {
                if (key in this.settings) {
                    this.settings[key] = settings[key];
                } else {
                    console.debug(`Ignoring non-user-configurable setting: ${key}`);
                }
            }
        }
        // Update UI elements to reflect the loaded settings
        $('#at_server').val(this.settings.provider_endpoint);
        $('#language_options').val(this.settings.language);
        //$('#voicemap').val(this.settings.voiceMap);
        $('#at_generation_method').val(this.settings.at_generation_method);
        $('#at_narrator_enabled').val(this.settings.narrator_enabled);
        $('#at_narrator_text_not_inside').val(this.settings.at_narrator_text_not_inside);
        $('#narrator_voice').val(this.settings.narrator_voice_gen);
        console.debug('AllTalkTTS: Settings loaded');
        try {
            // Check if TTS provider is ready
            await this.checkReady();
            await this.updateSettingsFromServer(); // Fetch dynamic settings from the TTS server
            await this.fetchTtsVoiceObjects(); // Fetch voices only if service is ready
            this.updateNarratorVoicesDropdown();
            this.updateLanguageDropdown();
            this.setupEventListeners();
            this.applySettingsToHTML();
            updateStatus('Ready');
        } catch (error) {
            console.error("Error loading settings:", error);
            updateStatus('Offline');
        }
    }
    applySettingsToHTML() {
        // Apply loaded settings or use defaults
        const narratorVoiceSelect = document.getElementById('narrator_voice');
        const atNarratorSelect = document.getElementById('at_narrator_enabled');
        const textNotInsideSelect = document.getElementById('at_narrator_text_not_inside');
        const generationMethodSelect = document.getElementById('at_generation_method');
        this.settings.narrator_voice = this.settings.narrator_voice_gen;
        // Apply settings to Narrator Voice dropdown
        if (narratorVoiceSelect && this.settings.narrator_voice) {
            narratorVoiceSelect.value = this.settings.narrator_voice.replace('.wav', '');
        }
        // Apply settings to AT Narrator Enabled dropdown
        if (atNarratorSelect) {
            // Sync the state with the checkbox in index.js
            const ttsPassAsterisksCheckbox = document.getElementById('tts_pass_asterisks'); // Access the checkbox from index.js
            const ttsNarrateQuotedCheckbox = document.getElementById('tts_narrate_quoted'); // Access the checkbox from index.js
            const ttsNarrateDialoguesCheckbox = document.getElementById('tts_narrate_dialogues'); // Access the checkbox from index.js
            // Sync the state with the checkbox in index.js
            if (this.settings.narrator_enabled) {
                ttsPassAsterisksCheckbox.checked = false;
                $('#tts_pass_asterisks').click(); // Simulate a click event
                $('#tts_pass_asterisks').trigger('change');
            }
            if (!this.settings.narrator_enabled) {
                ttsPassAsterisksCheckbox.checked = true;
                $('#tts_pass_asterisks').click(); // Simulate a click event
                $('#tts_pass_asterisks').trigger('change');
            }
            // Uncheck and set tts_narrate_quoted to false if narrator is enabled
            if (this.settings.narrator_enabledd) {
                ttsNarrateQuotedCheckbox.checked = true;
                ttsNarrateDialoguesCheckbox.checked = true;
                // Trigger click events instead of change events
                $('#tts_narrate_quoted').click();
                $('#tts_narrate_quoted').trigger('change');
                $('#tts_narrate_dialogues').click();
                $('#tts_narrate_dialogues').trigger('change');
            }
            atNarratorSelect.value = this.settings.narrator_enabled.toString();
            this.settings.narrator_enabled = this.settings.narrator_enabled.toString();
        }
        // Apply settings to the Language dropdown
        const languageSelect = document.getElementById('language_options');
        if (languageSelect && this.settings.language) {
            languageSelect.value = this.settings.language;
        }
        // Apply settings to Text Not Inside dropdown
        if (textNotInsideSelect && this.settings.text_not_inside) {
            textNotInsideSelect.value = this.settings.text_not_inside;
            this.settings.at_narrator_text_not_inside = this.settings.text_not_inside;
        }
        // Apply settings to Generation Method dropdown
        if (generationMethodSelect && this.settings.at_generation_method) {
            generationMethodSelect.value = this.settings.at_generation_method;
        }
        // Additional logic to disable/enable dropdowns based on the selected generation method
        const isStreamingEnabled = this.settings.at_generation_method === 'streaming_enabled';
        if (isStreamingEnabled) {
            // Disable certain dropdowns when streaming is enabled
            if (atNarratorSelect) atNarratorSelect.disabled = true;
            if (textNotInsideSelect) textNotInsideSelect.disabled = true;
            if (narratorVoiceSelect) narratorVoiceSelect.disabled = true;
        } else {
            // Enable dropdowns for standard generation
            if (atNarratorSelect) atNarratorSelect.disabled = false;
            if (textNotInsideSelect) textNotInsideSelect.disabled = !this.settings.narrator_enabled;
            if (narratorVoiceSelect) narratorVoiceSelect.disabled = !this.settings.narrator_enabled;
        }
        const modelSelect = document.getElementById('switch_model');
        if (this.settings.finetuned_model === 'true') {
            const ftOption = document.createElement('option');
            ftOption.value = 'XTTSv2 FT';
            ftOption.textContent = 'XTTSv2 FT';
            modelSelect.appendChild(ftOption);
        }
    }
    //##############################//
    // Check AT Server is Available //
    //##############################//
    async checkReady() {
        try {
            const response = await fetch(`${this.settings.provider_endpoint}/api/ready`);
            // Check if the HTTP request was successful
            if (!response.ok) {
                throw new Error(`HTTP Error Response: ${response.status} ${response.statusText}`);
            }
            const statusText = await response.text();
            // Check if the response is 'Ready'
            if (statusText === 'Ready') {
                this.ready = true; // Set the ready flag to true
                console.log('TTS service is ready.');
            } else {
                this.ready = false;
                console.log('TTS service is not ready.');
            }
        } catch (error) {
            console.error('Error checking TTS service readiness:', error);
            this.ready = false; // Ensure ready flag is set to false in case of error
        }
    }
    //######################//
    // Get Available Voices //
    //######################//
    async fetchTtsVoiceObjects() {
        const response = await fetch(`${this.settings.provider_endpoint}/api/voices`);
        if (!response.ok) {
            const errorText = await response.text();
            throw new Error(`HTTP ${response.status}: ${errorText}`);
        }
        const data = await response.json();
        const voices = data.voices.map(filename => {
            const voiceName = filename.replace('.wav', '');
            return {
                name: voiceName,
                voice_id: voiceName,
                preview_url: null, // Preview URL will be dynamically generated
                lang: 'en' // Default language
            };
        });
        this.voices = voices; // Assign to the class property
        return voices; // Also return this list
    }
    //##########################################//
    // Get Current AT Server Config & Update ST //
    //##########################################//
    async updateSettingsFromServer() {
        try {
            // Fetch current settings
            const response = await fetch(`${this.settings.provider_endpoint}/api/currentsettings`);
            if (!response.ok) {
                throw new Error(`Failed to fetch current settings: ${response.statusText}`);
            }
            const currentSettings = await response.json();
            // Update internal settings
            this.settings.modelsAvailable = currentSettings.models_available;
            this.settings.currentModel = currentSettings.current_model_loaded;
            this.settings.deepspeed_available = currentSettings.deepspeed_available;
            this.settings.deepSpeedEnabled = currentSettings.deepspeed_status;
            this.settings.lowVramEnabled = currentSettings.low_vram_status;
            this.settings.finetuned_model = currentSettings.finetuned_model
            // Update HTML elements
            this.updateModelDropdown();
            this.updateCheckboxes();
        } catch (error) {
            console.error(`Error updating settings from server: ${error}`);
        }
    }
    //###################################################//
    // Get Current AT Server Config & Update ST (Models) //
    //###################################################//
    updateModelDropdown() {
        const modelSelect = document.getElementById('switch_model');
        if (modelSelect) {
            modelSelect.innerHTML = ''; // Clear existing options
            this.settings.modelsAvailable.forEach(model => {
                const option = document.createElement('option');
                option.value = model.model_name;
                option.textContent = model.model_name; // Use model_name instead of name
                option.selected = model.model_name === this.settings.currentModel;
                modelSelect.appendChild(option);
            });
        }
    }
    //#######################################################//
    // Get Current AT Server Config & Update ST (DS and LVR) //
    //#######################################################//
    updateCheckboxes() {
        const deepspeedCheckbox = document.getElementById('deepspeed');
        const lowVramCheckbox = document.getElementById('low_vram');
        if (lowVramCheckbox) lowVramCheckbox.checked = this.settings.lowVramEnabled;
        if (deepspeedCheckbox) {
            deepspeedCheckbox.checked = this.settings.deepSpeedEnabled;
            deepspeedCheckbox.disabled = !this.settings.deepspeed_available; // Disable checkbox if deepspeed is not available
        }
    }
    //###############################################################//
    // Get Current AT Server Config & Update ST (AT Narrator Voices) //
    //###############################################################//
    updateNarratorVoicesDropdown() {
        const narratorVoiceSelect = document.getElementById('narrator_voice');
        if (narratorVoiceSelect && this.voices) {
            // Clear existing options
            narratorVoiceSelect.innerHTML = '';
            // Add new options
            for (let voice of this.voices) {
                const option = document.createElement('option');
                option.value = voice.voice_id;
                option.textContent = voice.name;
                narratorVoiceSelect.appendChild(option);
            }
        }
    }
    //######################################################//
    // Get Current AT Server Config & Update ST (Languages) //
    //######################################################//
    updateLanguageDropdown() {
        const languageSelect = document.getElementById('language_options');
        if (languageSelect) {
            // Ensure default language is set
            this.settings.language = this.settings.language;
            languageSelect.innerHTML = '';
            for (let language in this.languageLabels) {
                const option = document.createElement('option');
                option.value = this.languageLabels[language];
                option.textContent = language;
                if (this.languageLabels[language] === this.settings.language) {
                    option.selected = true;
                }
                languageSelect.appendChild(option);
            }
        }
    }
    //########################################//
    // Start AT TTS extenstion page listeners //
    //########################################//
    setupEventListeners() {
        let debounceTimeout;
        const debounceDelay = 500; // Milliseconds
        // Define the event handler function
        const onModelSelectChange = async (event) => {
            console.log("Model select change event triggered"); // Debugging statement
            const selectedModel = event.target.value;
            console.log(`Selected model: ${selectedModel}`); // Debugging statement 
            // Set status to Processing
            updateStatus('Processing');
            try {
                const response = await fetch(`${this.settings.provider_endpoint}/api/reload?tts_method=${encodeURIComponent(selectedModel)}`, {
                    method: 'POST'
                });
                if (!response.ok) {
                    throw new Error(`HTTP Error: ${response.status}`);
                }
                const data = await response.json();
                console.log("POST response data:", data); // Debugging statement 
                // Set status to Ready if successful
                updateStatus('Ready');
            } catch (error) {
                console.error("POST request error:", error); // Debugging statement
                // Set status to Error in case of failure
                updateStatus('Error');
            }
            // Handle response or error
        };
        const debouncedModelSelectChange = (event) => {
            clearTimeout(debounceTimeout);
            debounceTimeout = setTimeout(() => {
                onModelSelectChange(event);
            }, debounceDelay);
        };
        // Switch Model Listener
        const modelSelect = document.getElementById('switch_model');
        if (modelSelect) {
            // Remove the event listener if it was previously added
            modelSelect.removeEventListener('change', debouncedModelSelectChange);
            // Add the debounced event listener
            modelSelect.addEventListener('change', debouncedModelSelectChange);
        }
        // DeepSpeed Listener
        const deepspeedCheckbox = document.getElementById('deepspeed');
        if (deepspeedCheckbox) {
            deepspeedCheckbox.addEventListener('change', async (event) => {
                const deepSpeedValue = event.target.checked ? 'True' : 'False';
                // Set status to Processing
                updateStatus('Processing');
                try {
                    const response = await fetch(`${this.settings.provider_endpoint}/api/deepspeed?new_deepspeed_value=${deepSpeedValue}`, {
                        method: 'POST'
                    });
                    if (!response.ok) {
                        throw new Error(`HTTP Error: ${response.status}`);
                    }
                    const data = await response.json();
                    console.log("POST response data:", data); // Debugging statement
                    // Set status to Ready if successful
                    updateStatus('Ready');
                } catch (error) {
                    console.error("POST request error:", error); // Debugging statement
                    // Set status to Error in case of failure
                    updateStatus('Error');
                }
            });
        }
        // Low VRAM Listener
        const lowVramCheckbox = document.getElementById('low_vram');
        if (lowVramCheckbox) {
            lowVramCheckbox.addEventListener('change', async (event) => {
                const lowVramValue = event.target.checked ? 'True' : 'False';
                // Set status to Processing
                updateStatus('Processing');
                try {
                    const response = await fetch(`${this.settings.provider_endpoint}/api/lowvramsetting?new_low_vram_value=${lowVramValue}`, {
                        method: 'POST'
                    });
                    if (!response.ok) {
                        throw new Error(`HTTP Error: ${response.status}`);
                    }
                    const data = await response.json();
                    console.log("POST response data:", data); // Debugging statement
                    // Set status to Ready if successful
                    updateStatus('Ready');
                } catch (error) {
                    console.error("POST request error:", error); // Debugging statement
                    // Set status to Error in case of failure
                    updateStatus('Error');
                }
            });
        }
        // Narrator Voice Dropdown Listener
        const narratorVoiceSelect = document.getElementById('narrator_voice');
        if (narratorVoiceSelect) {
            narratorVoiceSelect.addEventListener('change', (event) => {
                this.settings.narrator_voice_gen = `${event.target.value}.wav`;
                this.onSettingsChange(); // Save the settings after change
            });
        }
        const textNotInsideSelect = document.getElementById('at_narrator_text_not_inside');
        if (textNotInsideSelect) {
            textNotInsideSelect.addEventListener('change', (event) => {
                this.settings.text_not_inside = event.target.value;
                this.onSettingsChange(); // Save the settings after change
            });
        }
        // AT Narrator Dropdown Listener
        const atNarratorSelect = document.getElementById('at_narrator_enabled');
        const ttsPassAsterisksCheckbox = document.getElementById('tts_pass_asterisks'); // Access the checkbox from index.js
        const ttsNarrateQuotedCheckbox = document.getElementById('tts_narrate_quoted'); // Access the checkbox from index.js
        const ttsNarrateDialoguesCheckbox = document.getElementById('tts_narrate_dialogues'); // Access the checkbox from index.js
        if (atNarratorSelect && textNotInsideSelect && narratorVoiceSelect) {
            atNarratorSelect.addEventListener('change', (event) => {
                const isNarratorEnabled = event.target.value === 'true';
                this.settings.narrator_enabled = isNarratorEnabled; // Update the setting here
                textNotInsideSelect.disabled = !isNarratorEnabled;
                narratorVoiceSelect.disabled = !isNarratorEnabled;
                // Sync the state with the checkbox in index.js
                if (isNarratorEnabled) {
                    ttsPassAsterisksCheckbox.checked = false;
                    $('#tts_pass_asterisks').click(); // Simulate a click event
                    $('#tts_pass_asterisks').trigger('change');
                }
                if (!isNarratorEnabled) {
                    ttsPassAsterisksCheckbox.checked = true;
                    $('#tts_pass_asterisks').click(); // Simulate a click event
                    $('#tts_pass_asterisks').trigger('change');
                }
                // Uncheck and set tts_narrate_quoted to false if narrator is enabled
                if (isNarratorEnabled) {
                    ttsNarrateQuotedCheckbox.checked = true;
                    ttsNarrateDialoguesCheckbox.checked = true;
                    // Trigger click events instead of change events
                    $('#tts_narrate_quoted').click();
                    $('#tts_narrate_quoted').trigger('change');
                    $('#tts_narrate_dialogues').click();
                    $('#tts_narrate_dialogues').trigger('change');
                }
                this.onSettingsChange(); // Save the settings after change
            });
        }
        // Event Listener for AT Generation Method Dropdown
        const atGenerationMethodSelect = document.getElementById('at_generation_method');
        const atNarratorEnabledSelect = document.getElementById('at_narrator_enabled');
        if (atGenerationMethodSelect) {
            atGenerationMethodSelect.addEventListener('change', (event) => {
                const selectedMethod = event.target.value;
                if (selectedMethod === 'streaming_enabled') {
                    // Disable and unselect AT Narrator
                    atNarratorEnabledSelect.disabled = true;
                    atNarratorEnabledSelect.value = 'false';
                    textNotInsideSelect.disabled = true;
                    narratorVoiceSelect.disabled = true;
                } else if (selectedMethod === 'standard_generation') {
                    // Enable AT Narrator
                    atNarratorEnabledSelect.disabled = false;
                }
                this.settings.at_generation_method = selectedMethod; // Update the setting here
                this.onSettingsChange(); // Save the settings after change
            });
        }
        // Listener for Language Dropdown
        const languageSelect = document.getElementById('language_options');
        if (languageSelect) {
            languageSelect.addEventListener('change', (event) => {
                this.settings.language = event.target.value;
                this.onSettingsChange(); // Save the settings after change
            });
        }
        // Listener for AllTalk Endpoint Input
        const atServerInput = document.getElementById('at_server');
        if (atServerInput) {
            atServerInput.addEventListener('input', (event) => {
                this.settings.provider_endpoint = event.target.value;
                this.onSettingsChange(); // Save the settings after change
            });
        }
    }
    //#############################//
    // Store ST interface settings //
    //#############################//
    onSettingsChange() {
        // Update settings based on the UI elements
        //this.settings.provider_endpoint = $('#at_server').val();
        this.settings.language = $('#language_options').val();
        //this.settings.voiceMap = $('#voicemap').val();
        this.settings.at_generation_method = $('#at_generation_method').val();
        this.settings.narrator_enabled = $('#at_narrator_enabled').val();
        this.settings.at_narrator_text_not_inside = $('#at_narrator_text_not_inside').val();
        this.settings.narrator_voice_gen = $('#narrator_voice').val();
        // Save the updated settings
        saveTtsProviderSettings();
    }
    //#########################//
    // ST Handle Reload button //
    //#########################//
    async onRefreshClick() {
        await this.checkReady(); // Check if the TTS provider is ready
        await this.loadSettings(this.settings); // Reload the settings
        // Additional actions as needed
    }
    //##################//
    // Preview AT Voice //
    //##################//
    async previewTtsVoice(voiceName) {
        try {
            // Prepare data for POST request
            const postData = new URLSearchParams();
            postData.append("voice", `${voiceName}.wav`);
            // Making the POST request
            const response = await fetch(`${this.settings.provider_endpoint}/api/previewvoice/`, {
                method: "POST",
                headers: {
                    'Content-Type': 'application/x-www-form-urlencoded'
                },
                body: postData,
            });
            if (!response.ok) {
                const errorText = await response.text();
                console.error(`[previewTtsVoice] Error Response Text:`, errorText);
                throw new Error(`HTTP ${response.status}: ${errorText}`);
            }
            // Assuming the server returns a URL to the .wav file
            const data = await response.json();
            if (data.output_file_url) {
                // Use an audio element to play the .wav file
                const audioElement = new Audio(`${this.settings.provider_endpoint}${data.output_file_url}`);
                audioElement.play().catch(e => console.error("Error playing audio:", e));
            } else {
                console.warn("[previewTtsVoice] No output file URL received in the response");
                throw new Error("No output file URL received in the response");
            }
        } catch (error) {
            console.error("[previewTtsVoice] Exception caught during preview generation:", error);
            throw error;
        }
    }
    //#####################//
    //  Populate ST voices //
    //#####################//
    async getVoice(voiceName, generatePreview = false) {
        // Ensure this.voices is populated
        if (this.voices.length === 0) {
            // Fetch voice objects logic
        }
        // Find the object where the name matches voiceName
        const match = this.voices.find(voice => voice.name === voiceName);
        if (!match) {
            // Error handling
        }
        // Generate preview URL only if requested
        if (!match.preview_url && generatePreview) {
            // Generate preview logic
        }
        return match; // Return the found voice object
    }
    //##########################################//
    //  Generate TTS Streaming or call Standard //
    //##########################################//
    async generateTts(inputText, voiceId) {
        try {
            if (this.settings.at_generation_method === 'streaming_enabled') {
                // Construct the streaming URL
                const streamingUrl = `${this.settings.provider_endpoint}/api/tts-generate-streaming?text=${encodeURIComponent(inputText)}&voice=${encodeURIComponent(voiceId)}.wav&language=${encodeURIComponent(this.settings.language)}&output_file=stream_output.wav`;
                console.log("Streaming URL:", streamingUrl);
                // Return the streaming URL directly
                return streamingUrl; 
            } else {
                // For standard method
                const outputUrl = await this.fetchTtsGeneration(inputText, voiceId);
                const audioResponse = await fetch(outputUrl);
                if (!audioResponse.ok) {
                    throw new Error(`HTTP ${audioResponse.status}: Failed to fetch audio data`);
                }
                return audioResponse; // Return the fetch response directly
            }
        } catch (error) {
            console.error("Error in generateTts:", error);
            throw error;
        }
    }
    //####################//
    //  Generate Standard //
    //####################//
    async fetchTtsGeneration(inputText, voiceId) {
        // Prepare the request payload
        const requestBody = new URLSearchParams({
            'text_input': inputText,
            'text_filtering': "standard",
            'character_voice_gen': voiceId + ".wav",
            'narrator_enabled': this.settings.narrator_enabled,
            'narrator_voice_gen': this.settings.narrator_voice_gen + ".wav",
            'text_not_inside': this.settings.at_narrator_text_not_inside,
            'language': this.settings.language,
            'output_file_name': "st_output",
            'output_file_timestamp': "true",
            'autoplay': "false",
            'autoplay_volume': "0.8"
        }).toString();
        try {
            const response = await doExtrasFetch(
                `${this.settings.provider_endpoint}/api/tts-generate`,
                {
                    method: 'POST',
                    headers: {
                        'Content-Type': 'application/x-www-form-urlencoded',
                        'Cache-Control': 'no-cache',
                    },
                    body: requestBody
                }
            );
            if (!response.ok) {
                const errorText = await response.text();
                console.error(`[fetchTtsGeneration] Error Response Text:`, errorText);
                // toastr.error(response.statusText, 'TTS Generation Failed');
                throw new Error(`HTTP ${response.status}: ${errorText}`);
            }
            const data = await response.json();
            //const outputUrl = data.output_file_url;
            const outputUrl = `${this.settings.provider_endpoint}${data.output_file_url}`;
            return outputUrl; // Return only the output_file_url
        } catch (error) {
            console.error("[fetchTtsGeneration] Exception caught:", error);
            throw error; // Rethrow the error for further handling
        }
    }
 }
 //#########################//
 //  Update Status Messages //
 //#########################//
 function updateStatus(message) {
    const statusElement = document.getElementById('status_info');
    if (statusElement) {
        statusElement.textContent = message;
        switch (message) {
            case 'Offline':
                statusElement.style.color = 'red';
                break;
            case 'Ready':
                statusElement.style.color = 'lightgreen';
                break;
            case 'Processing':
                statusElement.style.color = 'blue';
                break;
            case 'Error':
                statusElement.style.color = 'red';
                break;
        }
    }
 }
@@ -0,0 +1,17 @@
 Text-generation-webui remote extension for AllTalk! 
 This folder contains everything you need to set up a Text-generation-webui extension that can communicate with an AllTalk server remotely.
 To get started, follow these simple steps:
 1) In your Text-generation-webui's extensions folder, create a new folder and name it "alltalk_tts".
 2) Take all the files from this current folder and copy them into the newly created "alltalk_tts" folder.
 3) Start & Open Text-generation-webui and look for "alltalk_tts" in the list of extensions. Select alltalk_tts, save your settings and restart Text-generation-webui.
 4) Text-generation-webui will restart and load the AllTalk extension. You may see an error/warning as the extension will not yet have the IP address details of your remote AllTalk server.
 5) In the extension settings, enter the correct IP address and port of your AllTalk server. If your AllTalk server is running on a remote machine accessed through a secure tunnel, make sure to use "https" instead of "http".
 6) After entering the server details, click on the "Refresh settings & voices" button. If the settings are correct, the extension will download all the available settings from the AllTalk server.
 7) The first time you successfully generate speech using the extension, the settings will be automatically saved to a local configuration file.
 Please note that if the AllTalk server you're connected to doesn't support certain settings/features, those options will be greyed out and unavailable in the extension.
 Please use the "Refresh settings & voices" button to make sure the extension is in sync with your AllTalk server.
@@ -0,0 +1,29 @@
 {
    "branding": "AllTalk ",
    "tgwui": {
        "tgwui_activate_tts": true,
        "tgwui_autoplay_tts": true,
        "tgwui_narrator_enabled": "false",
        "tgwui_non_quoted_text_is": "silent",
        "tgwui_deepspeed_enabled": false,
        "tgwui_language": "English",
        "tgwui_lowvram_enabled": false,
        "tgwui_pitch_set": 0,
        "tgwui_temperature_set": 0.75,
        "tgwui_repetitionpenalty_set": 10,
        "tgwui_generationspeed_set": 1,
        "tgwui_narrator_voice": "en_US-ljspeech-high.onnx",
        "tgwui_show_text": true,
        "tgwui_character_voice": "en_US-ljspeech-high.onnx",
        "tgwui_rvc_char_voice": "Disabled",
        "tgwui_rvc_char_pitch": 0,
        "tgwui_rvc_narr_voice": "Disabled",
        "tgwui_rvc_narr_pitch": 0
    },
    "remote_connection": {
        "server_protocol": "http://",
        "server_address": "127.0.0.1:7851",
        "connection_timeout": 5,
        "use_legacy_api": false
    }
 }
@@ -0,0 +1,865 @@
 <!DOCTYPE html>
 <html lang="en">
 <head>
    <link rel="icon" href="./static/favicon.ico" type="image/x-icon">
    <meta charset="UTF-8">
    <meta name="viewport" content="width=device-width, initial-scale=1.0">
    <title>AllTalk TTS</title>
    <style>
        body {
            font-family: Arial, sans-serif;
            max-width: 1200px;
            margin: 40px auto;
            padding: 20px;
            background-color: #f4f4f4;
            border: 1px solid #ddd;
            border-radius: 8px;
            box-shadow: 0 0 10px rgba(0, 0, 0, 0.1);
        }
        h1,
        h2 {
            color: black;
            text-decoration: underline;
        }
        .form-row {
            display: flex;
            flex-wrap: wrap;
            margin-bottom: 15px;
        }
        .form-group {
            flex: 1;
            margin-right: 10px;
            min-width: 220px;
            /* Ensure a minimum width for the form groups */
        }
        .form-group label {
            display: block;
            margin-bottom: 5px;
        }
        .form-group select,
        .form-group input[type="text"],
        .form-group textarea {
            width: 100%;
            padding: 5px;
            box-sizing: border-box;
            /* Include padding and border in the element's total width and height */
        }
        .button-group {
            flex-basis: 100%;
            /* Ensure button groups take up the full width */
            display: flex;
            justify-content: flex-start;
            /* Align buttons to the start */
            align-items: center;
            /* Adjusted to center align buttons */
            gap: 10px;
            /* Add space between buttons */
            margin-top: 10px;
        }
        .button-group button {
            padding: 10px 20px;
            font-size: 14px;
            border: none;
            border-radius: 4px;
            cursor: pointer;
            background-color: #4caf50;
            color: white;
            transition: background-color 0.3s;
            flex: 1;
            /* Allow buttons to take up equal space */
        }
        button:hover {
            background-color: #45a049;
        }
        #refreshSettingsBtn {
            background-color: #3498db;
            color: #fff;
        }
        #submit {
            padding: 10px 20px;
            font-size: 16px;
            font-weight: bold;
            width: 100%;
            /* Make the submit button take the full width */
            margin-top: 10px;
        }
        #outputFile,
        #modelSelect {
            width: 100%;
            /* Set to full width */
            box-sizing: border-box;
            /* Include padding and border in the element's total width and height */
        }
        #outputFile {
            margin-right: 10px;
        }
        p,
        span {
            color: #555;
            font-size: 16px;
            margin-top: 0;
        }
        code {
            font-family: Arial, sans-serif;
            background-color: #e9e9e9;
            border: none;
            box-shadow: none;
            outline: none;
            color: #3366ff;
            font-size: 16px;
        }
        pre {
            background-color: #f8f8f8;
            border: 1px solid #ddd;
            border-radius: 4px;
            padding: 10px;
            overflow-x: auto;
            font-size: 14px;
        }
        ul {
            color: #555;
            list-style-type: square;
            margin-left: 2px;
        }
        li {
            font-size: 16px;
            margin-bottom: 8px;
        }
        a {
            color: #0077cc;
            text-decoration: none;
        }
        a:hover {
            text-decoration: underline;
        }
        strong {
            font-weight: bold;
        }
        #container {
            max-width: 1000px;
            margin: 50px auto;
            padding: 20px;
            background-color: #fff;
            border-radius: 8px;
            box-shadow: 0 0 10px rgba(0, 0, 0, 0.1);
        }
        form {
            display: flex;
            flex-direction: column;
            gap: 4px;
        }
        #outputFileContainer {
            display: flex;
            flex-direction: column;
            gap: 4px;
        }
        label {
            font-weight: bold;
            font-size: 14px;
            padding: 2px;
        }
        textarea,
        input,
        select,
        button {
            padding: 4px;
            font-size: 14px;
            border: 1px solid #ccc;
            border-radius: 4px;
        }
        textarea {
            resize: vertical;
        }
        p {
            margin-top: 20px;
        }
        #outputFilePath {
            font-weight: bold;
        }
        #audioSource {
            display: block;
            margin: auto;
        }
        #outputFilePath {
            display: none;
        }
        table {
            border-collapse: collapse;
            width: 100%;
            margin-bottom: 20px;
        }
        th,
        td {
            border: 1px solid #ddd;
            padding: 8px;
            text-align: left;
        }
        /* Responsive styling */
        @media (max-width: 768px) {
            .form-group {
                flex: 1 1 100%;
                /* Make each form group take full width on small screens */
                margin-right: 0;
            }
            .button-group {
                flex-direction: column;
                /* Stack buttons vertically on small screens */
                align-items: stretch;
            }
            .button-group button {
                width: 100%;
            }
        }
    </style>
    <script>
        // Declare variables in the global scope
        let protocol, host, port, cleanedBaseUrl, currentUrl;
        document.addEventListener('DOMContentLoaded', function () {
            // Use URL constructor for robust parsing
            currentUrl = new URL(window.location.href);
            console.log('CurrentURL', currentUrl.href);
            protocol = currentUrl.protocol; // "http:" or "https:"
            host = currentUrl.hostname; // "127.0.0.1"
            port = currentUrl.port || (protocol === 'https:' ? '443' : '80'); // "7851" or default ports
            const basePath = currentUrl.pathname.split('/')[1] ? `/${currentUrl.pathname.split('/')[1]}` : '';
            console.log('Protocol:', protocol);
            console.log('Host:', host);
            console.log('Port:', port);
            console.log('Base Path:', basePath);
            cleanedBaseUrl = `${protocol}//${host}:${port}${basePath}`;
            console.log('CleanedBaseURL', cleanedBaseUrl);
            // Fetch settings and populate the form, then set the URLs
            fetchSettings().then(() => {
                setUrls();
            }).catch(console.error);
        });
        async function fetchSettings() {
            try {
                const response = await fetch('/settings-json');
                console.log("Response is:", response);
                if (!response.ok) {
                    throw new Error('Network response was not ok ' + response.statusText);
                }
                const data = await response.json();
                console.log("Data is:", data);
                populateForm(data);
            } catch (error) {
                console.error('Failed to fetch settings:', error);
                throw error;
            }
        }
        function populateForm(data) {
            return new Promise((resolve, reject) => {
                try {
                    const deleteOutputWavsElem = document.getElementById('delete_output_wavs');
                    const gradioInterfaceElem = document.getElementById('gradio_interface');
                    const gradioPortNumberElem = document.getElementById('gradio_port_number');
                    const apiPortNumberElem = document.getElementById('api_port_number');
                    if (deleteOutputWavsElem) {
                        deleteOutputWavsElem.value = data.delete_output_wavs;
                    }
                    if (gradioInterfaceElem) {
                        gradioInterfaceElem.value = data.gradio_interface ? 'true' : 'false';
                    }
                    if (gradioPortNumberElem) {
                        gradioPortNumberElem.value = data.gradio_port_number;
                        console.log("Populated gradioPortNumberElem.value:", gradioPortNumberElem.value); // Add logging here
                    }
                    if (apiPortNumberElem) {
                        apiPortNumberElem.value = data.api_def.api_port_number;
                    }
                    resolve();
                } catch (error) {
                    reject(error);
                }
            });
        }
        // Function to set the URLs
        function setUrls() {
            const ttsGeneratorLink = document.getElementById('ttsGeneratorLink');
            ttsGeneratorLink.href = `${cleanedBaseUrl}/static/tts_generator/tts_generator.html`;
            ttsGeneratorLink.textContent = 'TTS Generator';
            const GradioLink = document.getElementById('GradioLink');
            const GradioLinkDark = document.getElementById('GradioLinkDark');
            const gradioPortNumberElem = document.getElementById('gradio_port_number');
            if (gradioPortNumberElem && gradioPortNumberElem.value) {
                GradioLink.href = `${protocol}//${host}:${gradioPortNumberElem.value}`;
                GradioLinkDark.href = `${protocol}//${host}:${gradioPortNumberElem.value}?__theme=dark`;
            } else {
                GradioLink.href = `${protocol}//${host}:7852/gradio`;
                GradioLinkDark.href = `${protocol}//${host}:7852/gradio?__theme=dark`;
                console.error('Gradio port number element not found or empty.');
            }
            GradioLink.textContent = 'Gradio (Light theme)';
            GradioLinkDark.textContent = 'Gradio (Dark theme)';
        }
    </script>
 </head>
 <body>
    <h1 id="toc">AllTalk TTS</h1>
    <hr>
    <div style="display: flex; justify-content: flex-start; align-items: flex-start;">
        <div style="flex: 1; padding-left: 20px;">
            <h3>Useful links</h3>
            <ul>
                <li><a href="" id="ttsGeneratorLink" target="_blank" rel="noopener">TTS Generator</a></li>
                <li><a href="" id="GradioLink" target="_blank" rel="noopener">Gradio (Light theme)</a></li>
                <li><a href="" id="GradioLinkDark" target="_blank" rel="noopener">Gradio (Dark theme)</a></li>
                <li>Documentation is inside the Gradio interface</li>
                </li>
            </ul>
        </div>
        <div style="flex: 1; padding-right: 20px;">
            <h3 id="index">Support Links</h3>
            <ul>
                <li><a href="https://github.com/erew123/alltalk_tts" target="_blank" rel="noopener">Github</a>
                </li>
                <li><a href="https://github.com/erew123/alltalk_tts/issues/25" target="_blank"
                        rel="noopener">Changelog</a></li>
                <li><a href="https://github.com/erew123/alltalk_tts/issues/74" target="_blank" rel="noopener">Feature
                        Requests</a></li>
                <li><a href="https://github.com/erew123/alltalk_tts?#-help-with-problems" target="_blank"
                        rel="noopener">Help Documents</a></li>
            </ul>
        </div>
    </div>
    <hr>
    <div id="container">
        <form method="post" action="/tts-demo-request" id="ttsForm">
            <div class="form-row">
                <div class="form-group">
                    <label for="text">Text:</label>
                    <textarea id="text" name="text" rows="4" required></textarea>
                </div>
            </div>
            <div class="form-row">
                <div class="form-group">
                    <label for="voice">Voice:</label>
                    <select id="voice" name="voice" required></select>
                </div>
                <div class="form-group">
                    <label for="rvcvoice">RVC Voice:</label>
                    <select id="rvcvoice" name="rvccharacter_voice_gen" required></select>
                </div>
                <div class="form-group">
                    <label for="language">Language:</label>
                    <select id="language" name="language" value="English" required>
                        <option value="en" selected>English</option>
                        <option value="ar">Arabic</option>
                        <option value="zh-cn">Chinese</option>
                        <option value="cs">Czech</option>
                        <option value="nl">Dutch</option>
                        <option value="fr">French</option>
                        <option value="de">German</option>
                        <option value="hu">Hungarian</option>
                        <option value="it">Italian</option>
                        <option value="ja">Japanese</option>
                        <option value="ko">Korean</option>
                        <option value="pl">Polish</option>
                        <option value="pt">Portuguese</option>
                        <option value="ru">Russian</option>
                        <option value="es">Spanish</option>
                        <option value="tr">Turkish</option>
                    </select>
                </div>
                <div class="form-group">
                    <label for="streaming">Streaming:</label>
                    <select id="streaming" name="streaming" required>
                        <option value="false" selected>No</option>
                        <option value="true">Yes</option>
                    </select>
                </div>
            </div>
            <div class="form-row">
                <div class="form-group">
                    <label for="outputFile">Output Filename:</label>
                    <input type="text" id="outputFile" name="output_file" value="demo_output" required>
                </div>
                <div class="form-group">
                    <label for="engineSelect">Available Engines:</label>
                    <select id="engineSelect" name="engineSelect" required></select>
                </div>
                <div class="form-group">
                    <label for="modelSelect">Available Models:</label>
                    <select id="modelSelect" name="modelSelect" required></select>
                </div>
            </div>
            <div class="form-row button-group">
                <button id="refreshSettingsBtn" type="button">Refresh Settings</button>
                <button id="changeEngineBtn" type="button">Change Engine</button>
                <button id="changeModelBtn" type="button">Change Model</button>
            </div>
            <div class="form-row">
                <button id="submit" type="submit">Generate TTS</button>
            </div>
            <audio controls autoplay id="audioSource">
                <source type="audio/wav" />
                Your browser does not support the audio element.
            </audio>
            <span id="outputFilePath">{{ output_file_path }}</span>
        </form>
    </div>
    <div id="container">
        <form method="post" action="/update-settings" id="settingsForm" enctype="application/x-www-form-urlencoded">
            <div class="form-row">
                <div class="form-group">
                    <label for="delete_output_wavs"">Del WAV's older than:</label>
                    <select id=" delete_output_wavs" name="delete_output_wavs" required>
                        <option value="Disabled">Disabled</option>
                        <option value="1 Day">1 Day</option>
                        <option value="2 Days">2 Days</option>
                        <option value="3 Days">3 Days</option>
                        <option value="4 Days">4 Days</option>
                        <option value="5 Days">5 Days</option>
                        <option value="6 Days">6 Days</option>
                        <option value="7 Days">7 Days</option>
                        <option value="14 Days">14 Days</option>
                        <option value="21 Days">21 Days</option>
                        <option value="28 Days">28 Days</option>
                        </select>
                </div>
                <div class="form-group">
                    <label for="api_port_number">API Port Number:</label>
                    <input type="number" id="api_port_number" name="api_port_number" required
                        style="width: 228px; height: 20px;">
                </div>
                <div class="form-group">
                    <label for="gradio_interface">Gradio Interface:</label>
                    <select id="gradio_interface" name="gradio_interface" required>
                        <option value="true">Enabled</option>
                        <option value="false">Disabled</option>
                    </select>
                </div>
                <div class="form-group">
                    <label for="gradio_port_number">Gradio Port Number:</label>
                    <input type="number" id="gradio_port_number" name="gradio_port_number" required
                        style="width: 228px; height: 20px;">
                </div>
            </div>
            <div class="form-row">
                <button id="submit" type="submit">Update Settings</button>
            </div>
        </form>
    </div>
    <hr>
    <h3 id="🛠️-about-this-project">🛠️ <strong>About this project</strong></h3>
    <p>AllTalk is a labour of love, developed and supported in my personal free time. As such, my ability to respond to
        support requests is limited. I prioritize issues based on their impact and the number of users affected. I
        appreciate your understanding and patience. If your inquiry isn&#39;t covered by the documentation or existing
        discussions, and it&#39;s not related to a bug or feature request, I&#39;ll do my best to assist as time allows.
    </p>
    <hr>
    <h3 id="💖-showing-your-support">💖 Showing Your Support</h3>
    <p>If AllTalk has been helpful to you, consider showing your support through a donation on my <a
            href="https://ko-fi.com/erew123">Ko-fi page</a>. Your support is greatly appreciated and helps ensure the
        continued development and improvement of AllTalk.</p>
    <hr>
    <h3 id="⚠️-showing-your-support">⚠️ Changing Available Engine</h3>
    <div>
        <p>Changing the <b>Available Engine</b> on this web page will cause the Gradio interface to become
            unsynchronized with the TTS Engine process <b>until</b> AllTalk is restarted. This may result in issues with
            the Gradio interface, including warnings such as <b><i>[Errno 10048] error while attempting to bind on
                    address</i></b> or <b><i>runner</i></b> errors when exiting AllTalk. For best results, please use
            the Gradio interface to change and manage your <b>Available Engine</b>"
        </p>
    </div>
    </div>
    <script>
        document.addEventListener('DOMContentLoaded', function () {
            // Use URL constructor for robust parsing
            const currentUrl = new URL(window.location.href);
            console.log('CurrentURL', currentUrl.href);
            protocol = currentUrl.protocol; // "http:" or "https:"
            host = currentUrl.hostname; // "127.0.0.1"
            port = currentUrl.port || (protocol === 'https:' ? '443' : '80'); // "7851" or default ports
            // Determine the base path by taking only the first part of the path
            const pathParts = currentUrl.pathname.split('/');
            const basePath = pathParts.length > 1 && pathParts[1] ? `/${pathParts[1]}` : '';
            console.log('Protocol:', protocol);
            console.log('Host:', host);
            console.log('Port:', port);
            console.log('Base Path:', basePath);
            cleanedBaseUrl = `${protocol}//${host}:${port}`;
            console.log('CleanedBaseURL', cleanedBaseUrl);
            setUrls();
        });
        document.addEventListener('DOMContentLoaded', function () {
            const engineSelect = document.getElementById('engineSelect');
            const modelSelect = document.getElementById('modelSelect');
            const changeEngineBtn = document.getElementById('changeEngineBtn');
            const changeModelBtn = document.getElementById('changeModelBtn');
            const refreshSettingsBtn = document.getElementById('refreshSettingsBtn');
            console.log('Protocol:', protocol);
            console.log('Host:', host);
            console.log('Port:', port);
            // Function to dynamically construct the URL for fetching voices
            function getVoicesUrl() {
                return `${cleanedBaseUrl}/api/voices`;
            }
            // Function to dynamically construct the URL for fetching RVC voices
            function getrvcVoicesUrl() {
                return `${cleanedBaseUrl}/api/rvcvoices`;
            }
            // Function to fetch current settings and populate the dropdown
            function fetchAndPopulateSettings() {
                fetch(`${cleanedBaseUrl}/api/currentsettings`)
                    .then(response => response.json())
                    .then(data => {
                        // Clear existing options
                        engineSelect.innerHTML = '';
                        modelSelect.innerHTML = '';
                        // Populate the dropdown with available engines
                        data.engines_available.forEach(engine => {
                            const option = document.createElement('option');
                            option.value = engine;
                            option.textContent = engine;
                            engineSelect.appendChild(option);
                        });
                        // Populate the dropdown with available models
                        data.models_available.forEach(model => {
                            const option = document.createElement('option');
                            option.value = model.name;
                            option.textContent = model.name;
                            modelSelect.appendChild(option);
                        });
                        // Set the current loaded engine and model as selected
                        engineSelect.value = data.current_engine_loaded;
                        modelSelect.value = data.current_model_loaded;
                    })
                    .catch(error => {
                        console.error('Error fetching current settings:', error);
                        // Retry fetching settings after 5 seconds on error
                        setTimeout(fetchAndPopulateSettings, 5000);
                    });
            }
            // Function to reload voices
            function reloadVoices() {
                // Clear existing options
                document.getElementById('voice').innerHTML = '';
                document.getElementById('rvcvoice').innerHTML = '';
                // Fetch voices and populate the dropdown
                fetch(getVoicesUrl())
                    .then(response => response.json())
                    .then(data => {
                        const voiceSelect = document.getElementById('voice');
                        data.voices.forEach(voice => {
                            const option = document.createElement('option');
                            option.value = voice;
                            option.textContent = voice;
                            voiceSelect.appendChild(option);
                        });
                        resetChangeEngineBtn();
                    })
                    .catch(error => console.error('Error fetching voices:', error));
                // Fetch RVC voices and populate the dropdown
                fetch(getrvcVoicesUrl())
                    .then(response => response.json())
                    .then(data => {
                        const rvcVoiceSelect = document.getElementById('rvcvoice');
                        data.rvcvoices.forEach(rvcvoice => {
                            const option = document.createElement('option');
                            option.value = rvcvoice;
                            option.textContent = rvcvoice;
                            rvcVoiceSelect.appendChild(option);
                        });
                        resetChangeEngineBtn();
                    })
                    .catch(error => console.error('Error fetching RVC voices:', error));
            }
            // Function to check if engine is ready
            function checkEngineReady() {
                fetch(`${cleanedBaseUrl}/api/ready`, {
                    method: 'GET'
                })
                    .then(response => response.text())
                    .then(data => {
                        if (data === 'Ready') {
                            console.log('Engine is ready');
                            reloadVoices(); // Call the reloadVoices function
                            fetchAndPopulateSettings(); // Refresh settings when engine is ready
                        } else {
                            console.log('Engine not ready yet, rechecking...');
                            setTimeout(() => checkEngineReady(), 1000); // Recheck after 1 second
                        }
                    })
                    .catch(error => {
                        console.error('Error checking engine readiness:', error);
                        setTimeout(() => checkEngineReady(), 1000); // Recheck after 1 second even on error
                    });
            }
            // Function to change the engine button to "Changing Engine" state
            function setChangeEngineBtnChanging() {
                changeEngineBtn.style.backgroundColor = 'gray';
                changeEngineBtn.textContent = 'Changing Engine';
                changeEngineBtn.disabled = true;
            }
            // Function to reset the engine button to its original state
            function resetChangeEngineBtn() {
                changeEngineBtn.style.backgroundColor = '#4caf50';
                changeEngineBtn.textContent = 'Change Engine';
                changeEngineBtn.disabled = false;
            }
            // Function to change the model button to "Changing Model" state
            function setChangeModelBtnChanging() {
                changeModelBtn.style.backgroundColor = 'gray';
                changeModelBtn.textContent = 'Changing Model';
                changeModelBtn.disabled = true;
            }
            // Function to reset the model button to its original state
            function resetChangeModelBtn() {
                changeModelBtn.style.backgroundColor = '#4caf50';
                changeModelBtn.textContent = 'Change Model';
                changeModelBtn.disabled = false;
            }
            // Event listener for the "Change Engine" button
            changeEngineBtn.addEventListener('click', function () {
                const selectedEngine = engineSelect.value;
                setChangeEngineBtnChanging(); // Set button to "Changing Engine" state
                fetch(`${cleanedBaseUrl}/api/enginereload?engine=${encodeURIComponent(selectedEngine)}`, {
                    method: 'POST'
                })
                    .catch(error => console.error('Error sending engine reload request:', error));
                console.log('Engine reload request sent, waiting 15 seconds before checking readiness...');
                setTimeout(() => {
                    checkEngineReady();
                }, 15000); // Wait 15 seconds before starting to check
            });
            // Event listener for the "Change Model" button
            changeModelBtn.addEventListener('click', function () {
                const selectedModel = modelSelect.value;
                setChangeModelBtnChanging(); // Set button to "Changing Model" state
                fetch(`${cleanedBaseUrl}/api/reload?tts_method=${encodeURIComponent(selectedModel)}`, {
                    method: 'POST'
                })
                    .then(response => response.json())
                    .then(data => {
                        if (data.status === 'model-success') {
                            console.log('Model changed successfully');
                            setTimeout(() => {
                                reloadVoices(); // Call the reloadVoices function after a delay
                                fetchAndPopulateSettings(); // Refresh settings
                            }, 12000); // Wait 12 second before refreshing settings and voices
                        } else {
                            console.error('Failed to change model');
                            resetChangeModelBtn(); // Reset button to original state
                        }
                    })
                    .catch(error => {
                        console.error('Error changing model:', error);
                        resetChangeModelBtn(); // Reset button to original state
                    });
            });
            // Event listener for the "Refresh Settings" button
            refreshSettingsBtn.addEventListener('click', function () {
                fetchAndPopulateSettings();
                reloadVoices();
                resetChangeEngineBtn(); // Reset button to original state
                resetChangeModelBtn(); // Reset button to original state
            });
            // Fetch settings and populate the dropdown on page load
            fetchAndPopulateSettings();
            reloadVoices();
        });
    </script>
    <script>
        // Audio player
        const audioPlayer = document.getElementById('audioSource');
        function enableLoader(enable) {
            // Change the submit button text and disable it while loading
            const submit = document.getElementById('submit');
            submit.disabled = enable ? true : false;
            submit.style.opacity = enable ? 0.7 : 1.0;
            submit.innerText = enable ? 'Generating...' : 'Generate TTS';
            // Disable the audio player while loading
            audioPlayer.disabled = enable ? true : false;
            audioPlayer.style.opacity = enable ? 0.7 : 1.0;
            audioPlayer.style.pointerEvents = enable ? 'none' : 'auto';
        }
        audioPlayer.addEventListener('canplay', () => {
            enableLoader(false);
            audioPlayer.play();
        });
        audioPlayer.addEventListener('abort', () => {
            enableLoader(false);
        });
        audioPlayer.addEventListener('error', (e) => {
            enableLoader(false);
        });
        // Streaming selector
        const streaming = document.getElementById('streaming');
        streaming.addEventListener('change', (event) => {
            const outputFileContainer = document.getElementById('outputFileContainer');
            if (event.target.value === 'true') {
                outputFileContainer.style.display = 'none';
            } else {
                outputFileContainer.style.display = 'flex';
            }
        });
        // Form submit event listener
        const form = document.getElementById('ttsForm');
        form.addEventListener('submit', async (event) => {
            event.preventDefault();
            enableLoader(true);
            audioPlayer.pause();
            const formData = new FormData(form);
            // Get and clean the text input from the textarea
            let textInput = formData.get('text');
            let cleanedText = textInput.replace(/ \- | \– /g, ' '); // Replace hyphens
            cleanedText = cleanedText.replace(/%/g, " percent");
            //cleanedText = cleanedText.replace(/([!?.])\1+/g, '$1');
            //cleanedText = cleanedText.replace(/[^a-zA-Z0-9\s\.,;:!?\-\'"À-ÿ\u0400-\u04FF]/g, '');
            //cleanedText = cleanedText.replace(/[^a-zA-Z0-9\s\.,;:!?\-\'"À-ÿ\u0400-\u04FF]\$/g, '')
            cleanedText = cleanedText.replace(/[^a-zA-Z0-9\s.,;:!?\-\'"$\u0400-\u04FF\u00C0-\u00FF\u0150\u0151\u0170\u0171\u011E\u011F\u0130\u0131\u015E\u015F\u00E7\u00C7\u00F6\u00D6\u0900-\u097F\u2018\u2019\u201C\u201D\u3001\u3002\u3040-\u309F\u30A0-\u30FF\u4E00-\u9FFF\u3400-\u4DBF\uF900-\uFAFF\u0600-\u06FF\u0750-\u077F\uFB50-\uFDFF\uFE70-\uFEFF\uAC00-\uD7A3\u1100-\u11FF\u3130-\u318F\uFF01\uFF0c\uFF1A\uFF1B\uFF1F]/g, '')
            cleanedText = cleanedText.replace(/\n+/g, ' ');
            cleanedText = cleanedText.replace(/\.\s'/g, ".'"); // Remove space between period and single quote
            //cleanedText = cleanedText.replace(/'/g, ""); // Remove all single quotes
            // Set the cleaned text back to the form data
            formData.set('text', cleanedText);
            // Check if streaming or not and handle accordingly
            if (formData.get('streaming') === 'true') {
                // For streaming, update the audio player src
                const searchParams = new URLSearchParams(formData).toString();
                audioPlayer.src = `${cleanedBaseUrl}/api/tts-generate-streaming?${searchParams}`;
                audioPlayer.load(); // Reload the audio player
            } else {
                // Add hard-coded settings to the formData object
                formData.append('text_filtering', 'standard');
                formData.append('narrator_enabled', false);
                formData.append('autoplay', false);
                formData.append('autoplay_volume', '0.1');
                // Change the field name from "voice" to "character_voice_gen"
                const voice = formData.get('voice');
                formData.delete('voice');
                formData.append('character_voice_gen', voice);
                // Update the field name from "output_file" to "output_file_name"
                const outputFile = formData.get('output_file');
                formData.delete('output_file');
                formData.append('output_file_name', outputFile);
                formData.delete('text');
                formData.set('text_input', cleanedText);
                // Convert "output_file_timestamp" value to boolean
                formData.append('output_file_timestamp', false);
                console.log('Form Data:');
                for (const [key, value] of formData.entries()) {
                    console.log(`${key}: ${value}`);
                }
                // Send the POST request to the API endpoint
                try {
                    const response = await fetch(`${cleanedBaseUrl}/api/tts-generate`, {
                        method: 'POST',
                        body: formData,
                        cache: "no-store"
                    });
                    if (!response.ok) {
                        throw new Error(`HTTP error! status: ${response.status}`);
                    }
                    // Process the response
                    const result = await response.json();
                    console.log('Response:', result);
                    // Update output file path and audio player source using the response data
                    const outputFilePath = document.getElementById('outputFilePath');
                    outputFilePath.textContent = result.output_file_path;
                    audioPlayer.src = cleanedBaseUrl + result.output_file_url;
                    audioPlayer.load();
                } catch (error) {
                    console.error('Error:', error);
                    // Handle the error (e.g., display an error message to the user)
                }
            }
        });
    </script>
    <hr>
 </body>
 </html>
@@ -89,25 +89,25 @@
    <h3>AllTalk Startup Settings</h3><br>
    <form action="/update-settings" method="post">
-        <label for="activate" style="color: #33cc33;">Activate TTS:</label>
+        <label for="activate" style="color: #b273fa;">TG-webui Activate TTS:</label>
        <select id="activate" name="activate" required="">
            <option {% if data.activate %}selected{% endif %} value="true">True</option>
            <option {% if not data.activate %}selected{% endif %} value="false">False</option>
        </select>
-        <label for="autoplay" style="color: #33cc33;">&nbsp; &nbsp; Autoplay TTS output:</label>
+        <label for="autoplay" style="color: #b273fa;">&nbsp; &nbsp; TG-webui Autoplay TTS:</label>
        <select id="autoplay" name="autoplay" required="">
            <option {% if data.autoplay %}selected{% endif %} value="true">True</option>
            <option {% if not data.autoplay %}selected{% endif %} value="false">False</option>
        </select>
-        <label for="show_text" style="color: #33cc33;"> Show Text on generation:</label>
+        <label for="show_text" style="color: #b273fa;">TG-webui Show Text:</label>
        <select id="show_text" name="show_text" required="">
            <option {% if data.show_text %}selected{% endif %} value="true">True</option>
            <option {% if not data.show_text %}selected{% endif %} value="false">False</option>
        </select>
-        <label for="voice" style="color: #33cc33;">&nbsp; &nbsp; Default voice:</label>
+        <label for="voice" style="color: #b273fa;">&nbsp; &nbsp;  TG-webui Default voice:</label>
        <select id="voice" name="voice" required="">
            {% for file in wav_files %}
            <option value="{{ file }}" {% if data.voice==file %}selected{% endif %}>{{ file }}</option>
@@ -115,7 +115,7 @@
        </select>
-        <label for="language" style="color: #33cc33;">Default Language:</label>
+        <label for="language" style="color: #b273fa;">TG-webui Default Language:</label>
        <select id="language" name="language" required="">
            <option value="Arabic" {% if data.language=='Arabic' %}selected{% endif %}>Arabic</option>
            <option value="Chinese" {% if data.language=='Chinese' %}selected{% endif %}>Chinese</option>
@@ -135,20 +135,20 @@
            <option value="Turkish" {% if data.language=='Turkish' %}selected{% endif %}>Turkish</option>
        </select>
-        <label for="narrator_voice" style="color: #33cc33;">&nbsp; &nbsp; Narrator voice:</label>
+        <label for="narrator_voice" style="color: #b273fa;">&nbsp; &nbsp; TG-webui Narrator voice:</label>
        <select id="narrator_voice" name="narrator_voice" required="">
            {% for file in wav_files %}
            <option value="{{ file }}" {% if data.narrator_voice==file %}selected{% endif %}>{{ file }}</option>
            {% endfor %}
        </select>
-        <label for="remove_trailing_dots" style="color: #33cc33;"> Remove Trailing Dots:</label>
+        <label for="remove_trailing_dots" style="color: #b273fa;">TG-webui Remove Trailing Dots:</label>
        <select id="remove_trailing_dots" name="remove_trailing_dots" required="">
            <option {% if data.remove_trailing_dots %}selected{% endif %} value="true">True</option>
            <option {% if not data.remove_trailing_dots %}selected{% endif %} value="false">False</option>
        </select>
-        <label for="narrator_enabled" style="color: #33cc33;">&nbsp; &nbsp; Narrator enabled:</label>
+        <label for="narrator_enabled" style="color: #b273fa;">&nbsp; &nbsp; TG-webui Narrator enabled:</label>
        <select id="narrator_enabled" name="narrator_enabled" required="">
            <option {% if data.narrator_enabled %}selected{% endif %} value="true">True</option>
            <option {% if not data.narrator_enabled %}selected{% endif %} value="false">False</option>
@@ -201,12 +201,12 @@
            value="{{ data.output_folder_wav }}" />
-        <input type="hidden" id="model_loaded" name="model_loaded" value="{{ data.model_loaded }}" />
+        <input type="hidden" id="tts_model_loaded" name="tts_model_loaded" value="{{ data.tts_model_loaded }}" />
        <!-- Continue updating other dropdowns and inputs in a similar manner -->
-        <label for="model_name" style="color: red;"> API TTS Model Name:</label>
+        <label for="tts_model_name" style="color: red;"> API TTS Model Name:</label>
-        <input id="model_name" name="model_name" required="" type="text" value="{{ data.model_name }}" />
+        <input id="tts_model_name" name="tts_model_name" required="" type="text" value="{{ data.tts_model_name }}" />
        <label for="model_version"
            style="color: red; font-size: 16px; line-height: 1; vertical-align: bottom; margin-bottom: 0;">&nbsp;
@@ -215,20 +215,8 @@
            modeldownload_model_path }}</span>
-
+        <label for="port_number" style="color: red;"> Port Number:</label>
-
+        <input id="port_number" name="port_number" required="" type="text" value="{{ data.port_number }}" /><br>
        <label for="ip_address" style="color: red;"> IP Address:</label>
        <input id="ip_address" name="ip_address" required="" type="text" value="{{ data.ip_address }}" />
        <label for="port_number" style="color: red;"> &nbsp; &nbsp; Port Number:</label>
        <input id="port_number" name="port_number" required="" type="text" value="{{ data.port_number }}" />
        <div class="radio-group">
            <label for="tts_method_api_local" style="color: #33cc33;">Model: <span
@@ -0,0 +1,296 @@
 import sys
 import time
 import signal
 import json
 import warnings
 import pyperclip
 import requests
 from PyQt5.QtWidgets import (QApplication, QSystemTrayIcon, QMenu, QWidget, QVBoxLayout, QLabel, 
                             QLineEdit, QPushButton, QMessageBox, QComboBox, QSlider)
 from PyQt5.QtCore import Qt, QThread, pyqtSignal, QTimer
 from PyQt5.QtGui import QIcon
 # Suppress the specific warning
 warnings.filterwarnings("ignore", category=UserWarning, module="charset_normalizer")
 SETTINGS_FILE = "tts_settings.json"
 TTS_SERVER_URL = "http://127.0.0.1:7851"
 LANGUAGES = {
    "ar": "Arabic", "zh-cn": "Chinese (Simplified)", "cs": "Czech", "nl": "Dutch",
    "en": "English", "fr": "French", "de": "German", "hi": "Hindi (limited support)",
    "hu": "Hungarian", "it": "Italian", "ja": "Japanese", "ko": "Korean",
    "pl": "Polish", "pt": "Portuguese", "ru": "Russian", "es": "Spanish", "tr": "Turkish"
 }
 class ClipboardMonitor(QThread):
    text_changed = pyqtSignal(str)
    def __init__(self):
        super().__init__()
        self.last_text = pyperclip.paste()
        self.running = True
    def run(self):
        while self.running:
            current_text = pyperclip.paste()
            if current_text != self.last_text:
                self.text_changed.emit(current_text)
                self.last_text = current_text
            time.sleep(0.5)
    def stop(self):
        self.running = False
 class TTSWorker(QThread):
    error_occurred = pyqtSignal(str)
    def __init__(self, api_url, params):
        super().__init__()
        self.api_url = api_url
        self.params = params
        self.text_queue = []
    def add_text(self, text):
        self.text_queue.append(text)
    def run(self):
        while True:
            if self.text_queue:
                text = self.text_queue.pop(0)
                self.send_to_tts(text)
            else:
                time.sleep(0.1)
    def send_to_tts(self, text):
        params = self.params.copy()
        params["text_input"] = text
        params["language"] = params["language"][:2]  # Ensure we only send the 2-letter code
        try:
            response = requests.post(f"{self.api_url}/api/tts-generate", data=params)
            if response.status_code != 200:
                error_message = f"TTS generation failed with status code: {response.status_code}. Response: {response.text}"
                print(error_message)
                self.error_occurred.emit(error_message)
        except requests.RequestException as e:
            error_message = f"Error sending request to TTS API: {e}"
            print(error_message)
            self.error_occurred.emit(error_message)
 class SettingsWindow(QWidget):
    settings_updated = pyqtSignal(dict)
    def __init__(self, tts_worker):
        super().__init__()
        self.tts_worker = tts_worker
        self.voices = []
        self.init_ui()
    def init_ui(self):
        layout = QVBoxLayout()
        self.setLayout(layout)
        layout.addWidget(QLabel("AllTalk API URL:"))
        self.api_url_input = QLineEdit(self.tts_worker.api_url)
        layout.addWidget(self.api_url_input)
        layout.addWidget(QLabel("Voice:"))
        self.voice_combo = QComboBox()
        layout.addWidget(self.voice_combo)
        layout.addWidget(QLabel("Language:"))
        self.language_combo = QComboBox()
        for code, name in LANGUAGES.items():
            self.language_combo.addItem(f"{name} ({code})", code)
        current_lang = self.tts_worker.params.get("language", "en")
        index = self.language_combo.findData(current_lang)
        if index >= 0:
            self.language_combo.setCurrentIndex(index)
        layout.addWidget(self.language_combo)
        layout.addWidget(QLabel("Text Filtering:"))
        self.text_filtering_input = QLineEdit(self.tts_worker.params["text_filtering"])
        layout.addWidget(self.text_filtering_input)
        layout.addWidget(QLabel("Playback Volume:"))
        self.volume_slider = QSlider(Qt.Horizontal)
        self.volume_slider.setRange(1, 10)
        self.volume_slider.setValue(int(float(self.tts_worker.params["autoplay_volume"]) * 10))
        layout.addWidget(self.volume_slider)
        save_button = QPushButton("Save")
        save_button.clicked.connect(self.save_settings)
        layout.addWidget(save_button)
        refresh_button = QPushButton("Refresh Voices")
        refresh_button.clicked.connect(self.refresh_voices)
        layout.addWidget(refresh_button)
        self.setWindowTitle("AllTalk Clipboard TTS")
        self.setGeometry(300, 300, 300, 400)
        self.refresh_voices()
    def refresh_voices(self):
        try:
            response = requests.get(f"{self.tts_worker.api_url}/api/voices")
            if response.status_code == 200:
                data = response.json()
                if data["status"] == "success" and "voices" in data:
                    self.voices = data["voices"]
                    self.voice_combo.clear()
                    for voice in self.voices:
                        self.voice_combo.addItem(voice)
                    if self.voices:
                        current_voice = self.tts_worker.params["character_voice_gen"]
                        if current_voice in self.voices:
                            self.voice_combo.setCurrentText(current_voice)
                        else:
                            self.voice_combo.setCurrentIndex(0)
                else:
                    QMessageBox.warning(self, "Error", "Failed to fetch voices: Unexpected response format")
            else:
                QMessageBox.warning(self, "Error", f"Failed to fetch voices. Status code: {response.status_code}")
        except requests.RequestException as e:
            QMessageBox.warning(self, "Error", f"Failed to fetch voices: {str(e)}")
    def save_settings(self):
            new_settings = {
                "api_url": self.api_url_input.text(),
                "params": {
                    "character_voice_gen": self.voice_combo.currentText(),
                    "language": self.language_combo.currentData(),
                    "text_filtering": self.text_filtering_input.text(),
                    "output_file_name": "clipboard_tts",
                    "output_file_timestamp": "true",
                    "autoplay": "true",
                    "autoplay_volume": str(self.volume_slider.value() / 10.0)
                }
            }
            self.settings_updated.emit(new_settings)
            self.hide()
 class TTSApp(QSystemTrayIcon):
    def __init__(self, app):
        super().__init__(parent=app)
        self.app = app
        self.load_settings()
        self.init_ui()
    def load_settings(self):
        try:
            with open(SETTINGS_FILE, 'r') as f:
                self.settings = json.load(f)
        except FileNotFoundError:
            self.settings = {
                "api_url": TTS_SERVER_URL,
                "params": {
                    "text_filtering": "standard",
                    "character_voice_gen": "",
                    "language": "en",
                    "output_file_name": "clipboard_tts",
                    "output_file_timestamp": "true",
                    "autoplay": "true",
                    "autoplay_volume": "0.8"
                }
            }
    def save_settings(self):
        with open(SETTINGS_FILE, 'w') as f:
            json.dump(self.settings, f)
    def init_ui(self):
        self.tts_worker = TTSWorker(self.settings["api_url"], self.settings["params"])
        self.tts_worker.error_occurred.connect(self.show_error_message)
        self.tts_worker.start()
        self.clipboard_monitor = ClipboardMonitor()
        self.clipboard_monitor.text_changed.connect(self.on_text_changed)
        self.setIcon(QIcon("icon.png"))
        self.setVisible(True)
        menu = QMenu()
        start_action = menu.addAction("Start Monitoring")
        start_action.triggered.connect(self.start_monitoring)
        stop_action = menu.addAction("Stop Monitoring")
        stop_action.triggered.connect(self.stop_monitoring)
        settings_action = menu.addAction("Settings")
        settings_action.triggered.connect(self.show_settings)
        exit_action = menu.addAction("Exit")
        exit_action.triggered.connect(self.exit_app)
        self.setContextMenu(menu)
        self.settings_window = SettingsWindow(self.tts_worker)
        self.settings_window.settings_updated.connect(self.update_settings)
    def on_text_changed(self, text):
        self.tts_worker.add_text(text)
    def start_monitoring(self):
        self.clipboard_monitor.start()
        self.showMessage("TTS App", "Clipboard monitoring started")
    def stop_monitoring(self):
        self.clipboard_monitor.stop()
        self.showMessage("TTS App", "Clipboard monitoring stopped")
    def show_settings(self):
        self.settings_window.show()
    def update_settings(self, new_settings):
        self.settings = new_settings
        self.tts_worker.api_url = new_settings["api_url"]
        self.tts_worker.params = new_settings["params"]
        self.save_settings()
        self.showMessage("TTS App", "Settings updated successfully")
    def show_error_message(self, message):
        QMessageBox.critical(None, "Error", message)
    def exit_app(self):
        self.clipboard_monitor.stop()
        self.tts_worker.quit()
        QApplication.instance().quit()
 def check_server_ready(url, timeout=10):
    start_time = time.time()
    while time.time() - start_time < timeout:
        try:
            response = requests.get(f"{url}/api/ready")
            if response.status_code == 200 and response.text == "Ready":
                return True
        except requests.RequestException:
            pass
        time.sleep(1)
    return False
 def signal_handler(signal, frame):
    print("[Clipboard to TTS] Exiting...")
    QApplication.instance().quit()
 if __name__ == "__main__":
    print("[Clipboard to TTS] AllTalk Clipboard to TTS is starting...")
    if not check_server_ready(TTS_SERVER_URL):
        print("[Clipboard to TTS] Error: AllTalk TTS server is not ready. Please start the AllTalk server and try again.")
        sys.exit(1)
    signal.signal(signal.SIGINT, signal_handler)
    app = QApplication(sys.argv)
    if not QSystemTrayIcon.isSystemTrayAvailable():
        print("[Clipboard to TTS] System tray is not available on this system.")
        sys.exit(1)
    tts_app = TTSApp(app)
    # This allows for Ctrl+C to work
    timer = QTimer()
    timer.start(500)
    timer.timeout.connect(lambda: None)
    print("[Clipboard to TTS] AllTalk Clipboard to TTS is running... Check your system tray for the icon.")
    print("[Clipboard to TTS] Press Ctrl+C to exit")
    sys.exit(app.exec_())
@@ -0,0 +1,13 @@
 This is a Windows only clipboard to TTS extension. Its a very rough bit of code with no logic or error handling built in.
 When run it will create a system tray icon that you can Start/Stop clipboard monitoring for the text stored in the clipboard changing. When the text changes, it will be sent to AllTalk for TTS generation. You can specify the voice to use in the System Tray icon Settings area.
 In the main folder, run "start_environment.bat" to start the AllTalk Python environment.
 You will need to "pip install pyqt5 pyperclip requests" for to install the requirements into the AllTalk Python environment (you will only need to do this the once, but always start the Python environment).
 Go to the "cliptotts" folder and "python clipboard-to-tts.py" which should start the application.
 Start monitoring of the clipboard on the icon in the system tray.
 When you Ctrl+C text to the clipboard, it will be sent over for TTS generation and it will play back at the AllTalk console.
@@ -0,0 +1,564 @@
 ======================================
 AllTalk Base Diagnostic log file for Standalone Installations
 ======================================
 Diagnostic log created at 2024-11-06 01:25:33.123679
 ======================================
 OPERATING SYSTEM:
 OS Version: Windows 10.0.22631
 Note: Windows 11 will list as build is 10.x.22xxx
 HARDWARE ENVIRONMENT:
 CPU: Physical Cores: 8, Total Cores: 16, Max Frequency: 3800.0 MHz
 System RAM: 19.46 GB available out of 31.34 GB total
 GPU INFORMATION:
 Fri Sep 27 20:50:15 2024       
 +-----------------------------------------------------------------------------------------+
 | NVIDIA-SMI 560.94                 Driver Version: 560.94         CUDA Version: 12.6     |
 |-----------------------------------------+------------------------+----------------------+
 | GPU  Name                  Driver-Model | Bus-Id          Disp.A | Volatile Uncorr. ECC |
 | Fan  Temp   Perf          Pwr:Usage/Cap |           Memory-Usage | GPU-Util  Compute M. |
 |                                         |                        |               MIG M. |
 |=========================================+========================+======================|
 |   0  NVIDIA GeForce RTX 4070      WDDM  |   00000000:01:00.0  On |                  N/A |
 |  0%   39C    P8             13W /  200W |    1071MiB /  12282MiB |      1%      Default |
 |                                         |                        |                  N/A |
 +-----------------------------------------+------------------------+----------------------+
 +-----------------------------------------------------------------------------------------+
 | Processes:                                                                              |
 |  GPU   GI   CI        PID   Type   Process name                              GPU Memory |
 |        ID   ID                                                               Usage      |
 |=========================================================================================|
 |    0   N/A  N/A      1508    C+G   ...nt.CBS_cw5n1h2txyewy\SearchHost.exe      N/A      |
 |    0   N/A  N/A      2756    C+G   ...\cef\cef.win7x64\steamwebhelper.exe      N/A      |
 |    0   N/A  N/A      5488    C+G   ...2txyewy\StartMenuExperienceHost.exe      N/A      |
 |    0   N/A  N/A      6504    C+G   ...ekyb3d8bbwe\PhoneExperienceHost.exe      N/A      |
 |    0   N/A  N/A      7212    C+G   ...t.LockApp_cw5n1h2txyewy\LockApp.exe      N/A      |
 |    0   N/A  N/A      8604    C+G   C:\Windows\explorer.exe                     N/A      |
 |    0   N/A  N/A      8852    C+G   ...ft Office\root\Office16\OUTLOOK.EXE      N/A      |
 |    0   N/A  N/A     11964    C+G   ...5.0_x64__cv1g1gvanyjgm\WhatsApp.exe      N/A      |
 |    0   N/A  N/A     12516    C+G   ...CBS_cw5n1h2txyewy\TextInputHost.exe      N/A      |
 |    0   N/A  N/A     13376    C+G   ...\cef\cef.win7x64\steamwebhelper.exe      N/A      |
 |    0   N/A  N/A     17632    C+G   ...nr4m\radeonsoftware\AMDRSSrcExt.exe      N/A      |
 |    0   N/A  N/A     18316    C+G   ...m\radeonsoftware\RadeonSoftware.exe      N/A      |
 |    0   N/A  N/A     18908    C+G   ...siveControlPanel\SystemSettings.exe      N/A      |
 |    0   N/A  N/A     19032    C+G   ...am Files\Microsoft VS Code\Code.exe      N/A      |
 |    0   N/A  N/A     21216    C+G   ...bytes\Anti-Malware\Malwarebytes.exe      N/A      |
 |    0   N/A  N/A     21476    C+G   ...crosoft\Edge\Application\msedge.exe      N/A      |
 |    0   N/A  N/A     21604    C+G   ...ta\Local\WeMod\app-9.10.3\WeMod.exe      N/A      |
 |    0   N/A  N/A     22344    C+G   ...__8wekyb3d8bbwe\WindowsTerminal.exe      N/A      |
 |    0   N/A  N/A     25292    C+G   ...5n1h2txyewy\ShellExperienceHost.exe      N/A      |
 |    0   N/A  N/A     26220      C   ...\alltalk_environment\env\python.exe      N/A      |
 |    0   N/A  N/A     27960    C+G   ...Brave-Browser\Application\brave.exe      N/A      |
 +-----------------------------------------------------------------------------------------+
 CUDA:
 CUDA Working: Success - CUDA is available and working.
 CUDA_HOME   : N/A
 CUDA Device : NVIDIA GeForce RTX 4070
 CUDA Memory : 11.99 GB
 CUDA Version: 12.1
 DISK INFORMATION:
 Drive: C:\ | Total: 1000.95 GB | Used: 852.08 GB | Free: 148.87 GB | Type: NTFS
 NETWORK PORT:
 Port Status : Port 7851 is available.
 Cublas64_11 Path: C:\myfiles\alltalk_tts\alltalk_environment\env\Lib\site-packages\nvidia/cublas\bin\cublas64_11.dll
 Windows C++ Build tools & Windows SDK:
 Windows Version: Windows 11
 Visual C++ Build Tools and/or Visual Studio found:
 Visual Studio 2019 Build Tools (Path: C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools)
 Windows SDK(s) found:
 SDK Version: 10.0.22000.0
 Path: C:\Program Files (x86)\Windows Kits\10
 Python setuptools version: 75.1.0
 Windows Espeak-ng:
 Espeak-ng version: eSpeak NG text-to-speech: 1.52-dev  Data at: C:\Program Files\eSpeak NG\/espeak-ng-data
 PYTHON & PYTORCH:
 Torch Version: 2.2.1
 Python Version: 3.11.10
 Python Version Info: sys.version_info(major=3, minor=11, micro=10, releaselevel='final', serial=0)
 Python Executable: C:\myfiles\alltalk_tts\alltalk_environment\env\python.exe
 Python Virtual Environment: N/A (Should be N/A when in Text-generation-webui Conda Python environment)
 Conda Environment: C:\myfiles\alltalk_tts\alltalk_environment\env
 Python Search Path:
  C:\myfiles\alltalk_tts
  C:\myfiles\alltalk_tts\alltalk_environment\env\python311.zip
  C:\myfiles\alltalk_tts\alltalk_environment\env\DLLs
  C:\myfiles\alltalk_tts\alltalk_environment\env\Lib
  C:\myfiles\alltalk_tts\alltalk_environment\env
  C:\myfiles\alltalk_tts\alltalk_environment\env\Lib\site-packages
  C:\myfiles\alltalk_tts\alltalk_environment\env\Lib\site-packages\win32
  C:\myfiles\alltalk_tts\alltalk_environment\env\Lib\site-packages\win32\lib
  C:\myfiles\alltalk_tts\alltalk_environment\env\Lib\site-packages\Pythonwin
 OS SEARCHPATH ENVIRONMENT:
  C:\myfiles\alltalk_tts\alltalk_environment\env\Lib\site-packages\PyQt6\Qt6\bin
  C:\myfiles\alltalk_tts\alltalk_environment\env
  C:\myfiles\alltalk_tts\alltalk_environment\env\Library\mingw-w64\bin
  C:\myfiles\alltalk_tts\alltalk_environment\env\Library\usr\bin
  C:\myfiles\alltalk_tts\alltalk_environment\env\Library\bin
  C:\myfiles\alltalk_tts\alltalk_environment\env\Scripts
  C:\myfiles\alltalk_tts\alltalk_environment\env\bin
  C:\Program Files (x86)\Common Files\Oracle\Java\javapath
  C:\Windows\system32
  C:\Windows
  C:\Windows\System32\Wbem
  C:\Windows\System32\WindowsPowerShell\v1.0
  C:\Program Files\dotnet
  C:\Program Files\NVIDIA Corporation\NVIDIA NvDLISR
  C:\Program Files\Git\cmd
  C:\ProgramData\chocolatey\bin
  C:
  C:\Program Files\nodejs
  C:\Program Files (x86)\Windows Kits\10\App Certification Kit
  C:\Program Files\Microsoft VS Code\bin
  C:\Program Files (x86)\NVIDIA Corporation\PhysX\Common
  C:\Program Files\NVIDIA Corporation\Nsight Compute 2023.1.0
  C:\Program Files\eSpeak NG
  C:\Program Files (x86)\eSpeak NG
 CONDA INFORMATION:
 Conda Executable: C:\myfiles\alltalk_tts\alltalk_environment\conda\condabin\..\Scripts\conda.exe
 Conda Version: 24.4.0
 Current Environment: C:\myfiles\alltalk_tts\alltalk_environment\env
 Conda Environments:
  - C:\myfiles\alltalk_tts\alltalk_environment\conda
  - C:\myfiles\alltalk_tts\alltalk_environment\env
 CONDA PACKAGES IN CURRENT ENVIRONMENT:
 CONDA-FORGE PACKAGES:
  aom                            Version: 3.9.1           Channel: conda-forge
  cairo                          Version: 1.18.0          Channel: conda-forge
  ffmpeg                         Version: 4.3.1           Channel: conda-forge
  fontconfig                     Version: 2.15.0          Channel: conda-forge
  freetype                       Version: 2.12.1          Channel: conda-forge
  harfbuzz                       Version: 9.0.0           Channel: conda-forge
  icu                            Version: 75.1            Channel: conda-forge
  lcms2                          Version: 2.15            Channel: conda-forge
  libexpat                       Version: 2.6.4           Channel: conda-forge
  libglib                        Version: 2.82.2          Channel: conda-forge
  libiconv                       Version: 1.17            Channel: conda-forge
  libintl                        Version: 0.22.5          Channel: conda-forge
  libopus                        Version: 1.3.1           Channel: conda-forge
  libpng                         Version: 1.6.44          Channel: conda-forge
  libsqlite                      Version: 3.47.0          Channel: conda-forge
  libxcb                         Version: 1.13            Channel: conda-forge
  libxml2                        Version: 2.13.5          Channel: conda-forge
  libzlib                        Version: 1.3.1           Channel: conda-forge
  openh264                       Version: 2.4.1           Channel: conda-forge
  openssl                        Version: 3.4.0           Channel: conda-forge
  pcre2                          Version: 10.44           Channel: conda-forge
  pixman                         Version: 0.43.4          Channel: conda-forge
  python                         Version: 3.11.10         Channel: conda-forge
  python_abi                     Version: 3.11            Channel: conda-forge
  svt-av1                        Version: 2.2.1           Channel: conda-forge
  tk                             Version: 8.6.13          Channel: conda-forge
  vc14_runtime                   Version: 14.42.34433     Channel: conda-forge
  vs2015_runtime                 Version: 14.42.34433     Channel: conda-forge
  x264                           Version: 1!164.3095      Channel: conda-forge
  x265                           Version: 3.5             Channel: conda-forge
  xorg-libxau                    Version: 1.0.11          Channel: conda-forge
  xorg-libxdmcp                  Version: 1.1.3           Channel: conda-forge
  zlib                           Version: 1.3.1           Channel: conda-forge
  zstd                           Version: 1.5.6           Channel: conda-forge
 PKGS/MAIN PACKAGES:
  blas                           Version: 1.0             Channel: pkgs/main
  brotli-python                  Version: 1.0.9           Channel: pkgs/main
  bzip2                          Version: 1.0.8           Channel: pkgs/main
  ca-certificates                Version: 2024.9.24       Channel: pkgs/main
  certifi                        Version: 2024.8.30       Channel: pkgs/main
  charset-normalizer             Version: 3.3.2           Channel: pkgs/main
  dav1d                          Version: 1.2.1           Channel: pkgs/main
  font-ttf-dejavu-sans-mono      Version: 2.37            Channel: pkgs/main
  font-ttf-inconsolata           Version: 2.001           Channel: pkgs/main
  font-ttf-source-code-pro       Version: 2.030           Channel: pkgs/main
  font-ttf-ubuntu                Version: 0.83            Channel: pkgs/main
  fonts-anaconda                 Version: 1               Channel: pkgs/main
  fonts-conda-ecosystem          Version: 1               Channel: pkgs/main
  gmpy2                          Version: 2.1.2           Channel: pkgs/main
  graphite2                      Version: 1.3.14          Channel: pkgs/main
  idna                           Version: 3.7             Channel: pkgs/main
  intel-openmp                   Version: 2023.1.0        Channel: pkgs/main
  jinja2                         Version: 3.1.4           Channel: pkgs/main
  jpeg                           Version: 9e              Channel: pkgs/main
  lerc                           Version: 3.0             Channel: pkgs/main
  libdeflate                     Version: 1.17            Channel: pkgs/main
  libffi                         Version: 3.4.4           Channel: pkgs/main
  libjpeg-turbo                  Version: 2.0.0           Channel: pkgs/main
  libtiff                        Version: 4.5.0           Channel: pkgs/main
  libuv                          Version: 1.48.0          Channel: pkgs/main
  libwebp-base                   Version: 1.3.2           Channel: pkgs/main
  lz4-c                          Version: 1.9.4           Channel: pkgs/main
  markupsafe                     Version: 2.1.3           Channel: pkgs/main
  mkl                            Version: 2023.1.0        Channel: pkgs/main
  mkl-service                    Version: 2.4.0           Channel: pkgs/main
  mkl_fft                        Version: 1.3.11          Channel: pkgs/main
  mkl_random                     Version: 1.2.8           Channel: pkgs/main
  mpc                            Version: 1.1.0           Channel: pkgs/main
  mpfr                           Version: 4.0.2           Channel: pkgs/main
  mpir                           Version: 3.0.0           Channel: pkgs/main
  mpmath                         Version: 1.3.0           Channel: pkgs/main
  networkx                       Version: 3.2.1           Channel: pkgs/main
  numpy                          Version: 1.26.4          Channel: pkgs/main
  numpy-base                     Version: 1.26.4          Channel: pkgs/main
  openjpeg                       Version: 2.5.2           Channel: pkgs/main
  packaging                      Version: 24.1            Channel: pkgs/main
  pip                            Version: 24.2            Channel: pkgs/main
  pthread-stubs                  Version: 0.3             Channel: pkgs/main
  pysocks                        Version: 1.7.1           Channel: pkgs/main
  pyyaml                         Version: 6.0.2           Channel: pkgs/main
  requests                       Version: 2.32.3          Channel: pkgs/main
  setuptools                     Version: 75.1.0          Channel: pkgs/main
  sqlite                         Version: 3.45.3          Channel: pkgs/main
  sympy                          Version: 1.13.2          Channel: pkgs/main
  tbb                            Version: 2021.8.0        Channel: pkgs/main
  ucrt                           Version: 10.0.20348.0    Channel: pkgs/main
  urllib3                        Version: 2.2.3           Channel: pkgs/main
  vc                             Version: 14.40           Channel: pkgs/main
  wheel                          Version: 0.44.0          Channel: pkgs/main
  win_inet_pton                  Version: 1.1.0           Channel: pkgs/main
  xz                             Version: 5.4.6           Channel: pkgs/main
  yaml                           Version: 0.2.5           Channel: pkgs/main
 PKGS/MSYS2 PACKAGES:
  m2w64-gcc-libgfortran          Version: 5.3.0           Channel: pkgs/msys2
  m2w64-gcc-libs                 Version: 5.3.0           Channel: pkgs/msys2
  m2w64-gcc-libs-core            Version: 5.3.0           Channel: pkgs/msys2
  m2w64-gmp                      Version: 6.1.0           Channel: pkgs/msys2
  m2w64-libwinpthread-git        Version: 5.0.0.4634.697f757 Channel: pkgs/msys2
  msys2-conda-epoch              Version: 20160418        Channel: pkgs/msys2
 OTHER PACKAGES:
  faiss-cpu                      Version: 1.9.0           Channel: pytorch
  libfaiss                       Version: 1.9.0           Channel: pytorch
  pytorch                        Version: 2.2.1           Channel: pytorch
  pytorch-cuda                   Version: 12.1            Channel: pytorch
  pytorch-mutex                  Version: 1.0             Channel: pytorch
 PACKAGE VERSIONS vs REQUIREMENTS FILE:
 coqui-tts           Required: >= 0.24.3        Installed: 0.24.3
 filelock            Required: >= 3.16.1        Installed: 3.13.4
 fuzzywuzzy          Required: >= 0.18.0        Installed: 0.18.0
 ffmpeg-python       Required: >= 0.2.0         Installed: 0.2.0
 gradio              Required: == 4.44.1        Installed: 4.44.1
 importlib_metadata  Required: >= 7.2.1         Installed: 8.5.0
 inputimeout         Required: >= 1.0.4         Installed: 1.0.4
 Jinja2              Required: >= 3.1.4         Installed: 3.1.4
 librosa             Required: >= 0.10.2.post1  Installed: 0.10.2.post1
 onnxruntime-gpu     Required: >= 1.18.1        Installed: 1.20.1
 pydantic            Required: >= 2.8.2         Installed: 2.10.1
 python-Levenshtein  Required: >= 0.25.1        Installed: 0.26.1
 praat-parselmouth   Required: >= 0.4.4         Installed: 0.4.5
 pyworld             Required: >= 0.3.4         Installed: 0.3.4
 sounddevice         Required: >= 0.4.7         Installed: 0.5.1
 soundfile           Required: >= 0.12.1        Installed: 0.12.1
 spacy               Required: >= 3.7.1         Installed: 3.7.5
 torchcrepe          Required: >= 0.0.2         Installed: 0.0.23
 tqdm                Required: >= 4.66.5        Installed: 4.67.1
 unidic-lite         Required: >= 1.0.8         Installed: 1.0.8
 uvicorn             Required: >= 0.29.0        Installed: 0.32.1
 pillow              Required: == 10.3.0        Installed: 10.3.0
 pypinyin            Required: >= 0.52.0        Installed: 0.53.0
 word2number         Required: >= 1.1           Installed: 1.1
 cutlet              Required: == 0.4.0         Installed: 0.4.0
 fugashi             Required: == 1.3.1         Installed: 1.3.1
 fastapi             Required: == 0.112.2       Installed: 0.112.2
 plotly              Required: == 5.24.1        Installed: 5.24.1
 scipy               Required: == 1.14.1        Installed: 1.14.1
 PYTHON PACKAGES:
 absl-py                       = 2.1.0
 accelerate                    = 1.1.1
 aiofiles                      = 23.2.1
 aiohappyeyeballs              = 2.4.3
 aiohttp                       = 3.11.7
 aiosignal                     = 1.3.1
 annotated-types               = 0.7.0
 antlr4-python3-runtime        = 4.9.3
 anyascii                      = 0.3.2
 anyio                         = 4.6.2.post1
 argbind                       = 0.3.9
 asttokens                     = 2.4.1
 attrs                         = 24.2.0
 audioread                     = 3.0.1
 babel                         = 2.16.0
 bitarray                      = 3.0.0
 bitsandbytes                  = 0.44.1
 blis                          = 0.7.11
 boto3                         = 1.35.69
 botocore                      = 1.35.69
 Brotli                        = 1.0.9
 cached_path                   = 1.6.4
 cachetools                    = 5.5.0
 catalogue                     = 2.0.10
 certifi                       = 2024.8.30
 cffi                          = 1.17.1
 charset-normalizer            = 3.3.2
 click                         = 8.1.7
 cloudpathlib                  = 0.20.0
 colorama                      = 0.4.6
 coloredlogs                   = 15.0.1
 confection                    = 0.1.5
 contourpy                     = 1.3.1
 coqpit                        = 0.0.17
 coqui-tts                     = 0.24.3
 coqui-tts-trainer             = 0.1.7
 cutlet                        = 0.4.0
 cycler                        = 0.12.1
 cymem                         = 2.0.10
 Cython                        = 3.0.11
 datasets                      = 3.1.0
 dateparser                    = 1.1.8
 decorator                     = 5.1.1
 deepspeed                     = 0.14.0+ce78a63
 descript-audiotools           = 0.7.4
 descript-audio-codec          = 1.0.0
 dill                          = 0.3.8
 docker-pycreds                = 0.4.0
 docopt                        = 0.6.2
 docstring_parser              = 0.16
 einops                        = 0.8.0
 einx                          = 0.3.0
 ema-pytorch                   = 0.7.6
 encodec                       = 0.1.1
 executing                     = 2.1.0
 fairseq                       = 0.12.4
 faiss                         = 1.9.0
 fastapi                       = 0.112.2
 ffmpeg-python                 = 0.2.0
 ffmpy                         = 0.4.0
 filelock                      = 3.13.4
 fire                          = 0.7.0
 flatbuffers                   = 24.3.25
 flatten-dict                  = 0.4.2
 fonttools                     = 4.55.0
 frozendict                    = 2.4.6
 frozenlist                    = 1.5.0
 fsspec                        = 2024.9.0
 fugashi                       = 1.3.1
 future                        = 1.0.0
 fuzzywuzzy                    = 0.18.0
 gitdb                         = 4.0.11
 GitPython                     = 3.1.43
 gmpy2                         = 2.1.2
 googleapis-common-protos      = 1.66.0
 google-api-core               = 2.23.0
 google-auth                   = 2.36.0
 google-cloud-core             = 2.4.1
 google-cloud-storage          = 2.18.2
 google-crc32c                 = 1.6.0
 google-resumable-media        = 2.7.2
 gradio                        = 4.44.1
 gradio_client                 = 1.3.0
 grpcio                        = 1.68.0
 gruut                         = 2.4.0
 gruut-ipa                     = 0.13.0
 gruut_lang_de                 = 2.0.1
 gruut_lang_en                 = 2.0.1
 gruut_lang_es                 = 2.0.1
 gruut_lang_fr                 = 2.0.2
 h11                           = 0.14.0
 hangul-romanize               = 0.1.0
 hjson                         = 3.1.0
 httpcore                      = 1.0.7
 httpx                         = 0.27.2
 huggingface-hub               = 0.26.2
 humanfriendly                 = 10.0
 hydra-core                    = 1.3.2
 idna                          = 3.7
 importlib_metadata            = 8.5.0
 importlib_resources           = 6.4.5
 inflect                       = 7.4.0
 inputimeout                   = 1.0.4
 ipython                       = 8.29.0
 jaconv                        = 0.4.0
 jedi                          = 0.19.2
 jieba                         = 0.42.1
 Jinja2                        = 3.1.4
 jmespath                      = 1.0.1
 joblib                        = 1.4.2
 jsonlines                     = 1.2.0
 julius                        = 0.2.7
 kiwisolver                    = 1.4.7
 langcodes                     = 3.5.0
 language_data                 = 1.3.0
 lazy_loader                   = 0.4
 Levenshtein                   = 0.26.1
 librosa                       = 0.10.2.post1
 llvmlite                      = 0.43.0
 local-attention               = 1.9.15
 lxml                          = 5.3.0
 marisa-trie                   = 1.2.1
 Markdown                      = 3.7
 markdown2                     = 2.5.1
 markdown-it-py                = 3.0.0
 MarkupSafe                    = 2.1.3
 matplotlib                    = 3.9.2
 matplotlib-inline             = 0.1.7
 mdurl                         = 0.1.2
 mkl_fft                       = 1.3.11
 mkl_random                    = 1.2.8
 mkl-service                   = 2.4.0
 mojimoji                      = 0.0.13
 more-itertools                = 10.5.0
 mpmath                        = 1.3.0
 msgpack                       = 1.1.0
 multidict                     = 6.1.0
 multiprocess                  = 0.70.16
 murmurhash                    = 1.0.11
 networkx                      = 3.2.1
 ninja                         = 1.11.1.2
 num2words                     = 0.5.13
 numba                         = 0.60.0
 numpy                         = 1.26.4
 omegaconf                     = 2.3.0
 onnxruntime-gpu               = 1.20.1
 openai-whisper                = 20240930
 orjson                        = 3.10.12
 packaging                     = 24.1
 pandas                        = 2.2.3
 parler_tts                    = 0.2.1
 parso                         = 0.8.4
 pillow                        = 10.3.0
 pip                           = 24.2
 platformdirs                  = 4.3.6
 plotly                        = 5.24.1
 pooch                         = 1.8.2
 portalocker                   = 3.0.0
 praat-parselmouth             = 0.4.5
 preshed                       = 3.0.9
 prompt_toolkit                = 3.0.48
 propcache                     = 0.2.0
 protobuf                      = 4.25.5
 proto-plus                    = 1.25.0
 psutil                        = 6.1.0
 pure_eval                     = 0.2.3
 pyarrow                       = 18.1.0
 pyasn1                        = 0.6.1
 pyasn1_modules                = 0.4.1
 pycparser                     = 2.22
 pydantic                      = 2.10.1
 pydantic_core                 = 2.27.1
 pydub                         = 0.25.1
 Pygments                      = 2.18.0
 pyloudnorm                    = 0.1.1
 pynvml                        = 11.5.3
 pyparsing                     = 3.2.0
 pypinyin                      = 0.53.0
 PyQt6                         = 6.7.1
 PyQt6-Qt6                     = 6.7.3
 PyQt6_sip                     = 13.8.0
 pyreadline3                   = 3.5.4
 pysbd                         = 0.3.4
 PySocks                       = 1.7.1
 pystoi                        = 0.4.1
 python-crfsuite               = 0.9.11
 python-dateutil               = 2.9.0.post0
 python-Levenshtein            = 0.26.1
 python-multipart              = 0.0.17
 pytz                          = 2024.2
 pywin32                       = 308
 pyworld                       = 0.3.4
 PyYAML                        = 6.0.2
 py-cpuinfo                    = 9.0.0
 randomname                    = 0.2.1
 RapidFuzz                     = 3.10.1
 regex                         = 2024.11.6
 requests                      = 2.32.3
 resampy                       = 0.4.3
 rich                          = 13.9.4
 rotary-embedding-torch        = 0.8.5
 rsa                           = 4.9
 ruff                          = 0.8.0
 s3transfer                    = 0.10.4
 sacrebleu                     = 2.4.3
 safetensors                   = 0.4.5
 scikit-learn                  = 1.5.2
 scipy                         = 1.14.1
 semantic-version              = 2.10.0
 sentencepiece                 = 0.2.0
 sentry-sdk                    = 2.19.0
 setproctitle                  = 1.3.4
 setuptools                    = 75.1.0
 shellingham                   = 1.5.4
 six                           = 1.16.0
 smart-open                    = 7.0.5
 smmap                         = 5.0.1
 sniffio                       = 1.3.1
 sounddevice                   = 0.5.1
 soundfile                     = 0.12.1
 soxr                          = 0.5.0.post1
 spacy                         = 3.7.5
 spacy-legacy                  = 3.0.12
 spacy-loggers                 = 1.0.5
 srsly                         = 2.4.8
 stack-data                    = 0.6.3
 starlette                     = 0.38.6
 SudachiDict-core              = 20241021
 SudachiPy                     = 0.6.9
 sympy                         = 1.13.2
 tabulate                      = 0.9.0
 tenacity                      = 9.0.0
 tensorboard                   = 2.18.0
 tensorboard-data-server       = 0.7.2
 termcolor                     = 2.5.0
 thinc                         = 8.2.5
 threadpoolctl                 = 3.5.0
 tiktoken                      = 0.8.0
 tokenizers                    = 0.20.3
 tomli                         = 2.1.0
 tomlkit                       = 0.12.0
 torch                         = 2.2.1
 torchaudio                    = 2.2.1
 torchcrepe                    = 0.0.23
 torchdiffeq                   = 0.2.5
 torchvision                   = 0.17.1
 torch-stoi                    = 0.2.3
 tqdm                          = 4.67.1
 traitlets                     = 5.14.3
 transformers                  = 4.46.1
 transformers-stream-generator = 0.0.5
 typeguard                     = 4.4.1
 typer                         = 0.13.1
 typing_extensions             = 4.12.2
 tzdata                        = 2024.2
 tzlocal                       = 5.2
 unidic-lite                   = 1.0.8
 urllib3                       = 2.2.3
 uvicorn                       = 0.32.1
 vocos                         = 0.1.0
 wandb                         = 0.18.7
 wasabi                        = 1.1.3
 wcwidth                       = 0.2.13
 weasel                        = 0.4.1
 websockets                    = 12.0
 Werkzeug                      = 3.1.3
 wheel                         = 0.44.0
 win-inet-pton                 = 1.1.0
 word2number                   = 1.1
 wrapt                         = 1.17.0
 xxhash                        = 3.5.0
 x-transformers                = 1.42.17
 yarl                          = 1.18.0
 zipp                          = 3.21.0
 DeepSpeed Installation Requirements:
  OS     : Windows
  Python : 3.11.10
  PyTorch: 2.2.1
  CUDA   : 12.1
 To install DeepSpeed, run the following command based on your environment:
  pip install deepspeed==<version> --no-cache-dir
 Ensure to choose the correct DeepSpeed version matching your environment.
@@ -0,0 +1,382 @@
 """
 firstrun.py
 This script handles the first-time setup for the Alltalk TTS system. It allows the user to download and configure 
 the initial Text-to-Speech (TTS) engine and associated models. It provides automatic detection for Docker or Google 
 Colab environments to simplify the setup process, as well as an interactive menu or command-line arguments for manual configuration.
 Features:
 ---------
 1. **Automatic Environment Detection**:
    - Detects if the script is running in a Docker container or Google Colab environment.
    - Automatically sets up the Piper TTS engine in these cases.
 2. **Command-Line Arguments**:
    - Accepts a `--tts_model` argument to bypass the interactive menu and directly set up a specific TTS engine:
        - `piper`: Sets up the Piper TTS engine.
        - `vits`: Sets up the VITS TTS engine.
        - `xtts`: Sets up the XTTS TTS engine.
        - `none`: Skips model setup entirely.
 3. **Interactive Menu**:
    - If no command-line argument is provided and the environment is not Docker/Colab, it presents an interactive 
      menu for the user to select a TTS model.
 4. **Configuration Management**:
    - Updates the Alltalk TTS configuration files to reflect the selected TTS engine and marks the initial setup as complete.
 Usage:
 ------
 Command-Line:
    python firstrun.py --tts_model [piper|vits|xtts|none]
 Interactive Mode:
    python firstrun.py
 In Docker/Colab:
    The script automatically selects and sets up the Piper TTS engine without user interaction.
 Functions:
 ----------
 - **is_running_in_colab**: Detects if the script is running in Google Colab.
 - **is_running_in_docker**: Detects if the script is running in a Docker container.
 - **download_file**: Downloads a file with a progress bar and retry logic.
 - **setup_piper**: Sets up the Piper TTS engine and downloads associated files.
 - **setup_vits**: Sets up the VITS TTS engine and downloads associated files.
 - **setup_xtts**: Sets up the XTTS TTS engine and downloads associated files.
 - **update_tts_engines**: Updates the configuration to reflect the selected TTS engine.
 - **set_firstrun_model_false**: Marks the initial setup as complete in the configuration.
 - **warning_message**: Displays post-setup warnings or tips for the user.
 Notes:
 ------
 - Ensure network access is available for downloading models during setup.
 - Configuration files are automatically updated and backed up if needed.
 """
 import argparse
 import json
 from pathlib import Path
 import os
 import shutil
 import time
 import sys
 import zipfile
 import requests
 from tqdm import tqdm
 from inputimeout import inputimeout, TimeoutOccurred
 # Setting the module search path:
 sys.path.insert(0, str(Path(__file__).parent.parent.parent.resolve()))
 from config import AlltalkTTSEnginesConfig, AlltalkConfig #pylint: disable=wrong-import-position
 this_dir = Path(__file__).parent.resolve().parent.resolve().parent.resolve()
 def is_running_in_colab():
    """Test for google colab"""
    try:
        import google.colab
        return True
    except ImportError:
        return False
 def is_running_in_docker():
    """Test for google colab"""
    path = '/proc/self/cgroup'
    return os.path.exists('/.dockerenv') or (
        os.path.exists(path) and any('docker' in line for line in open(path, encoding='utf-8'))
    )
 def download_file(url: str, dest_path: str, timeout: int = 30, retries: int = 3) -> bool:
    """
    Download a file from a URL to a specified destination path with progress bar.
    Args:
        url (str): URL of the file to download.
        dest_path (str): Destination path where the file will be saved.
        timeout (int, optional): Connection and read timeout in seconds. Defaults to 30.
        retries (int, optional): Number of retry attempts for failed downloads. Defaults to 3.
    Returns:
        bool: True if the file was successfully downloaded, False otherwise.
    Raises:
        ValueError: If the downloaded file size does not match the expected size.
    """
    progress_bar = None
    for attempt in range(1, retries + 1):
        try:
            print(f"Attempting to download: {url} (Attempt {attempt}/{retries})")
            response = requests.get(url, stream=True, timeout=(timeout, timeout))
            response.raise_for_status()
            total_size = int(response.headers.get('content-length', 0))
            block_size = 1024  # 1 Kibibyte
            # Initialize progress bar
            progress_bar = tqdm(total=total_size, unit='iB', unit_scale=True,
                                desc=f"Downloading {Path(dest_path).name} (Attempt {attempt}/{retries})")
            Path(dest_path).parent.mkdir(parents=True, exist_ok=True)
            with open(dest_path, 'wb') as dwn_file:
                for data in response.iter_content(block_size):
                    if data:
                        dwn_file.write(data)
                        progress_bar.update(len(data))
            if progress_bar:
                progress_bar.close()
            # Verify file size
            if progress_bar.n != total_size and total_size > 0:
                raise ValueError(
                    f"Downloaded size ({progress_bar.n} bytes) does not match expected size ({total_size} bytes)."
                )
            print(f"Download completed: {dest_path}")
            return True
        except requests.Timeout:
            print(f"Timeout error during download: {url}")
        except requests.ConnectionError:
            print(f"Connection error during download: {url}")
        except requests.exceptions.RequestException as e:
            print(f"HTTP error during download: {url}. Details: {str(e)}")
        except ValueError as e:
            print(f"File validation error: {str(e)}")
            if Path(dest_path).exists():
                Path(dest_path).unlink()  # Remove incomplete file
            break  # Stop retrying on file validation errors
        except Exception as e:
            print(f"Unexpected error during download: {str(e)}")
            break  # Stop retrying on unexpected errors
        finally:
            if progress_bar:
                progress_bar.close()
        if Path(dest_path).exists():
            Path(dest_path).unlink()  # Remove incomplete file
        if attempt < retries:
            print(f"Retrying in 5 seconds... (Attempt {attempt + 1}/{retries})")
            time.sleep(5)
        else:
            print(f"Exhausted all retries for {url}.")
    # Return False if all attempts fail
    return False
 def setup_piper():
    """Download base piper files"""
    os.makedirs(this_dir / 'models/piper', exist_ok=True)
    os.makedirs(this_dir / 'models/xtts', exist_ok=True)
    download_file("https://huggingface.co/rhasspy/piper-voices/resolve/v1.0.0/en/en_US/ljspeech/high/en_US-ljspeech-high.onnx?download=true",
                  this_dir / "models/piper/en_US-ljspeech-high.onnx")
    download_file("https://huggingface.co/rhasspy/piper-voices/resolve/v1.0.0/en/en_US/ljspeech/high/en_US-ljspeech-high.onnx.json?download=true.json",
                  this_dir / "models/piper/en_US-ljspeech-high.onnx.json")
 def setup_vits():
    """Download base vits files"""
    os.makedirs(this_dir / 'models/vits', exist_ok=True)
    os.makedirs(this_dir / 'models/xtts', exist_ok=True)
    zip_path = this_dir / 'models/vits/vits.zip'
    download_file("https://coqui.gateway.scarf.sh/v0.6.1_models/tts_models--en--vctk--vits.zip", zip_path)
    with zipfile.ZipFile(zip_path, 'r') as zip_ref:
        zip_ref.extractall(this_dir / 'models/vits')
    os.remove(zip_path)
 def setup_xtts():
    """Download base xtts files"""
    os.makedirs(this_dir / 'models/xtts/xttsv2_2.0.3', exist_ok=True)
    base_url = "https://huggingface.co/coqui/XTTS-v2/resolve/v2.0.3/"
    files = [
        "LICENSE.txt",
        "README.md",
        "config.json",
        "model.pth",
        "dvae.pth",
        "mel_stats.pth",
        "speakers_xtts.pth",
        "vocab.json"
    ]
    for each_file in files:
        download_file(base_url + each_file + "?download=true", this_dir / f"models/xtts/xttsv2_2.0.3/{each_file}")
 def update_tts_engines(engine):
    """Set engine to users choice that matches the download"""
    try:
        config_instance = AlltalkTTSEnginesConfig.get_instance()
        config_instance.change_engine(engine)
        config_instance.save()
    except Exception as e:
        print(f"[{branding}TTS] Error updating TTS engine configuration: {str(e)}")
 def set_firstrun_model_false():
    """
    Marks the first-run setup as complete in the Alltalk configuration.
    This function updates the `firstrun_model` flag in the configuration file to `False`,
    indicating that the initial setup has been completed. It attempts to save the updated 
    configuration and verifies the changes, retrying with a direct file write if necessary.
    """
    try:
        # Get a fresh instance of the config
        config_fr = AlltalkConfig.get_instance()
        config_fr.firstrun_model = False
        # Force a flush to disk immediately
        config_fr.save()
        # Verify the save worked
        config_path = AlltalkConfig.default_config_path()
        with open(config_path, 'r', encoding="utf-8") as f:
            saved_config = json.load(f)
            if saved_config.get('firstrun_model', True):
                print(f"[{branding}TTS] Warning: Failed to save firstrun_model=False")
                # Try one more time with direct file write
                saved_config['firstrun_model'] = False
                with open(config_path, 'w', encoding="utf-8") as f:
                    json.dump(saved_config, f, indent=2)
    except Exception as e:
        print(f"[{branding}TTS] Error saving firstrun configuration: {str(e)}")
 def warning_message():
    """AllTalk V1 Warning message re updating"""
    print(f"[{branding}TTS]")
    print(f"[{branding}TTS] \033[93mIf you have you have UPGRADED from v1 ensure you have re-installed\033[0m")
    print(f"[{branding}TTS] \033[93mthe requirements. Otherwise you will get failures and errors!\033[0m")
    print(f"[{branding}TTS] \033[93mOn Linux ignore the \033[0m'sparse_attn requires a torch version' \033[93mand\033[0m")
    print(f"[{branding}TTS] \033[0m'using untested triton version' \033[93mmessages.\033[0m")
    print(f"[{branding}TTS]")
 # Load confignew.json (might be version 1 or 2):
 with open(AlltalkConfig.default_config_path(), 'r', encoding="utf-8") as file:
    config_json = json.load(file)
 branding = config_json['branding']
 # Check if the config file is version 1
 if "ip_address" in config_json:
    print(f"[{branding}TTS] Detected version 1 configuration file.")
    print(f"[{branding}TTS] Upgrading to version 2 configuration file...")
    # Backup the old configuration file
    backup_path = AlltalkConfig.default_config_path() + ".backup"
    shutil.copyfile(AlltalkConfig.default_config_path(), backup_path)
    # Copy the new configuration template to replace the old config file
    new_config_template_path = this_dir / 'system/config/confignew.json'
    shutil.copyfile(new_config_template_path, AlltalkConfig.default_config_path())
    print(f"[{branding}TTS] Configuration file upgraded successfully.")
 config = AlltalkConfig.get_instance()
 branding = config.branding
 # Argument parser setup
 parser = argparse.ArgumentParser(description="TTS Setup Script")
 parser.add_argument('--tts_model', type=str, choices=['piper', 'vits', 'xtts', 'none'],
                    help="Specify TTS model to set up (piper, vits, xtts, or none)")
 args = parser.parse_args()
 # Check if firstrun_model is true
 if config.firstrun_model:
    # Handle command-line argument for tts_model
    if args.tts_model:
        if args.tts_model == 'piper':
            setup_piper()
            update_tts_engines('piper')
            set_firstrun_model_false()
        elif args.tts_model == 'vits':
            setup_vits()
            update_tts_engines('vits')
            set_firstrun_model_false()
        elif args.tts_model == 'xtts':
            setup_xtts()
            update_tts_engines('xtts')
            set_firstrun_model_false()
        elif args.tts_model == 'none':
            print(f"[{branding}TTS] No TTS model setup requested.")
        print(f"[{branding}TTS] Setup completed for {args.tts_model}. Exiting.")
        sys.exit()
    # Check for Colab or Docker environment first
    if is_running_in_colab() or is_running_in_docker():
        print(f"[{branding}TTS] Detected Colab/Docker environment - automatically selecting Piper")
        setup_piper()
        update_tts_engines('piper')
        set_firstrun_model_false()
        warning_message()
        sys.exit()
    # Present the menu if no argument is passed and not in Colab/Docker
    print(f"[{branding}TTS]")
    print(f"[{branding}TTS] \033[92mThis is the first-time startup.. Please download a start TTS model. Other TTS engines\033[0m")
    print(f"[{branding}TTS] \033[92mand TTS models can be downloaded/managed in the Gradio Interface `TTS Engines Settings`\033[0m")
    print(f"[{branding}TTS] \033[92mtab after initial setup.\033[0m")
    print(f"[{branding}TTS]")
    # List of available models
    models = [
        {"name": "piper", "model": "piper"},
        {"name": "vits", "model": "tts_models--en--vctk--vits"},
        {"name": "xtts", "model": "xttsv2_2.0.3"},
    ]
    # Display models to the user
    print(f"[{branding}TTS]    \033[94mAvailable First Time Start-up models:\033[0m")
    print(f"[{branding}TTS]")
    for idx, model in enumerate(models):
        print(f"[{branding}TTS]    \033[93m{idx + 1}. \033[94m{model['name']} - {model['model']}\033[0m")
    print(f"[{branding}TTS]    \033[93m{len(models) + 1}.\033[94m I have my own models already\033[0m")
    print(f"[{branding}TTS]")
    print(f"[{branding}TTS]    \033[94mIn \033[91m60 seconds\033[0m \033[94ma Piper model will be \033[91mdownloaded automatically.\033[0m")
    print(f"[{branding}TTS]")
    # Auto-select model after 60 seconds
    selected_model = models[0]  # Default to the first model (piper)
    try:
        user_choice = inputimeout(prompt=f"[{branding}TTS]    \033[92mEnter your choice 1-4: \033[0m ", timeout=60)
    except TimeoutOccurred:
        user_choice = None
    if user_choice is None:
        print(f"[{branding}TTS]")
        print(f"[{branding}TTS] No input received. Proceeding with the default model (piper).")
    elif user_choice.lower() == 'own' or (user_choice.isdigit() and int(user_choice) == len(models) + 1):
        print(f"[{branding}TTS]")
        print(f"[{branding}TTS] Please use the Gradio interface TTS Engine Settings > [Engine Name] > Model Download")
        print(f"[{branding}TTS] To download models for your selected TTS Engine. Or use the [Engine Name] help sections")
        print(f"[{branding}TTS] for instructions on using your own TTS models.")
        print(f"[{branding}TTS]")
        os.makedirs('models/piper', exist_ok=True)
        os.makedirs('models/vits', exist_ok=True)
        os.makedirs('models/xtts/', exist_ok=True)
        os.makedirs('models/f5-tts/', exist_ok=True)
        os.makedirs('models/rvc_voices/', exist_ok=True)
        update_tts_engines('piper')
        set_firstrun_model_false()
        warning_message()
        exit()
    if user_choice is not None and user_choice.isdigit() and 1 <= int(user_choice) <= len(models):
        selected_model = models[int(user_choice) - 1]
        warning_message()
    if user_choice is None or user_choice.lower() != 'own':
        if selected_model['name'] == 'piper':
            setup_piper()
            update_tts_engines('piper')
        elif selected_model['name'] == 'vits':
            setup_vits()
            update_tts_engines('vits')
        elif selected_model['name'] == 'xtts':
            setup_xtts()
            update_tts_engines('xtts')
        print(f"[{branding}TTS] {selected_model['name']} model downloaded and configuration updated successfully.")
        # Set firstrun_model to false
        set_firstrun_model_false()
        warning_message()
        sys.exit()
 else:
    sys.exit()
@@ -0,0 +1,287 @@
 import json
 import os
 import shutil
 import sys
 import time
 import zipfile
 import requests
 from tqdm import tqdm
 from inputimeout import inputimeout, TimeoutOccurred
 from pathlib import Path
 from typing import Optional, Dict, List, Any
 # Setting the module search path:
 sys.path.insert(0, str(Path(__file__).parent.parent.parent.resolve()))
 from config import AlltalkTTSEnginesConfig, AlltalkConfig
 this_dir = Path(__file__).parent.resolve().parent.resolve().parent.resolve()
 def is_running_in_colab() -> bool:
    """Check if the script is running in Google Colab."""
    try:
        import google.colab
        return True
    except ImportError:
        return False
 def is_running_in_docker() -> bool:
    """Check if the script is running in a Docker container."""
    return os.path.exists('/.dockerenv') or (
        os.path.exists('/proc/self/cgroup') and 
        any('docker' in line for line in open('/proc/self/cgroup'))
    )
 def download_file(url: str, dest_path: str, timeout: int = 30) -> bool:
    """
    Download a file from a URL with progress bar and error handling.
    Args:
        url: URL to download from
        dest_path: Where to save the file
        timeout: Connection timeout in seconds
    Returns:
        bool: True if download was successful
    """
    progress_bar = None
    try:
        response = requests.get(url, stream=True, timeout=timeout)
        response.raise_for_status()
        total_size = int(response.headers.get('content-length', 0))
        block_size = 1024
        progress_bar = tqdm(total=total_size, unit='iB', unit_scale=True, 
                          desc=f"Downloading {Path(dest_path).name}")
        Path(dest_path).parent.mkdir(parents=True, exist_ok=True)
        with open(dest_path, 'wb') as file:
            for data in response.iter_content(block_size):
                if data:
                    file.write(data)
                    progress_bar.update(len(data))
        if progress_bar.n != total_size and total_size > 0:
            raise requests.exceptions.RequestException(
                f"Incomplete download: {progress_bar.n}/{total_size} bytes"
            )
        return True
    except Exception as e:
        print(f"Error downloading {url}: {str(e)}")
        if Path(dest_path).exists():
            Path(dest_path).unlink()
        return False
    finally:
        if progress_bar:
            progress_bar.close()
 def setup_model(model_name: str) -> bool:
    """
    Set up a specific TTS model.
    Args:
        model_name: Name of the model to setup ('piper', 'vits', or 'xtts')
    Returns:
        bool: True if setup was successful
    """
    try:
        if model_name == "piper":
            os.makedirs(this_dir / 'models/piper', exist_ok=True)
            os.makedirs(this_dir / 'models/xtts', exist_ok=True)
            return all([
                download_file(
                    "https://huggingface.co/rhasspy/piper-voices/resolve/v1.0.0/en/en_US/ljspeech/high/en_US-ljspeech-high.onnx?download=true",
                    this_dir / "models/piper/en_US-ljspeech-high.onnx"
                ),
                download_file(
                    "https://huggingface.co/rhasspy/piper-voices/resolve/v1.0.0/en/en_US/ljspeech/high/en_US-ljspeech-high.onnx.json?download=true.json",
                    this_dir / "models/piper/en_US-ljspeech-high.onnx.json"
                )
            ])
        elif model_name == "vits":
            os.makedirs(this_dir / 'models/vits', exist_ok=True)
            os.makedirs(this_dir / 'models/xtts', exist_ok=True)
            zip_path = this_dir / 'models/vits/vits.zip'
            if not download_file(
                "https://coqui.gateway.scarf.sh/v0.6.1_models/tts_models--en--vctk--vits.zip",
                zip_path
            ):
                return False
            try:
                with zipfile.ZipFile(zip_path, 'r') as zip_ref:
                    zip_ref.extractall(this_dir / 'models/vits')
                os.remove(zip_path)
                return True
            except Exception as e:
                print(f"Error extracting VITS model: {str(e)}")
                return False
        elif model_name == "xtts":
            os.makedirs(this_dir / 'models/xtts/xttsv2_2.0.3', exist_ok=True)
            base_url = "https://huggingface.co/coqui/XTTS-v2/resolve/v2.0.3/"
            files = [
                "LICENSE.txt", "README.md", "config.json", "model.pth",
                "dvae.pth", "mel_stats.pth", "speakers_xtts.pth", "vocab.json"
            ]
            return all(
                download_file(
                    base_url + file + "?download=true",
                    this_dir / f"models/xtts/xttsv2_2.0.3/{file}"
                ) for file in files
            )
        return False
    except Exception as e:
        print(f"Error setting up {model_name}: {str(e)}")
        return False
 def update_tts_engines(engine: str) -> bool:
    """Update TTS engine configuration with error handling."""
    try:
        AlltalkTTSEnginesConfig.get_instance().change_engine(engine).save()
        return True
    except Exception as e:
        print(f"Error updating TTS engine configuration: {str(e)}")
        return False
 def set_firstrun_model_false() -> bool:
    """Set firstrun_model to false with verification."""
    try:
        # Get fresh config instance and update
        config = AlltalkConfig.get_instance()
        config.firstrun_model = False
        config.save()
        # Verify the save worked
        config_path = AlltalkConfig.default_config_path()
        with open(config_path, 'r') as f:
            saved_config = json.load(f)
            if saved_config.get('firstrun_model', True):
                print(f"Warning: Failed to save firstrun_model=False")
                # Try direct file write as fallback
                saved_config['firstrun_model'] = False
                with open(config_path, 'w') as f:
                    json.dump(saved_config, f, indent=2)
        return True
    except Exception as e:
        print(f"Error saving firstrun configuration: {str(e)}")
        return False
 def warning_message(branding: str):
    """Display warning messages about installation and compatibility."""
    print(f"[{branding}TTS]")
    print(f"[{branding}TTS] \033[93mIf you have UPGRADED from v1 ensure you have re-installed\033[0m")
    print(f"[{branding}TTS] \033[93mthe requirements. Otherwise you will get failures and errors!\033[0m")
    print(f"[{branding}TTS] \033[93mOn Linux ignore the \033[0m'sparse_attn requires a torch version' \033[93mand\033[0m")
    print(f"[{branding}TTS] \033[0m'using untested triton version' \033[93mmessages.\033[0m")
    print(f"[{branding}TTS]")
 def main():
    """Main execution function."""
    try:
        # Load configuration
        with open(AlltalkConfig.default_config_path(), 'r') as file:
            config_json = json.load(file)
        branding = config_json['branding']
        # Handle version 1 config upgrade
        if "ip_address" in config_json:
            print(f"[{branding}TTS] Detected version 1 configuration file.")
            print(f"[{branding}TTS] Upgrading to version 2 configuration file...")
            backup_path = AlltalkConfig.default_config_path() + ".backup"
            shutil.copyfile(AlltalkConfig.default_config_path(), backup_path)
            new_config_template_path = this_dir / 'system/config/confignew.json'
            shutil.copyfile(new_config_template_path, AlltalkConfig.default_config_path())
            print(f"[{branding}TTS] Configuration file upgraded successfully.")
        config = AlltalkConfig.get_instance()
        branding = config.branding
        if not config.firstrun_model:
            return
        # Auto-select Piper for Colab/Docker
        if is_running_in_colab() or is_running_in_docker():
            print(f"[{branding}TTS] Detected Colab/Docker environment - automatically selecting Piper")
            if setup_model('piper'):
                update_tts_engines('piper')
                set_firstrun_model_false()
                warning_message(branding)
            return
        # Normal interactive setup
        models = [
            {"name": "piper", "model": "piper"},
            {"name": "vits", "model": "tts_models--en--vctk--vits"},
            {"name": "xtts", "model": "xttsv2_2.0.3"},
        ]
        print(f"[{branding}TTS]")
        print(f"[{branding}TTS] \033[92mThis is the first time startup. Please download a start TTS model.\033[0m")
        print(f"[{branding}TTS]")
        print(f"[{branding}TTS]    \033[94mAvailable First Time Start-up models:\033[0m")
        print(f"[{branding}TTS]")
        for idx, model in enumerate(models):
            print(f"[{branding}TTS]    \033[93m{idx + 1}. \033[94m{model['name']} - {model['model']}\033[0m")
        print(f"[{branding}TTS]    \033[93m{len(models) + 1}.\033[94m I have my own models already\033[0m")
        print(f"[{branding}TTS]")
        print(f"[{branding}TTS]    \033[94mIn \033[91m60 seconds\033[0m \033[94ma Piper model will be \033[91mdownloaded automatically.\033[0m")
        print(f"[{branding}TTS]")
        try:
            user_choice = inputimeout(
                prompt=f"[{branding}TTS]    \033[92mEnter your choice 1-4: \033[0m ",
                timeout=60
            )
        except TimeoutOccurred:
            user_choice = None
        if user_choice is None:
            print(f"[{branding}TTS] No input received. Proceeding with the default model (piper).")
            selected_model = models[0]
        elif user_choice.lower() == 'own' or (user_choice.isdigit() and int(user_choice) == len(models) + 1):
            print(f"[{branding}TTS] If you have your own XTTS models, please move them into the")
            print(f"[{branding}TTS] \033[93m/models/xtts/\033[0m folder")
            os.makedirs('models/piper', exist_ok=True)
            os.makedirs('models/vits', exist_ok=True)
            os.makedirs('models/xtts/', exist_ok=True)
            update_tts_engines('piper')
            set_firstrun_model_false()
            warning_message(branding)
            return
        elif user_choice.isdigit() and 1 <= int(user_choice) <= len(models):
            selected_model = models[int(user_choice) - 1]
            warning_message(branding)
        else:
            print(f"[{branding}TTS] Invalid choice. Proceeding with the default model (piper).")
            selected_model = models[0]
        if setup_model(selected_model['name']):
            update_tts_engines(selected_model['name'])
            set_firstrun_model_false()
            print(f"[{branding}TTS] {selected_model['name']} model downloaded and configuration updated successfully.")
            warning_message(branding)
    except Exception as e:
        print(f"Unexpected error: {str(e)}")
        sys.exit(1)
 if __name__ == "__main__":
    main()
@@ -0,0 +1,92 @@
 #!/bin/bash
 # Get the conda environment path
 env_path="${CONDA_DEFAULT_ENV}/lib"
 # List of broken symlinks
 declare -A symlinks=(
    ["libcufile.so"]="libcufile.so"
    ["libcufile_rdma.so"]="libcufile_rdma.so"
    ["libcudart.so"]="libcudart.so"
    ["libnvjpeg.so"]="libnvjpeg.so"
    ["libcurand.so"]="libcurand.so"
    ["libnvJitLink.so"]="libnvJitLink.so"
    ["libnvrtc-builtins.so"]="libnvrtc-builtins.so"
    ["libnvrtc.so"]="libnvrtc.so"
 )
 # Function to test for broken symlinks
 test_broken_symlinks() {
    echo "Testing for broken symlinks..."
    broken_found=false
    for link in "${!symlinks[@]}"; do
        if [ -L "$env_path/$link" ] && [ ! -e "$env_path/$link" ]; then
            echo "Broken symlink found: $env_path/$link"
            broken_found=true
        fi
    done
    if [ "$broken_found" = false ]; then
        echo "No broken symlinks found."
    fi
 }
 # Function to fix broken symlinks
 fix_broken_symlinks() {
    echo "Fixing broken symlinks..."
    for link in "${!symlinks[@]}"; do
        if [ -L "$env_path/$link" ] && [ ! -e "$env_path/$link" ]; then
            echo "Removing broken link: $env_path/$link"
            rm "$env_path/$link"
            # Find all possible target files
            target_files=($(find "$env_path" -name "${symlinks[$link]}*"))
            if [ ${#target_files[@]} -gt 0 ]; then
                echo "Select a target file for $link:"
                for i in "${!target_files[@]}"; do
                    echo "$((i+1)). ${target_files[$i]}"
                done
                read -p "Enter the number of the target file to use: " choice
                choice=$((choice-1))
                if [[ $choice -ge 0 && $choice -lt ${#target_files[@]} ]]; then
                    target="${target_files[$choice]}"
                    echo "Creating new symlink: $env_path/$link -> $target"
                    ln -s "$target" "$env_path/$link"
                else
                    echo "Invalid choice, skipping $link."
                fi
            else
                echo "No target files found for $link."
            fi
        fi
    done
    echo "Verification of symbolic links:"
    ls -l $env_path | grep -E "libcufile.so|libcudart.so|libcufile_rdma.so|libnvjpeg.so|libcurand.so|libnvJitLink.so|libnvrtc-builtins.so|libnvrtc.so"
 }
 # Menu
 while true; do
    echo "Select an option:"
    echo "1. Test for broken symlinks"
    echo "2. Fix broken symlinks"
    echo "3. Exit"
    read -p "Enter your choice [1-3]: " choice
    case $choice in
        1)
            test_broken_symlinks
            ;;
        2)
            fix_broken_symlinks
            ;;
        3)
            echo "Exiting."
            break
            ;;
        *)
            echo "Invalid choice. Please select 1, 2, or 3."
            ;;
    esac
 done
@@ -0,0 +1,720 @@
 The birch canoe slid on the smooth planks.
 Glue the sheet to the dark blue background.
 It's easy to tell the depth of a well.
 These days a chicken leg is a rare dish.
 Rice is often served in round bowls.
 The juice of lemons makes fine punch.
 The box was thrown beside the parked truck.
 The hogs were fed chopped corn and garbage.
 Four hours of steady work faced us.
 A large size in stockings is hard to sell.
 The boy was there when the sun rose.
 A rod is used to catch pink salmon.
 The source of the huge river is the clear spring.
 Kick the ball straight and follow through.
 Help the woman get back to her feet.
 A pot of tea helps to pass the evening.
 Smoky fires lack flame and heat.
 The soft cushion broke the man's fall.
 The salt breeze came across from the sea.
 The girl at the booth sold fifty bonds.
 The small pup gnawed a hole in the sock.
 The fish twisted and turned on the bent hook.
 Press the pants and sew a button on the vest.
 The swan dive was far short of perfect.
 The beauty of the view stunned the young boy.
 Two blue fish swam in the tank.
 Her purse was full of useless trash.
 The colt reared and threw the tall rider.
 It snowed, rained, and hailed the same morning.
 Read verse out loud for pleasure.
 Hoist the load to your left shoulder.
 Take the winding path to reach the lake.
 Note closely the size of the gas tank.
 Wipe the grease off his dirty face.
 Mend the coat before you go out.
 The wrist was badly strained and hung limp.
 The stray cat gave birth to kittens.
 The young girl gave no clear response.
 The meal was cooked before the bell rang.
 What joy there is in living.
 A king ruled the state in the early days.
 The ship was torn apart on the sharp reef.
 Sickness kept him home the third week.
 The wide road shimmered in the hot sun.
 The lazy cow lay in the cool grass.
 Lift the square stone over the fence.
 The rope will bind the seven books at once.
 Hop over the fence and plunge in.
 The friendly gang left the drug store.
 Mesh wire keeps chicks inside.
 The frosty air passed through the coat.
 The crooked maze failed to fool the mouse.
 Adding fast leads to wrong sums.
 The show was a flop from the very start.
 A saw is a tool used for making boards.
 The wagon moved on well oiled wheels.
 March the soldiers past the next hill.
 A cup of sugar makes sweet fudge.
 Place a rosebush near the porch steps.
 Both lost their lives in the raging storm.
 We talked of the side show in the circus.
 Use a pencil to write the first draft.
 He ran half way to the hardware store.
 The clock struck to mark the third period.
 A small creek cut across the field.
 Cars and busses stalled in snow drifts.
 The set of china hit the floor with a crash.
 This is a grand season for hikes on the road.
 The dune rose from the edge of the water.
 Those words were the cue for the actor to leave.
 A yacht slid around the point into the bay.
 The two met while playing on the sand.
 The ink stain dried on the finished page.
 The walled town was seized without a fight.
 The lease ran out in sixteen weeks.
 A tame squirrel makes a nice pet.
 The horn of the car woke the sleeping cop.
 The heart beat strongly and with firm strokes.
 The pearl was worn in a thin silver ring.
 The fruit peel was cut in thick slices.
 The Navy attacked the big task force.
 See the cat glaring at the scared mouse.
 There are more than two factors here.
 The hat brim was wide and too droopy.
 The lawyer tried to lose his case.
 The grass curled around the fence post.
 Cut the pie into large parts.
 Men strive but seldom get rich.
 Always close the barn door tight.
 He lay prone and hardly moved a limb.
 The slush lay deep along the street.
 A wisp of cloud hung in the blue air.
 A pound of sugar costs more than eggs.
 The fin was sharp and cut the clear water.
 The play seems dull and quite stupid.
 Bail the boat to stop it from sinking.
 The term ended in late June that year.
 A tusk is used to make costly gifts.
 Ten pins were set in order.
 The bill was paid every third week.
 Oak is strong and also gives shade.
 Cats and dogs each hate the other.
 The pipe began to rust while new.
 Open the crate but don't break the glass.
 Add the sum to the product of these three.
 Thieves who rob friends deserve jail.
 The ripe taste of cheese improves with age.
 Act on these orders with great speed.
 The hog crawled under the high fence.
 Move the vat over the hot fire.
 The bark of the pine tree was shiny and dark.
 Leaves turn brown and yellow in the fall.
 The pennant waved when the wind blew.
 Split the log with a quick, sharp blow.
 Burn peat after the logs give out.
 He ordered peach pie with ice cream.
 Weave the carpet on the right hand side.
 Hemp is a weed found in parts of the tropics.
 A lame back kept his score low.
 We find joy in the simplest things.
 Type out three lists of orders.
 The harder he tried the less he got done.
 The boss ran the show with a watchful eye.
 The cup cracked and spilled its contents.
 Paste can cleanse the most dirty brass.
 The slang word for raw whiskey is booze.
 It caught its hind paw in a rusty trap.
 The wharf could be seen at the farther shore.
 Feel the heat of the weak dying flame.
 The tiny girl took off her hat.
 A cramp is no small danger on a swim.
 He said the same phrase thirty times.
 Pluck the bright rose without leaves.
 Two plus seven is less than ten.
 The glow deepened in the eyes of the sweet girl.
 Bring your problems to the wise chief.
 Write a fond note to the friend you cherish.
 Clothes and lodging are free to new men.
 We frown when events take a bad turn.
 Port is a strong wine with a smoky taste.
 The young kid jumped the rusty gate.
 Guess the results from the first scores.
 A salt pickle tastes fine with ham.
 The just claim got the right verdict.
 These thistles bend in a high wind.
 Pure bred poodles have curls.
 The tree top waved in a graceful way.
 The spot on the blotter was made by green ink.
 Mud was spattered on the front of his white shirt.
 The cigar burned a hole in the desk top.
 The empty flask stood on the tin tray.
 A speedy man can beat this track mark.
 He broke a new shoelace that day.
 The coffee stand is too high for the couch.
 The urge to write short stories is rare.
 The pencils have all been used.
 The pirates seized the crew of the lost ship.
 We tried to replace the coin but failed.
 She sewed the torn coat quite neatly.
 The sofa cushion is red and of light weight.
 The jacket hung on the back of the wide chair.
 At that high level the air is pure.
 Drop the two when you add the figures.
 A filing case is now hard to buy.
 An abrupt start does not win the prize.
 Wood is best for making toys and blocks.
 The office paint was a dull, sad tan.
 He knew the skill of the great young actress.
 A rag will soak up spilled water.
 A shower of dirt fell from the hot pipes.
 Steam hissed from the broken valve.
 The child almost hurt the small dog.
 There was a sound of dry leaves outside.
 The sky that morning was clear and bright blue.
 Torn scraps littered the stone floor.
 Sunday is the best part of the week.
 The doctor cured him with these pills.
 The new girl was fired today at noon.
 They felt gay when the ship arrived in port.
 Add the store's account to the last cent.
 Acid burns holes in wool cloth.
 Fairy tales should be fun to write.
 Eight miles of woodland burned to waste.
 The third act was dull and tired the players.
 A young child should not suffer fright.
 Add the column and put the sum here.
 We admire and love a good cook.
 There the flood mark is ten inches.
 He carved a head from the round block of marble.
 She has a smart way of wearing clothes.
 The fruit of a fig tree is apple-shaped.
 Corn cobs can be used to kindle a fire.
 Where were they when the noise started.
 The paper box is full of thumb tacks.
 Sell your gift to a buyer at a good gain.
 The tongs lay beside the ice pail.
 The petals fall with the next puff of wind.
 Bring your best compass to the third class.
 They could laugh although they were sad.
 Farmers came in to thresh the oat crop.
 The brown house was on fire to the attic.
 The lure is used to catch trout and flounder.
 Float the soap on top of the bath water.
 A blue crane is a tall wading bird.
 A fresh start will work such wonders.
 The club rented the rink for the fifth night.
 After the dance, they went straight home.
 The hostess taught the new maid to serve.
 He wrote his last novel there at the inn.
 Even the worst will beat his low score.
 The cement had dried when he moved it.
 The loss of the second ship was hard to take.
 The fly made its way along the wall.
 Do that with a wooden stick.
 Live wires should be kept covered.
 The large house had hot water taps.
 It is hard to erase blue or red ink.
 Write at once or you may forget it.
 The doorknob was made of bright clean brass.
 The wreck occurred by the bank on Main Street.
 A pencil with black lead writes best.
 Coax a young calf to drink from a bucket.
 Schools for ladies teach charm and grace.
 The lamp shone with a steady green flame.
 They took the axe and the saw to the forest.
 The ancient coin was quite dull and worn.
 The shaky barn fell with a loud crash.
 Jazz and swing fans like fast music.
 Rake the rubbish up and then burn it.
 Slash the gold cloth into fine ribbons.
 Try to have the court decide the case.
 They are pushed back each time they attack.
 He broke his ties with groups of former friends.
 They floated on the raft to sun their white backs.
 The map had an X that meant nothing.
 Whitings are small fish caught in nets.
 Some ads serve to cheat buyers.
 Jerk the rope and the bell rings weakly.
 A waxed floor makes us lose balance.
 Madam, this is the best brand of corn.
 On the islands the sea breeze is soft and mild.
 The play began as soon as we sat down.
 This will lead the world to more sound and fury.
 Add salt before you fry the egg.
 The rush for funds reached its peak Tuesday.
 The birch looked stark white and lonesome.
 The box is held by a bright red snapper.
 To make pure ice, you freeze water.
 The first worm gets snapped early.
 Jump the fence and hurry up the bank.
 Yell and clap as the curtain slides back.
 They are men who walk the middle of the road.
 Both brothers wear the same size.
 In some form or other we need fun.
 The prince ordered his head chopped off.
 The houses are built of red clay bricks.
 Ducks fly north but lack a compass.
 Fruit flavors are used in fizz drinks.
 These pills do less good than others.
 Canned pears lack full flavor.
 The dark pot hung in the front closet.
 Carry the pail to the wall and spill it there.
 The train brought our hero to the big town.
 We are sure that one war is enough.
 Gray paint stretched for miles around.
 The rude laugh filled the empty room.
 High seats are best for football fans.
 Tea served from the brown jug is tasty.
 A dash of pepper spoils beef stew.
 A zestful food is the hot-cross bun.
 The horse trotted around the field at a brisk pace.
 Find the twin who stole the pearl necklace.
 Cut the cord that binds the box tightly.
 The red tape bound the smuggled food.
 Look in the corner to find the tan shirt.
 The cold drizzle will halt the bond drive.
 Nine men were hired to dig the ruins.
 The junk yard had a mouldy smell.
 The flint sputtered and lit a pine torch.
 Soak the cloth and drown the sharp odor.
 The shelves were bare of both jam or crackers.
 A joy to every child is the swan boat.
 All sat frozen and watched the screen.
 A cloud of dust stung his tender eyes.
 To reach the end he needs much courage.
 Shape the clay gently into block form.
 A ridge on a smooth surface is a bump or flaw.
 Hedge apples may stain your hands green.
 Quench your thirst, then eat the crackers.
 Tight curls get limp on rainy days.
 The mute muffled the high tones of the horn.
 The gold ring fits only a pierced ear.
 The old pan was covered with hard fudge.
 Watch the log float in the wide river.
 The node on the stalk of wheat grew daily.
 The heap of fallen leaves was set on fire.
 Write fast if you want to finish early.
 His shirt was clean but one button was gone.
 The barrel of beer was a brew of malt and hops.
 Tin cans are absent from store shelves.
 Slide the box into that empty space.
 The plant grew large and green in the window.
 The beam dropped down on the workmen's head.
 Pink clouds floated with the breeze.
 She danced like a swan, tall and graceful.
 The tube was blown and the tire flat and useless.
 It is late morning on the old wall clock.
 Let's all join as we sing the last chorus.
 The last switch cannot be turned off.
 The fight will end in just six minutes.
 The store walls were lined with colored frocks.
 The peace league met to discuss their plans.
 The rise to fame of a person takes luck.
 Paper is scarce, so write with much care.
 The quick fox jumped on the sleeping cat.
 The nozzle of the fire hose was bright brass.
 Screw the round cap on as tight as needed.
 Time brings us many changes.
 The purple tie was ten years old.
 Men think and plan and sometimes act.
 Fill the ink jar with sticky glue.
 He smoke a big pipe with strong contents.
 We need grain to keep our mules healthy.
 Pack the records in a neat thin case.
 The crunch of feet in the snow was the only sound.
 The copper bowl shone in the sun's rays.
 Boards will warp unless kept dry.
 The plush chair leaned against the wall.
 Glass will clink when struck by metal.
 Bathe and relax in the cool green grass.
 Nine rows of soldiers stood in line.
 The beach is dry and shallow at low tide.
 The idea is to sew both edges straight.
 The kitten chased the dog down the street.
 Pages bound in cloth make a book.
 Try to trace the fine lines of the painting.
 Women form less than half of the group.
 The zones merge in the central part of town.
 A gem in the rough needs work to polish.
 Code is used when secrets are sent.
 Most of the news is easy for us to hear.
 He used the lathe to make brass objects.
 The vane on top of the pole revolved in the wind.
 Mince pie is a dish served to children.
 The clan gathered on each dull night.
 Let it burn, it gives us warmth and comfort.
 A castle built from sand fails to endure.
 A child's wit saved the day for us.
 Tack the strip of carpet to the worn floor.
 Next Tuesday we must vote.
 Pour the stew from the pot into the plate.
 Each penny shone like new.
 The man went to the woods to gather sticks.
 The dirt piles were lines along the road.
 The logs fell and tumbled into the clear stream.
 Just hoist it up and take it away.
 A ripe plum is fit for a king's palate.
 Our plans right now are hazy.
 Brass rings are sold by these natives.
 It takes a good trap to capture a bear.
 Feed the white mouse some flower seeds.
 The thaw came early and freed the stream.
 He took the lead and kept it the whole distance.
 The key you designed will fit the lock.
 Plead to the council to free the poor thief.
 Better hash is made of rare beef.
 This plank was made for walking on.
 The lake sparkled in the red hot sun.
 He crawled with care along the ledge.
 Tend the sheep while the dog wanders.
 It takes a lot of help to finish these.
 Mark the spot with a sign painted red.
 Take two shares as a fair profit.
 The fur of cats goes by many names.
 North winds bring colds and fevers.
 He asks no person to vouch for him.
 Go now and come here later.
 A sash of gold silk will trim her dress.
 Soap can wash most dirt away.
 That move means the game is over.
 He wrote down a long list of items.
 A siege will crack the strong defense.
 Grape juice and water mix well.
 Roads are paved with sticky tar.
 Fake stones shine but cost little.
 The drip of the rain made a pleasant sound.
 Smoke poured out of every crack.
 Serve the hot rum to the tired heroes.
 Much of the story makes good sense.
 The sun came up to light the eastern sky.
 Heave the line over the port side.
 A lathe cuts and trims any wood.
 It's a dense crowd in two distinct ways.
 His hip struck the knee of the next player.
 The stale smell of old beer lingers.
 The desk was firm on the shaky floor.
 It takes heat to bring out the odor.
 Beef is scarcer than some lamb.
 Raise the sail and steer the ship northward.
 A cone costs five cents on Mondays.
 A pod is what peas always grow in.
 Jerk the dart from the cork target.
 No cement will hold hard wood.
 We now have a new base for shipping.
 A list of names is carved around the base.
 The sheep were led home by a dog.
 Three for a dime, the young peddler cried.
 The sense of smell is better than that of touch.
 No hardship seemed to keep him sad.
 Grace makes up for lack of beauty.
 Nudge gently but wake her now.
 The news struck doubt into restless minds.
 Once we stood beside the shore.
 A chink in the wall allowed a draft to blow.
 Fasten two pins on each side.
 A cold dip restores health and zest.
 He takes the oath of office each March.
 The sand drifts over the sill of the old house.
 The point of the steel pen was bent and twisted.
 There is a lag between thought and act.
 Seed is needed to plant the spring corn.
 Draw the chart with heavy black lines.
 The boy owed his pal thirty cents.
 The chap slipped into the crowd and was lost.
 Hats are worn to tea and not to dinner.
 The ramp led up to the wide highway.
 Beat the dust from the rug onto the lawn.
 Say it slowly but make it ring clear.
 The straw nest housed five robins.
 Screen the porch with woven straw mats.
 This horse will nose his way to the finish.
 The dry wax protects the deep scratch.
 He picked up the dice for a second roll.
 These coins will be needed to pay his debt.
 The nag pulled the frail cart along.
 Twist the valve and release hot steam.
 The vamp of the shoe had a gold buckle.
 The smell of burned rags itches my nose.
 New pants lack cuffs and pockets.
 The marsh will freeze when cold enough.
 They slice the sausage thin with a knife.
 The bloom of the rose lasts a few days.
 A gray mare walked before the colt.
 Breakfast buns are fine with a hot drink.
 Bottles hold four kinds of rum.
 The man wore a feather in his felt hat.
 He wheeled the bike past the winding road.
 Drop the ashes on the worn old rug.
 The desk and both chairs were painted tan.
 Throw out the used paper cup and plate.
 A clean neck means a neat collar.
 The couch cover and hall drapes were blue.
 The stems of the tall glasses cracked and broke.
 The wall phone rang loud and often.
 The clothes dried on a thin wooden rack.
 Turn on the lantern which gives us light.
 The cleat sank deeply into the soft turf.
 The bills were mailed promptly on the tenth of the month.
 To have is better than to wait and hope.
 The price is fair for a good antique clock.
 The music played on while they talked.
 Dispense with a vest on a day like this.
 The bunch of grapes was pressed into wine.
 He sent the figs, but kept the ripe cherries.
 The hinge on the door creaked with old age.
 The screen before the fire kept in the sparks.
 Fly by night, and you waste little time.
 Thick glasses helped him read the print.
 Birth and death mark the limits of life.
 The chair looked strong but had no bottom.
 The kite flew wildly in the high wind.
 A fur muff is stylish once more.
 The tin box held priceless stones.
 We need an end of all such matter.
 The case was puzzling to the old and wise.
 The bright lanterns were gay on the dark lawn.
 We don't get much money but we have fun.
 The youth drove with zest, but little skill.
 Five years he lived with a shaggy dog.
 A fence cuts through the corner lot.
 The way to save money is not to spend much.
 Shut the hatch before the waves push it in.
 The odor of spring makes young hearts jump.
 Crack the walnut with your sharp side teeth.
 He offered proof in the form of a large chart.
 Send the stuff in a thick paper bag.
 A quart of milk is water for the most part.
 They told wild tales to frighten him.
 The three story house was built of stone.
 In the rear of the ground floor was a large passage.
 A man in a blue sweater sat at the desk.
 Oats are a food eaten by horse and man.
 Their eyelids droop for want of sleep.
 A sip of tea revives his tired friend.
 There are many ways to do these things.
 Tuck the sheet under the edge of the mat.
 A force equal to that would move the earth.
 We like to see clear weather.
 The work of the tailor is seen on each side.
 Take a chance and win a china doll.
 Shake the dust from your shoes, stranger.
 She was kind to sick old people.
 The square wooden crate was packed to be shipped.
 The dusty bench stood by the stone wall.
 We dress to suit the weather of most days.
 Smile when you say nasty words.
 A bowl of rice is free with chicken stew.
 The water in this well is a source of good health.
 Take shelter in this tent, but keep still.
 That guy is the writer of a few banned books.
 The little tales they tell are false.
 The door was barred, locked, and bolted as well.
 Ripe pears are fit for a queen's table.
 A big wet stain was on the round carpet.
 The kite dipped and swayed, but stayed aloft.
 The pleasant hours fly by much too soon.
 The room was crowded with a wild mob.
 This strong arm shall shield your honor.
 She blushed when he gave her a white orchid.
 The beetle droned in the hot June sun.
 Press the pedal with your left foot.
 Neat plans fail without luck.
 The black trunk fell from the landing.
 The bank pressed for payment of the debt.
 The theft of the pearl pin was kept secret.
 Shake hands with this friendly child.
 The vast space stretched into the far distance.
 A rich farm is rare in this sandy waste.
 His wide grin earned many friends.
 Flax makes a fine brand of paper.
 Hurdle the pit with the aid of a long pole.
 A strong bid may scare your partner stiff.
 Even a just cause needs power to win.
 Peep under the tent and see the clowns.
 The leaf drifts along with a slow spin.
 Cheap clothes are flashy but don't last.
 A thing of small note can cause despair.
 Flood the mails with requests for this book.
 A thick coat of black paint covered all.
 The pencil was cut to be sharp at both ends.
 Those last words were a strong statement.
 He wrote his name boldly at the top of the sheet.
 Dill pickles are sour but taste fine.
 Down that road is the way to the grain farmer.
 Either mud or dust are found at all times.
 The best method is to fix it in place with clips.
 If you mumble your speech will be lost.
 At night the alarm roused him from a deep sleep.
 Read just what the meter says.
 Fill your pack with bright trinkets for the poor.
 The small red neon lamp went out.
 Clams are small, round, soft, and tasty.
 The fan whirled its round blades softly.
 The line where the edges join was clean.
 Breathe deep and smell the piny air.
 It matters not if he reads these words or those.
 A brown leather bag hung from its strap.
 A toad and a frog are hard to tell apart.
 A white silk jacket goes with any shoes.
 A break in the dam almost caused a flood.
 Paint the sockets in the wall dull green.
 The child crawled into the dense grass.
 Bribes fail where honest men work.
 Trample the spark, else the flames will spread.
 The hilt of the sword was carved with fine designs.
 A round hole was drilled through the thin board.
 Footprints showed the path he took up the beach.
 She was waiting at my front lawn.
 A vent near the edge brought in fresh air.
 Prod the old mule with a crooked stick.
 It is a band of steel three inches wide.
 The pipe ran almost the length of the ditch.
 It was hidden from sight by a mass of leaves and shrubs.
 The weight of the package was seen on the high scale.
 Wake and rise, and step into the green outdoors.
 The green light in the brown box flickered.
 The brass tube circled the high wall.
 The lobes of her ears were pierced to hold rings.
 Hold the hammer near the end to drive the nail.
 Next Sunday is the twelfth of the month.
 Every word and phrase he speaks is true.
 He put his last cartridge into the gun and fired.
 They took their kids from the public school.
 Drive the screw straight into the wood.
 Keep the hatch tight and the watch constant.
 Sever the twine with a quick snip of the knife.
 Paper will dry out when wet.
 Slide the catch back and open the desk.
 Help the weak to preserve their strength.
 A sullen smile gets few friends.
 Stop whistling and watch the boys march.
 Jerk the cord, and out tumbles the gold.
 Slide the tray across the glass top.
 The cloud moved in a stately way and was gone.
 Light maple makes for a swell room.
 Set the piece here and say nothing.
 Dull stories make her laugh.
 A stiff cord will do to fasten your shoe.
 Get the trust fund to the bank early.
 Choose between the high road and the low.
 A plea for funds seems to come again.
 He lent his coat to the tall gaunt stranger.
 There is a strong chance it will happen once more.
 The duke left the park in a silver coach.
 Greet the new guests and leave quickly.
 When the frost has come it is time for turkey.
 Sweet words work better than fierce.
 A thin stripe runs down the middle.
 A six comes up more often than a ten.
 Lush fern grow on the lofty rocks.
 The ram scared the school children off.
 The team with the best timing looks good.
 The farmer swapped his horse for a brown ox.
 Sit on the perch and tell the others what to do.
 A steep trail is painful for our feet.
 The early phase of life moves fast.
 Green moss grows on the northern side.
 Tea in thin china has a sweet taste.
 Pitch the straw through the door of the stable.
 The latch on the back gate needed a nail.
 The goose was brought straight from the old market.
 The sink is the thing in which we pile dishes.
 A whiff of it will cure the most stubborn cold.
 The facts don't always show who is right.
 She flaps her cape as she parades the street.
 The loss of the cruiser was a blow to the fleet.
 Loop the braid to the left and then over.
 Plead with the lawyer to drop the lost cause.
 Calves thrive on tender spring grass.
 Post no bills on this office wall.
 Tear a thin sheet from the yellow pad.
 A cruise in warm waters in a sleek yacht is fun.
 A streak of color ran down the left edge.
 It was done before the boy could see it.
 Crouch before you jump or miss the mark.
 Pack the kits and don't forget the salt.
 The square peg will settle in the round hole.
 Fine soap saves tender skin.
 Poached eggs and tea must suffice.
 Bad nerves are jangled by a door slam.
 Ship maps are different from those for planes.
 Dimes showered down from all sides.
 They sang the same tunes at each party.
 The sky in the west is tinged with orange red.
 The pods of peas ferment in bare fields.
 The horse balked and threw the tall rider.
 The hitch between the horse and cart broke.
 Pile the coal high in the shed corner.
 A gold vase is both rare and costly.
 The knife was hung inside its bright sheath.
 The rarest spice comes from the far East.
 The roof should be tilted at a sharp slant.
 A smatter of French is worse than none.
 The mule trod the treadmill day and night.
 The aim of the contest is to raise a great fund.
 To send it now in large amounts is bad.
 There is a fine hard tang in salty air.
 Cod is the main business of the north shore.
 The slab was hewn from heavy blocks of slate.
 Dunk the stale biscuits into strong drink.
 Hang tinsel from both branches.
 Cap the jar with a tight brass cover.
 The poor boy missed the boat again.
 Be sure to set the lamp firmly in the hole.
 Pick a card and slip it under the pack.
 A round mat will cover the dull spot.
 The first part of the plan needs changing.
 A good book informs of what we ought to know.
 The mail comes in three batches per day.
 You cannot brew tea in a cold pot.
 Dots of light betrayed the black cat.
 Put the chart on the mantel and tack it down.
 The night shift men rate extra pay.
 The red paper brightened the dim stage.
 See the player scoot to third base.
 Slide the bill between the two leaves.
 Many hands help get the job done.
 We don't like to admit our small faults.
 No doubt about the way the wind blows.
 Dig deep in the earth for pirate's gold.
 The steady drip is worse than a drenching rain.
 A flat pack takes less luggage space.
 Green ice frosted the punch bowl.
 A stuffed chair slipped from the moving van.
 The stitch will serve but needs to be shortened.
 A thin book fits in the side pocket.
 The gloss on top made it unfit to read.
 The hail pattered on the burnt brown grass.
 Seven seals were stamped on great sheets.
 Our troops are set to strike heavy blows.
 The store was jammed before the sale could start.
 It was a bad error on the part of the new judge.
 One step more and the board will collapse.
 Take the match and strike it against your shoe.
 The pot boiled, but the contents failed to jell.
 The baby puts his right foot in his mouth.
 The bombs left most of the town in ruins.
 Stop and stare at the hard working man.
 The streets are narrow and full of sharp turns.
 The pup jerked the leash as he saw a feline shape.
 Open your book to the first page.
 Fish evade the net and swim off.
 Dip the pail once and let it settle.
 Will you please answer that phone.
 The big red apple fell to the ground.
 The curtain rose and the show was on.
 The young prince became heir to the throne.
 He sent the boy on a short errand.
 Leave now and you will arrive on time.
 The corner store was robbed last night.
 A gold ring will please most any girl.
 The long journey home took a year.
 She saw a cat in the neighbor's house.
 A pink shell was found on the sandy beach.
 Small children came to see him.
 The grass and bushes were wet with dew.
 The blind man counted his old coins.
 A severe storm tore down the barn.
 She called his name many times.
 When you hear the bell, come quickly.
@@ -0,0 +1,18 @@
 {
  "Arabic": "ar",
  "Chinese": "zh-cn",
  "Czech": "cs",
  "Dutch": "nl",
  "English": "en",
  "French": "fr",
  "German": "de",
  "Hungarian": "hu",
  "Italian": "it",
  "Japanese": "ja",
  "Korean": "ko",
  "Polish": "pl",
  "Portuguese": "pt",
  "Russian": "ru",
  "Spanish": "es",
  "Turkish": "tr"
 }
@@ -0,0 +1,13 @@
 find {addpathheretoscan}/alltalk_environment/env/lib -xtype l
 here are the list on Cuda-Toolkit 12.1, others vary
 .../alltalk_environment/env/lib/libcufile.so
 .../alltalk_environment/env/lib/libcudart.so
 .../alltalk_environment/env/lib/libcufile_rdma.so
 .../alltalk_environment/env/lib/libnvjpeg.so
 .../alltalk_environment/env/lib/libcurand.so
 .../alltalk_environment/env/lib/libnvJitLink.so
 .../alltalk_environment/env/lib/libnvrtc-builtins.so
 .../alltalk_environment/env/lib/libnvrtc.so
@@ -0,0 +1,54 @@
 import json
 import os
 def merge_vocabularies(base_vocab_path, new_vocab_path, output_path):
    # Load the base model's vocab.json
    with open(base_vocab_path, 'r') as f:
        base_data = json.load(f)
    # Load the new bpe_tokenizer.json
    with open(new_vocab_path, 'r') as f:
        new_data = json.load(f)
    # Extract the vocabularies
    base_vocab = base_data['model']['vocab']
    new_vocab = new_data['model']['vocab']
    # Find the maximum value in the base vocabulary
    max_value = max(base_vocab.values())
    # Merge the vocabularies
    for key, value in new_vocab.items():
        if key not in base_vocab:
            max_value += 1
            base_vocab[key] = max_value
    # Update the base data with the merged vocabulary
    base_data['model']['vocab'] = base_vocab
    # Extract the merges
    base_merges = base_data['model']['merges']
    new_merges = new_data['model']['merges']
    # Merge the merges
    merged_merges = base_merges.copy()
    for merge in new_merges:
        if merge not in merged_merges:
            merged_merges.append(merge)
    # Update the base data with the merged merges
    base_data['model']['merges'] = merged_merges
    # Write the merged vocabulary and merges to the output file
    with open(output_path, 'w') as f:
        json.dump(base_data, f, ensure_ascii=False, indent=2)
    print(f"Merged vocabulary and merges saved to {output_path}")
 # Define file paths
 base_vocab_path = "/alltalk_tts/models/xtts/xttsv2_2.0.2/originalvocab.json" # base model vocab.json path (2.0.2)
 new_vocab_path = "/alltalk_tts/expanded_models/tortoise_tokenizer2xttsv2.json" # path to the custom dataset vocab.json
 output_path = "/alltalk_tts/expanded_models/combined_vocab.json" # location for combined vocab.json
 # Merge the vocabularies
 merge_vocabularies(base_vocab_path, new_vocab_path, output_path)
@@ -0,0 +1,60 @@
 # Credit Jarod Mica https://github.com/JarodMica/tortoise_dataset_tools/blob/master/bpe_tokenizer_tools/train_bpe_tokenizer.py
 # provide a cleaned txt file with only the transcription. Youll get back out the datasets vocab.json
 from tokenizers import Tokenizer
 from tokenizers.models import BPE
 from tokenizers.trainers import BpeTrainer
 from tokenizers.pre_tokenizers import Whitespace
 from tkinter import Tk, filedialog
 import json
 import re
 def clean_text(input_file_path, output_file_path):
    # Define the pattern to match numbers, specific symbols, and new lines
    # add \d to match any digit, and | is used to specify alternatives
    pattern = r'|�|«|\$|\n'
    with open(input_file_path, 'r', encoding='utf-8') as input_file:
        text = input_file.read()
        cleaned_text = re.sub(pattern, '', text)
    with open(output_file_path, 'w', encoding='utf-8') as output_file:
        output_file.write(cleaned_text)
 def train_tokenizer(input_path, tokenizer_path, language, special_tokens=["[STOP]", "[UNK]", "[SPACE]" ], vocab_size=256):
    # Initialize a tokenizer with the BPE model
    tokenizer = Tokenizer(BPE(unk_token="[UNK]"))
    # Use a basic whitespace pre-tokenizer
    tokenizer.pre_tokenizer = Whitespace()
    # trainer = BpeTrainer(special_tokens=["[STOP]", "[UNK]", "[SPACE]", "0","1","2","3","4","5","6","7","8","9",], vocab_size=256)
    trainer = BpeTrainer(special_tokens=special_tokens, vocab_size=vocab_size)
    clean_text(input_path, input_path)
    tokenizer.train([input_path], trainer)
    tokenizer.save(tokenizer_path)
    with open(tokenizer_path, 'r', encoding='utf-8') as f:
        tokenizer_json = json.load(f)
    # Add language to tokenizer
    tokenizer_json['model']['language'] = language
    with open(tokenizer_path, 'w', encoding='utf-8') as f:
        json.dump(tokenizer_json, f, ensure_ascii=False, indent=4)
 def choose_file():
    root = Tk()
    root.withdraw()
    file = filedialog.askopenfilename()
    root.destroy()
    return file
 if __name__ == "__main__":
    input_path = choose_file()
    tokenizer_path = "/alltalk_tts/expanded_models/tortoise_tokenizer2xttsv2.json" # define path to put the newly create vocab.json
    special_tokens = ["[STOP]", "[UNK]", "[SPACE]"]
    vocab_size = 256 # model is stuck at this size
    train_tokenizer(input_path, tokenizer_path, language='multi', special_tokens=special_tokens, vocab_size=vocab_size)
@@ -0,0 +1,121 @@
 """
 This script does not have any specific alltalk integration yet. So we are manually entering paths and it works as a standalone.
 The script does the following:
    - Expands the embedding layer of the base XTTSv2 model according to the user created/trained bpe_tokenizer-vocab.json and the base model vocab.json.
 Set variable paths with the base config and base model.pth. 
 Set the new tokenizer/vocab bpe_tokenizer-vocab.json location.
 The new model will be saved at \expanded_models\expanded_model.pth
 Once this is done the new expanded model must be swapped in for the base model.pth then combined with the dvae.pth, vocab.json(bpe_tokenizer-vocab.json), base model config.json,
 base model speaker_xtts.pth and base model vocoder.json.
 I left some print debug statements in the script, they may be nice for the user to see during the process.
 """
 import torch
 import torch.nn as nn
 import json
 from TTS.tts.models.xtts import Xtts
 from TTS.tts.layers.xtts.trainer.gpt_trainer import GPTTrainerConfig
 from TTS.tts.layers.xtts.tokenizer import VoiceBpeTokenizer
 config_path = "/alltalk_tts/models/xtts/xttsv2_2.0.3/config.json"                        # Path to the base model config.json
 pretrained_model_path = "/alltalk_tts/models/xtts/xttsv2_2.0.3/model.pth"                # Path to the base model.pth
 new_tokenizer_path = "/expanded_model/expanded_vocab.json"                               # Path to the new combined expanded_vocab.json
 expanded_model_path = "/expanded_model/expanded_model.pth"                               # Path to where you want the new expanded_model.pth
 # Open and load the configuration file
 with open(config_path, "r") as f:
    config_dict = json.load(f)
 # Create a GPTTrainerConfig object and populate it with the loaded configuration
 config = GPTTrainerConfig()
 config.from_dict(data=config_dict)
 # Function to get the vocabulary size from a tokenizer file
 def get_vocab_size(tokenizer_path):
    tokenizer = VoiceBpeTokenizer(vocab_file=tokenizer_path)
    return len(tokenizer.tokenizer.get_vocab())
 # Function to adjust the pretrained model with a new tokenizer
 def adjust_pretrained_model(
    pretrained_model_path, adjusted_model_path, new_tokenizer_path):
    state_dict = torch.load(pretrained_model_path)
    pretrained_state_dict = state_dict["model"]
    model = Xtts(config)
    # Load the pretrained state dictionary into the new model
    missing_keys, unexpected_keys = model.load_state_dict(pretrained_state_dict, strict=False)
    if missing_keys:
        print(f"Missing keys: {missing_keys}")
    if unexpected_keys:
        print(f"Unexpected keys: {unexpected_keys}")
    print("Pretrained model loaded successfully.")
    # Create a new tokenizer with the new vocabulary
    new_tokenizer = VoiceBpeTokenizer(vocab_file=new_tokenizer_path)
    # Get the old and new vocabulary sizes, and the embedding dimension
    old_vocab_size = model.gpt.text_embedding.num_embeddings
    new_vocab_size = len(new_tokenizer.tokenizer.get_vocab())
    embedding_dim = model.gpt.text_embedding.embedding_dim
    print(f"Old vocab size: {old_vocab_size}")
    print(f"New vocab size: {new_vocab_size}")
    print(f"Embedding dimension: {embedding_dim}")
    # Adjust the embedding layer with the new vocabulary size
    adjust_embedding_layer(model, new_vocab_size, adjusted_model_path)
    # Freeze all parameters except the position embeddings
    freeze_except_position_embeddings(model)
    # Function to adjust the embedding layer for the new vocabulary size
 def adjust_embedding_layer(model, new_vocab_size, adjusted_model_path):
    old_vocab_size = model.gpt.text_embedding.num_embeddings
    embedding_dim = model.gpt.text_embedding.embedding_dim
    # Create new embedding and linear layers with the new vocabulary size
    new_text_embedding = nn.Embedding(new_vocab_size, embedding_dim)
    new_text_head = nn.Linear(embedding_dim, new_vocab_size)
    # Copy weights from the old embedding layer to the new one
    if new_vocab_size > old_vocab_size:
        new_text_embedding.weight.data[:old_vocab_size] = model.gpt.text_embedding.weight.data
        new_text_head.weight.data[:old_vocab_size] = model.gpt.text_head.weight.data
        new_text_head.bias.data[:old_vocab_size] = model.gpt.text_head.bias.data
        new_text_embedding.weight.data[old_vocab_size:].normal_(mean=0.0, std=0.02)
        new_text_head.weight.data[old_vocab_size:].normal_(mean=0.0, std=0.02)
        new_text_head.bias.data[old_vocab_size:].normal_(mean=0.0, std=0.02)
    else:
        new_text_embedding.weight.data = model.gpt.text_embedding.weight.data[:new_vocab_size]
        new_text_head.weight.data = model.gpt.text_head.weight.data[:new_vocab_size]
        new_text_head.bias.data = model.gpt.text_head.bias.data[:new_vocab_size]
    model.gpt.text_embedding = new_text_embedding
    model.gpt.text_head = new_text_head
    checkpoint = {"model": model.state_dict()}
    torch.save(checkpoint, adjusted_model_path)
    print(f"Adjusted model saved to {adjusted_model_path}")
 # Function to freeze all parameters except the position embeddings
 def freeze_except_position_embeddings(model):
    for param in model.parameters():
        param.requires_grad = False
    for name, param in model.named_parameters():
        if 'pos_embedding' in name:
            param.requires_grad = True
    # Verify which parameters are frozen and which are not, comment this out if you dont want to debug. You should see only two true values
    for name, param in model.named_parameters():
        print(f"{name}: requires_grad={param.requires_grad}")
 # Expand the pretrained model with the new tokenizer
 adjust_pretrained_model(pretrained_model_path, expanded_model_path, new_tokenizer_path)
@@ -0,0 +1,26 @@
 # Simple script to remove LJ Speech formatting for the tokenizer
 # Combine metadata_train and metadata_eval.csv into a single file then run
 import csv
 # Input and output file names
 input_file = '/alltalkbeta/metadata_eval.csv'  # combine metadata_train and metadata_eval.csv
 output_file = '/alltalkbeta/dataset.txt' # this goes to the tokenizer
 # Read the input CSV and write to the output file
 with open(input_file, 'r', newline='', encoding='utf-8') as infile, \
     open(output_file, 'w', newline='', encoding='utf-8') as outfile:
    # Create CSV reader and writer objects
    reader = csv.reader(infile, delimiter='|')
    writer = csv.writer(outfile, delimiter='|')
    # Skip the header
    next(reader, None)
    # Process each row
    for row in reader:
        if len(row) >= 2:
            # Write only the second column (index 1) to the output file
            writer.writerow([row[1]])
 print(f"Processing complete. Output written to {output_file}")
@@ -0,0 +1,869 @@
 import os
 import re
 import textwrap
 from functools import cached_property
 import pypinyin
 import torch
 from hangul_romanize import Transliter
 from hangul_romanize.rule import academic
 from num2words import num2words
 from spacy.lang.ar import Arabic
 from spacy.lang.en import English
 from spacy.lang.es import Spanish
 from spacy.lang.ja import Japanese
 from spacy.lang.zh import Chinese
 from tokenizers import Tokenizer
 from TTS.tts.layers.xtts.zh_num2words import TextNorm as zh_num2words
 def get_spacy_lang(lang):
    if lang == "zh":
        return Chinese()
    elif lang == "ja":
        return Japanese()
    elif lang == "ar":
        return Arabic()
    elif lang == "es":
        return Spanish()
    else:
        # For most languages, Enlish does the job
        return English()
 def split_sentence(text, lang, text_split_length=250):
    """Preprocess the input text"""
    text_splits = []
    if text_split_length is not None and len(text) >= text_split_length:
        text_splits.append("")
        nlp = get_spacy_lang(lang)
        nlp.add_pipe("sentencizer")
        doc = nlp(text)
        for sentence in doc.sents:
            if len(text_splits[-1]) + len(str(sentence)) <= text_split_length:
                # if the last sentence + the current sentence is less than the text_split_length
                # then add the current sentence to the last sentence
                text_splits[-1] += " " + str(sentence)
                text_splits[-1] = text_splits[-1].lstrip()
            elif len(str(sentence)) > text_split_length:
                # if the current sentence is greater than the text_split_length
                for line in textwrap.wrap(
                    str(sentence),
                    width=text_split_length,
                    drop_whitespace=True,
                    break_on_hyphens=False,
                    tabsize=1,
                ):
                    text_splits.append(str(line))
            else:
                text_splits.append(str(sentence))
        if len(text_splits) > 1:
            if text_splits[0] == "":
                del text_splits[0]
    else:
        text_splits = [text.lstrip()]
    return text_splits
 _whitespace_re = re.compile(r"\s+")
 # List of (regular expression, replacement) pairs for abbreviations:
 _abbreviations = {
    "en": [
        (re.compile("\\b%s\\." % x[0], re.IGNORECASE), x[1])
        for x in [
            ("mrs", "misess"),
            ("mr", "mister"),
            ("dr", "doctor"),
            ("st", "saint"),
            ("co", "company"),
            ("jr", "junior"),
            ("maj", "major"),
            ("gen", "general"),
            ("drs", "doctors"),
            ("rev", "reverend"),
            ("lt", "lieutenant"),
            ("hon", "honorable"),
            ("sgt", "sergeant"),
            ("capt", "captain"),
            ("esq", "esquire"),
            ("ltd", "limited"),
            ("col", "colonel"),
            ("ft", "fort"),
        ]
    ],
    "es": [
        (re.compile("\\b%s\\." % x[0], re.IGNORECASE), x[1])
        for x in [
            ("sra", "señora"),
            ("sr", "señor"),
            ("dr", "doctor"),
            ("dra", "doctora"),
            ("st", "santo"),
            ("co", "compañía"),
            ("jr", "junior"),
            ("ltd", "limitada"),
        ]
    ],
    "fr": [
        (re.compile("\\b%s\\." % x[0], re.IGNORECASE), x[1])
        for x in [
            ("mme", "madame"),
            ("mr", "monsieur"),
            ("dr", "docteur"),
            ("st", "saint"),
            ("co", "compagnie"),
            ("jr", "junior"),
            ("ltd", "limitée"),
        ]
    ],
    "de": [
        (re.compile("\\b%s\\." % x[0], re.IGNORECASE), x[1])
        for x in [
            ("fr", "frau"),
            ("dr", "doktor"),
            ("st", "sankt"),
            ("co", "firma"),
            ("jr", "junior"),
        ]
    ],
    "pt": [
        (re.compile("\\b%s\\." % x[0], re.IGNORECASE), x[1])
        for x in [
            ("sra", "senhora"),
            ("sr", "senhor"),
            ("dr", "doutor"),
            ("dra", "doutora"),
            ("st", "santo"),
            ("co", "companhia"),
            ("jr", "júnior"),
            ("ltd", "limitada"),
        ]
    ],
    "it": [
        (re.compile("\\b%s\\." % x[0], re.IGNORECASE), x[1])
        for x in [
            # ("sig.ra", "signora"),
            ("sig", "signore"),
            ("dr", "dottore"),
            ("st", "santo"),
            ("co", "compagnia"),
            ("jr", "junior"),
            ("ltd", "limitata"),
        ]
    ],
    "pl": [
        (re.compile("\\b%s\\." % x[0], re.IGNORECASE), x[1])
        for x in [
            ("p", "pani"),
            ("m", "pan"),
            ("dr", "doktor"),
            ("sw", "święty"),
            ("jr", "junior"),
        ]
    ],
    "ar": [
        (re.compile("\\b%s\\." % x[0], re.IGNORECASE), x[1])
        for x in [
            # There are not many common abbreviations in Arabic as in English.
        ]
    ],
    "zh": [
        (re.compile("\\b%s\\." % x[0], re.IGNORECASE), x[1])
        for x in [
            # Chinese doesn't typically use abbreviations in the same way as Latin-based scripts.
        ]
    ],
    "cs": [
        (re.compile("\\b%s\\." % x[0], re.IGNORECASE), x[1])
        for x in [
            ("dr", "doktor"),  # doctor
            ("ing", "inženýr"),  # engineer
            ("p", "pan"),  # Could also map to pani for woman but no easy way to do it
            # Other abbreviations would be specialized and not as common.
        ]
    ],
    "ru": [
        (re.compile("\\b%s\\b" % x[0], re.IGNORECASE), x[1])
        for x in [
            ("г-жа", "госпожа"),  # Mrs.
            ("г-н", "господин"),  # Mr.
            ("д-р", "доктор"),  # doctor
            # Other abbreviations are less common or specialized.
        ]
    ],
    "nl": [
        (re.compile("\\b%s\\." % x[0], re.IGNORECASE), x[1])
        for x in [
            ("dhr", "de heer"),  # Mr.
            ("mevr", "mevrouw"),  # Mrs.
            ("dr", "dokter"),  # doctor
            ("jhr", "jonkheer"),  # young lord or nobleman
            # Dutch uses more abbreviations, but these are the most common ones.
        ]
    ],
    "tr": [
        (re.compile("\\b%s\\." % x[0], re.IGNORECASE), x[1])
        for x in [
            ("b", "bay"),  # Mr.
            ("byk", "büyük"),  # büyük
            ("dr", "doktor"),  # doctor
            # Add other Turkish abbreviations here if needed.
        ]
    ],
    "hu": [
        (re.compile("\\b%s\\." % x[0], re.IGNORECASE), x[1])
        for x in [
            ("dr", "doktor"),  # doctor
            ("b", "bácsi"),  # Mr.
            ("nőv", "nővér"),  # nurse
            # Add other Hungarian abbreviations here if needed.
        ]
    ],
    "ko": [
        (re.compile("\\b%s\\." % x[0], re.IGNORECASE), x[1])
        for x in [
            # Korean doesn't typically use abbreviations in the same way as Latin-based scripts.
        ]
    ],
    "ja": [
        (re.compile("\\b%s\\b" % x[0]), x[1])
        for x in [
            ("氏", "さん"),  # Mr.
            ("夫人", "おんなのひと"),  # Mrs.
            ("博士", "はかせ"),  # Doctor or PhD
            ("株", "株式会社"),  # Corporation
            ("有", "有限会社"),  # Limited company
            ("大学", "だいがく"),   # University
            ("先生", "せんせい"),   # Teacher/Professor/Master
            ("君", "くん")   # Used at the end of boys' names to express familiarity or affection.
        ]
    ],
 }
 def expand_abbreviations_multilingual(text, lang="en"):
    for regex, replacement in _abbreviations[lang]:
        text = re.sub(regex, replacement, text)
    return text
 _symbols_multilingual = {
    "en": [
        (re.compile(r"%s" % re.escape(x[0]), re.IGNORECASE), x[1])
        for x in [
            ("&", " and "),
            ("@", " at "),
            ("%", " percent "),
            ("#", " hash "),
            ("$", " dollar "),
            ("£", " pound "),
            ("°", " degree "),
        ]
    ],
    "es": [
        (re.compile(r"%s" % re.escape(x[0]), re.IGNORECASE), x[1])
        for x in [
            ("&", " y "),
            ("@", " arroba "),
            ("%", " por ciento "),
            ("#", " numeral "),
            ("$", " dolar "),
            ("£", " libra "),
            ("°", " grados "),
        ]
    ],
    "fr": [
        (re.compile(r"%s" % re.escape(x[0]), re.IGNORECASE), x[1])
        for x in [
            ("&", " et "),
            ("@", " arobase "),
            ("%", " pour cent "),
            ("#", " dièse "),
            ("$", " dollar "),
            ("£", " livre "),
            ("°", " degrés "),
        ]
    ],
    "de": [
        (re.compile(r"%s" % re.escape(x[0]), re.IGNORECASE), x[1])
        for x in [
            ("&", " und "),
            ("@", " at "),
            ("%", " prozent "),
            ("#", " raute "),
            ("$", " dollar "),
            ("£", " pfund "),
            ("°", " grad "),
        ]
    ],
    "pt": [
        (re.compile(r"%s" % re.escape(x[0]), re.IGNORECASE), x[1])
        for x in [
            ("&", " e "),
            ("@", " arroba "),
            ("%", " por cento "),
            ("#", " cardinal "),
            ("$", " dólar "),
            ("£", " libra "),
            ("°", " graus "),
        ]
    ],
    "it": [
        (re.compile(r"%s" % re.escape(x[0]), re.IGNORECASE), x[1])
        for x in [
            ("&", " e "),
            ("@", " chiocciola "),
            ("%", " per cento "),
            ("#", " cancelletto "),
            ("$", " dollaro "),
            ("£", " sterlina "),
            ("°", " gradi "),
        ]
    ],
    "pl": [
        (re.compile(r"%s" % re.escape(x[0]), re.IGNORECASE), x[1])
        for x in [
            ("&", " i "),
            ("@", " małpa "),
            ("%", " procent "),
            ("#", " krzyżyk "),
            ("$", " dolar "),
            ("£", " funt "),
            ("°", " stopnie "),
        ]
    ],
    "ar": [
        # Arabic
        (re.compile(r"%s" % re.escape(x[0]), re.IGNORECASE), x[1])
        for x in [
            ("&", " و "),
            ("@", " على "),
            ("%", " في المئة "),
            ("#", " رقم "),
            ("$", " دولار "),
            ("£", " جنيه "),
            ("°", " درجة "),
        ]
    ],
    "zh": [
        # Chinese
        (re.compile(r"%s" % re.escape(x[0]), re.IGNORECASE), x[1])
        for x in [
            ("&", " 和 "),
            ("@", " 在 "),
            ("%", " 百分之 "),
            ("#", " 号 "),
            ("$", " 美元 "),
            ("£", " 英镑 "),
            ("°", " 度 "),
        ]
    ],
    "cs": [
        # Czech
        (re.compile(r"%s" % re.escape(x[0]), re.IGNORECASE), x[1])
        for x in [
            ("&", " a "),
            ("@", " na "),
            ("%", " procento "),
            ("#", " křížek "),
            ("$", " dolar "),
            ("£", " libra "),
            ("°", " stupně "),
        ]
    ],
    "ru": [
        # Russian
        (re.compile(r"%s" % re.escape(x[0]), re.IGNORECASE), x[1])
        for x in [
            ("&", " и "),
            ("@", " собака "),
            ("%", " процентов "),
            ("#", " номер "),
            ("$", " доллар "),
            ("£", " фунт "),
            ("°", " градус "),
        ]
    ],
    "nl": [
        # Dutch
        (re.compile(r"%s" % re.escape(x[0]), re.IGNORECASE), x[1])
        for x in [
            ("&", " en "),
            ("@", " bij "),
            ("%", " procent "),
            ("#", " hekje "),
            ("$", " dollar "),
            ("£", " pond "),
            ("°", " graden "),
        ]
    ],
    "tr": [
        (re.compile(r"%s" % re.escape(x[0]), re.IGNORECASE), x[1])
        for x in [
            ("&", " ve "),
            ("@", " at "),
            ("%", " yüzde "),
            ("#", " diyez "),
            ("$", " dolar "),
            ("£", " sterlin "),
            ("°", " derece "),
        ]
    ],
    "hu": [
        (re.compile(r"%s" % re.escape(x[0]), re.IGNORECASE), x[1])
        for x in [
            ("&", " és "),
            ("@", " kukac "),
            ("%", " százalék "),
            ("#", " kettőskereszt "),
            ("$", " dollár "),
            ("£", " font "),
            ("°", " fok "),
        ]
    ],
    "ko": [
        # Korean
        (re.compile(r"%s" % re.escape(x[0]), re.IGNORECASE), x[1])
        for x in [
            ("&", " 그리고 "),
            ("@", " 에 "),
            ("%", " 퍼센트 "),
            ("#", " 번호 "),
            ("$", " 달러 "),
            ("£", " 파운드 "),
            ("°", " 도 "),
        ]
    ],
    "ja": [
        (re.compile(r"%s" % re.escape(x[0])), x[1])
        for x in [
            ("&", " と "),
            ("@", " アットマーク "),
            ("%", " パーセント "),
            ("#", " ナンバー "),
            ("$", " ドル "),
            ("£", " ポンド "),
            ("°", " 度"),
            ]
    ],
 }
 def expand_symbols_multilingual(text, lang="en"):
    for regex, replacement in _symbols_multilingual[lang]:
        text = re.sub(regex, replacement, text)
        text = text.replace("  ", " ")  # Ensure there are no double spaces
    return text.strip()
 _ordinal_re = {
    "en": re.compile(r"([0-9]+)(st|nd|rd|th)"),
    "es": re.compile(r"([0-9]+)(º|ª|er|o|a|os|as)"),
    "fr": re.compile(r"([0-9]+)(º|ª|er|re|e|ème)"),
    "de": re.compile(r"([0-9]+)(st|nd|rd|th|º|ª|\.(?=\s|$))"),
    "pt": re.compile(r"([0-9]+)(º|ª|o|a|os|as)"),
    "it": re.compile(r"([0-9]+)(º|°|ª|o|a|i|e)"),
    "pl": re.compile(r"([0-9]+)(º|ª|st|nd|rd|th)"),
    "ar": re.compile(r"([0-9]+)(ون|ين|ث|ر|ى)"),
    "cs": re.compile(r"([0-9]+)\.(?=\s|$)"),  # In Czech, a dot is often used after the number to indicate ordinals.
    "ru": re.compile(r"([0-9]+)(-й|-я|-е|-ое|-ье|-го)"),
    "nl": re.compile(r"([0-9]+)(de|ste|e)"),
    "tr": re.compile(r"([0-9]+)(\.|inci|nci|uncu|üncü|\.)"),
    "hu": re.compile(r"([0-9]+)(\.|adik|edik|odik|edik|ödik|ödike|ik)"),
    "ko": re.compile(r"([0-9]+)(번째|번|차|째)"),
    "ja": re.compile(r"([0-9]+)(番|回|つ|目|等|位)")
 }
 _number_re = re.compile(r"[0-9]+")
 _currency_re = {
    "USD": re.compile(r"((\$[0-9\.\,]*[0-9]+)|([0-9\.\,]*[0-9]+\$))"),
    "GBP": re.compile(r"((£[0-9\.\,]*[0-9]+)|([0-9\.\,]*[0-9]+£))"),
    "EUR": re.compile(r"(([0-9\.\,]*[0-9]+€)|((€[0-9\.\,]*[0-9]+)))"),
 }
 _comma_number_re = re.compile(r"\b\d{1,3}(,\d{3})*(\.\d+)?\b")
 _dot_number_re = re.compile(r"\b\d{1,3}(.\d{3})*(\,\d+)?\b")
 _decimal_number_re = re.compile(r"([0-9]+[.,][0-9]+)")
 def _remove_commas(m):
    text = m.group(0)
    if "," in text:
        text = text.replace(",", "")
    return text
 def _remove_dots(m):
    text = m.group(0)
    if "." in text:
        text = text.replace(".", "")
    return text
 def _expand_decimal_point(m, lang="en"):
    amount = m.group(1).replace(",", ".")
    return num2words(float(amount), lang=lang if lang != "cs" else "cz")
 def _expand_currency(m, lang="en", currency="USD"):
    amount = float((re.sub(r"[^\d.]", "", m.group(0).replace(",", "."))))
    full_amount = num2words(amount, to="currency", currency=currency, lang=lang if lang != "cs" else "cz")
    and_equivalents = {
        "en": ", ",
        "es": " con ",
        "fr": " et ",
        "de": " und ",
        "pt": " e ",
        "it": " e ",
        "pl": ", ",
        "cs": ", ",
        "ru": ", ",
        "nl": ", ",
        "ar": ", ",
        "tr": ", ",
        "hu": ", ",
        "ko": ", ",
    }
    if amount.is_integer():
        last_and = full_amount.rfind(and_equivalents[lang])
        if last_and != -1:
            full_amount = full_amount[:last_and]
    return full_amount
 def _expand_ordinal(m, lang="en"):
    return num2words(int(m.group(1)), ordinal=True, lang=lang if lang != "cs" else "cz")
 def _expand_number(m, lang="en"):
    return num2words(int(m.group(0)), lang=lang if lang != "cs" else "cz")
 def expand_numbers_multilingual(text, lang="en"):
    if lang == "zh":
        text = zh_num2words()(text)
    else:
        if lang in ["en", "ru"]:
            text = re.sub(_comma_number_re, _remove_commas, text)
        else:
            text = re.sub(_dot_number_re, _remove_dots, text)
        try:
            text = re.sub(_currency_re["GBP"], lambda m: _expand_currency(m, lang, "GBP"), text)
            text = re.sub(_currency_re["USD"], lambda m: _expand_currency(m, lang, "USD"), text)
            text = re.sub(_currency_re["EUR"], lambda m: _expand_currency(m, lang, "EUR"), text)
        except:
            pass
        if lang != "tr":
            text = re.sub(_decimal_number_re, lambda m: _expand_decimal_point(m, lang), text)
        text = re.sub(_ordinal_re[lang], lambda m: _expand_ordinal(m, lang), text)
        text = re.sub(_number_re, lambda m: _expand_number(m, lang), text)
    return text
 def lowercase(text):
    return text.lower()
 def collapse_whitespace(text):
    return re.sub(_whitespace_re, " ", text)
 def multilingual_cleaners(text, lang):
    text = text.replace('"', "")
    if lang == "tr":
        text = text.replace("İ", "i")
        text = text.replace("Ö", "ö")
        text = text.replace("Ü", "ü")
    text = lowercase(text)
    text = expand_numbers_multilingual(text, lang)
    text = expand_abbreviations_multilingual(text, lang)
    text = expand_symbols_multilingual(text, lang=lang)
    text = collapse_whitespace(text)
    return text
 def basic_cleaners(text):
    """Basic pipeline that lowercases and collapses whitespace without transliteration."""
    text = lowercase(text)
    text = collapse_whitespace(text)
    return text
 def chinese_transliterate(text):
    return "".join(
        [p[0] for p in pypinyin.pinyin(text, style=pypinyin.Style.TONE3, heteronym=False, neutral_tone_with_five=True)]
    )
 def japanese_cleaners(text, katsu):
    text = katsu.romaji(text)
    text = lowercase(text)
    return text
 def korean_transliterate(text):
    r = Transliter(academic)
    return r.translit(text)
 DEFAULT_VOCAB_FILE = os.path.join(os.path.dirname(os.path.realpath(__file__)), "../data/tokenizer.json")
 class VoiceBpeTokenizer:
    def __init__(self, vocab_file=None):
        self.tokenizer = None
        if vocab_file is not None:
            self.tokenizer = Tokenizer.from_file(vocab_file)
        self.char_limits = {
            "en": 250,
            "de": 253,
            "fr": 273,
            "es": 239,
            "it": 213,
            "pt": 203,
            "pl": 224,
            "zh": 82,
            "ar": 166,
            "cs": 186,
            "ru": 182,
            "nl": 251,
            "tr": 226,
            "ja": 71,
            "hu": 224,
            "ko": 95,
        }
    @cached_property
    def katsu(self):
        import cutlet
        return cutlet.Cutlet()
    def check_input_length(self, txt, lang):
        lang = lang.split("-")[0]  # remove the region
        limit = self.char_limits.get(lang, 250)
        if len(txt) > limit:
            print(
                f"[!] Warning: The text length exceeds the character limit of {limit} for language '{lang}', this might cause truncated audio."
            )
    def preprocess_text(self, txt, lang):
        if lang in {"ar", "cs", "de", "en", "es", "fr", "hu", "it", "nl", "pl", "pt", "ru", "tr", "zh", "ko"}:
            txt = multilingual_cleaners(txt, lang)
            if lang == "zh":
                txt = chinese_transliterate(txt)
            if lang == "ko":
                txt = korean_transliterate(txt)
        elif lang == "ja":
            txt = japanese_cleaners(txt, self.katsu)
        elif lang == "hi":
            # @manmay will implement this
            txt = basic_cleaners(txt)
        else:
            raise NotImplementedError(f"Language '{lang}' is not supported.")
        return txt
    def encode(self, txt, lang):
        lang = lang.split("-")[0]  # remove the region
        self.check_input_length(txt, lang)
        txt = self.preprocess_text(txt, lang)
        lang = "zh-cn" if lang == "zh" else lang
        txt = f"[{lang}]{txt}"
        txt = txt.replace(" ", "[SPACE]")
        return self.tokenizer.encode(txt).ids
    def decode(self, seq):
        if isinstance(seq, torch.Tensor):
            seq = seq.cpu().numpy()
        txt = self.tokenizer.decode(seq, skip_special_tokens=False).replace(" ", "")
        txt = txt.replace("[SPACE]", " ")
        txt = txt.replace("[STOP]", "")
        txt = txt.replace("[UNK]", "")
        return txt
    def __len__(self):
        return self.tokenizer.get_vocab_size()
    def get_number_tokens(self):
        return max(self.tokenizer.get_vocab().values()) + 1
 def test_expand_numbers_multilingual():
    test_cases = [
        # English
        ("In 12.5 seconds.", "In twelve point five seconds.", "en"),
        ("There were 50 soldiers.", "There were fifty soldiers.", "en"),
        ("This is a 1st test", "This is a first test", "en"),
        ("That will be $20 sir.", "That will be twenty dollars sir.", "en"),
        ("That will be 20€ sir.", "That will be twenty euro sir.", "en"),
        ("That will be 20.15€ sir.", "That will be twenty euro, fifteen cents sir.", "en"),
        ("That's 100,000.5.", "That's one hundred thousand point five.", "en"),
        # French
        ("En 12,5 secondes.", "En douze virgule cinq secondes.", "fr"),
        ("Il y avait 50 soldats.", "Il y avait cinquante soldats.", "fr"),
        ("Ceci est un 1er test", "Ceci est un premier test", "fr"),
        ("Cela vous fera $20 monsieur.", "Cela vous fera vingt dollars monsieur.", "fr"),
        ("Cela vous fera 20€ monsieur.", "Cela vous fera vingt euros monsieur.", "fr"),
        ("Cela vous fera 20,15€ monsieur.", "Cela vous fera vingt euros et quinze centimes monsieur.", "fr"),
        ("Ce sera 100.000,5.", "Ce sera cent mille virgule cinq.", "fr"),
        # German
        ("In 12,5 Sekunden.", "In zwölf Komma fünf Sekunden.", "de"),
        ("Es gab 50 Soldaten.", "Es gab fünfzig Soldaten.", "de"),
        ("Dies ist ein 1. Test", "Dies ist ein erste Test", "de"),  # Issue with gender
        ("Das macht $20 Herr.", "Das macht zwanzig Dollar Herr.", "de"),
        ("Das macht 20€ Herr.", "Das macht zwanzig Euro Herr.", "de"),
        ("Das macht 20,15€ Herr.", "Das macht zwanzig Euro und fünfzehn Cent Herr.", "de"),
        # Spanish
        ("En 12,5 segundos.", "En doce punto cinco segundos.", "es"),
        ("Había 50 soldados.", "Había cincuenta soldados.", "es"),
        ("Este es un 1er test", "Este es un primero test", "es"),
        ("Eso le costará $20 señor.", "Eso le costará veinte dólares señor.", "es"),
        ("Eso le costará 20€ señor.", "Eso le costará veinte euros señor.", "es"),
        ("Eso le costará 20,15€ señor.", "Eso le costará veinte euros con quince céntimos señor.", "es"),
        # Italian
        ("In 12,5 secondi.", "In dodici virgola cinque secondi.", "it"),
        ("C'erano 50 soldati.", "C'erano cinquanta soldati.", "it"),
        ("Questo è un 1° test", "Questo è un primo test", "it"),
        ("Ti costerà $20 signore.", "Ti costerà venti dollari signore.", "it"),
        ("Ti costerà 20€ signore.", "Ti costerà venti euro signore.", "it"),
        ("Ti costerà 20,15€ signore.", "Ti costerà venti euro e quindici centesimi signore.", "it"),
        # Portuguese
        ("Em 12,5 segundos.", "Em doze vírgula cinco segundos.", "pt"),
        ("Havia 50 soldados.", "Havia cinquenta soldados.", "pt"),
        ("Este é um 1º teste", "Este é um primeiro teste", "pt"),
        ("Isso custará $20 senhor.", "Isso custará vinte dólares senhor.", "pt"),
        ("Isso custará 20€ senhor.", "Isso custará vinte euros senhor.", "pt"),
        (
            "Isso custará 20,15€ senhor.",
            "Isso custará vinte euros e quinze cêntimos senhor.",
            "pt",
        ),  # "cêntimos" should be "centavos" num2words issue
        # Polish
        ("W 12,5 sekundy.", "W dwanaście przecinek pięć sekundy.", "pl"),
        ("Było 50 żołnierzy.", "Było pięćdziesiąt żołnierzy.", "pl"),
        ("To będzie kosztować 20€ panie.", "To będzie kosztować dwadzieścia euro panie.", "pl"),
        ("To będzie kosztować 20,15€ panie.", "To będzie kosztować dwadzieścia euro, piętnaście centów panie.", "pl"),
        # Arabic
        ("في الـ 12,5 ثانية.", "في الـ اثنا عشر  , خمسون ثانية.", "ar"),
        ("كان هناك 50 جنديًا.", "كان هناك خمسون جنديًا.", "ar"),
        # ("ستكون النتيجة $20 يا سيد.", 'ستكون النتيجة عشرون دولار يا سيد.', 'ar'), # $ and € are mising from num2words
        # ("ستكون النتيجة 20€ يا سيد.", 'ستكون النتيجة عشرون يورو يا سيد.', 'ar'),
        # Czech
        ("Za 12,5 vteřiny.", "Za dvanáct celá pět vteřiny.", "cs"),
        ("Bylo tam 50 vojáků.", "Bylo tam padesát vojáků.", "cs"),
        ("To bude stát 20€ pane.", "To bude stát dvacet euro pane.", "cs"),
        ("To bude 20.15€ pane.", "To bude dvacet euro, patnáct centů pane.", "cs"),
        # Russian
        ("Через 12.5 секунды.", "Через двенадцать запятая пять секунды.", "ru"),
        ("Там было 50 солдат.", "Там было пятьдесят солдат.", "ru"),
        ("Это будет 20.15€ сэр.", "Это будет двадцать евро, пятнадцать центов сэр.", "ru"),
        ("Это будет стоить 20€ господин.", "Это будет стоить двадцать евро господин.", "ru"),
        # Dutch
        ("In 12,5 seconden.", "In twaalf komma vijf seconden.", "nl"),
        ("Er waren 50 soldaten.", "Er waren vijftig soldaten.", "nl"),
        ("Dat wordt dan $20 meneer.", "Dat wordt dan twintig dollar meneer.", "nl"),
        ("Dat wordt dan 20€ meneer.", "Dat wordt dan twintig euro meneer.", "nl"),
        # Chinese (Simplified)
        ("在12.5秒内", "在十二点五秒内", "zh"),
        ("有50名士兵", "有五十名士兵", "zh"),
        # ("那将是$20先生", '那将是二十美元先生', 'zh'), currency doesn't work
        # ("那将是20€先生", '那将是二十欧元先生', 'zh'),
        # Turkish
        # ("12,5 saniye içinde.", 'On iki virgül beş saniye içinde.', 'tr'), # decimal doesn't work for TR
        ("50 asker vardı.", "elli asker vardı.", "tr"),
        ("Bu 1. test", "Bu birinci test", "tr"),
        # ("Bu 100.000,5.", 'Bu yüz bin virgül beş.', 'tr'),
        # Hungarian
        ("12,5 másodperc alatt.", "tizenkettő egész öt tized másodperc alatt.", "hu"),
        ("50 katona volt.", "ötven katona volt.", "hu"),
        ("Ez az 1. teszt", "Ez az első teszt", "hu"),
        # Korean
        ("12.5 초 안에.", "십이 점 다섯 초 안에.", "ko"),
        ("50 명의 병사가 있었다.", "오십 명의 병사가 있었다.", "ko"),
        ("이것은 1 번째 테스트입니다", "이것은 첫 번째 테스트입니다", "ko"),
    ]
    for a, b, lang in test_cases:
        out = expand_numbers_multilingual(a, lang=lang)
        assert out == b, f"'{out}' vs '{b}'"
 def test_abbreviations_multilingual():
    test_cases = [
        # English
        ("Hello Mr. Smith.", "Hello mister Smith.", "en"),
        ("Dr. Jones is here.", "doctor Jones is here.", "en"),
        # Spanish
        ("Hola Sr. Garcia.", "Hola señor Garcia.", "es"),
        ("La Dra. Martinez es muy buena.", "La doctora Martinez es muy buena.", "es"),
        # French
        ("Bonjour Mr. Dupond.", "Bonjour monsieur Dupond.", "fr"),
        ("Mme. Moreau est absente aujourd'hui.", "madame Moreau est absente aujourd'hui.", "fr"),
        # German
        ("Frau Dr. Müller ist sehr klug.", "Frau doktor Müller ist sehr klug.", "de"),
        # Portuguese
        ("Olá Sr. Silva.", "Olá senhor Silva.", "pt"),
        ("Dra. Costa, você está disponível?", "doutora Costa, você está disponível?", "pt"),
        # Italian
        ("Buongiorno, Sig. Rossi.", "Buongiorno, signore Rossi.", "it"),
        # ("Sig.ra Bianchi, posso aiutarti?", 'signora Bianchi, posso aiutarti?', 'it'), # Issue with matching that pattern
        # Polish
        ("Dzień dobry, P. Kowalski.", "Dzień dobry, pani Kowalski.", "pl"),
        ("M. Nowak, czy mogę zadać pytanie?", "pan Nowak, czy mogę zadać pytanie?", "pl"),
        # Czech
        ("P. Novák", "pan Novák", "cs"),
        ("Dr. Vojtěch", "doktor Vojtěch", "cs"),
        # Dutch
        ("Dhr. Jansen", "de heer Jansen", "nl"),
        ("Mevr. de Vries", "mevrouw de Vries", "nl"),
        # Russian
        ("Здравствуйте Г-н Иванов.", "Здравствуйте господин Иванов.", "ru"),
        ("Д-р Смирнов здесь, чтобы увидеть вас.", "доктор Смирнов здесь, чтобы увидеть вас.", "ru"),
        # Turkish
        ("Merhaba B. Yılmaz.", "Merhaba bay Yılmaz.", "tr"),
        ("Dr. Ayşe burada.", "doktor Ayşe burada.", "tr"),
        # Hungarian
        ("Dr. Szabó itt van.", "doktor Szabó itt van.", "hu"),
    ]
    for a, b, lang in test_cases:
        out = expand_abbreviations_multilingual(a, lang=lang)
        assert out == b, f"'{out}' vs '{b}'"
 def test_symbols_multilingual():
    test_cases = [
        ("I have 14% battery", "I have 14 percent battery", "en"),
        ("Te veo @ la fiesta", "Te veo arroba la fiesta", "es"),
        ("J'ai 14° de fièvre", "J'ai 14 degrés de fièvre", "fr"),
        ("Die Rechnung beträgt £ 20", "Die Rechnung beträgt pfund 20", "de"),
        ("O meu email é ana&joao@gmail.com", "O meu email é ana e joao arroba gmail.com", "pt"),
        ("linguaggio di programmazione C#", "linguaggio di programmazione C cancelletto", "it"),
        ("Moja temperatura to 36.6°", "Moja temperatura to 36.6 stopnie", "pl"),
        ("Mám 14% baterie", "Mám 14 procento baterie", "cs"),
        ("Těším se na tebe @ party", "Těším se na tebe na party", "cs"),
        ("У меня 14% заряда", "У меня 14 процентов заряда", "ru"),
        ("Я буду @ дома", "Я буду собака дома", "ru"),
        ("Ik heb 14% batterij", "Ik heb 14 procent batterij", "nl"),
        ("Ik zie je @ het feest", "Ik zie je bij het feest", "nl"),
        ("لدي 14% في البطارية", "لدي 14 في المئة في البطارية", "ar"),
        ("我的电量为 14%", "我的电量为 14 百分之", "zh"),
        ("Pilim %14 dolu.", "Pilim yüzde 14 dolu.", "tr"),
        ("Az akkumulátorom töltöttsége 14%", "Az akkumulátorom töltöttsége 14 százalék", "hu"),
        ("배터리 잔량이 14%입니다.", "배터리 잔량이 14 퍼센트입니다.", "ko"),
    ]
    for a, b, lang in test_cases:
        out = expand_symbols_multilingual(a, lang=lang)
        assert out == b, f"'{out}' vs '{b}'"
 if __name__ == "__main__":
    test_expand_numbers_multilingual()
    test_abbreviations_multilingual()
    test_symbols_multilingual()
@@ -0,0 +1,76 @@
 import os
 from pathlib import Path
 import gradio as gr
 # Define directories
 this_dir = Path(__file__).parent.resolve()
 main_dir = this_dir.parent.parent.resolve()
 # Folder paths
 folders = {
    "models": main_dir / "models",
    "finetune": main_dir / "finetune",
    "voices": main_dir / "voices",
    "outputs": main_dir / "outputs",
    "alltalk_environment": main_dir / "alltalk_environment"
 }
 def get_folder_size_scandir(folder_path):
    total_size = 0
    for entry in os.scandir(folder_path):
        if entry.is_file(follow_symlinks=False):
            try:
                total_size += entry.stat(follow_symlinks=False).st_size
            except OSError as e:
                print(f"Error getting size for file {entry.path}: {e}")
        elif entry.is_dir(follow_symlinks=False):
            try:
                total_size += get_folder_size_scandir(entry.path)
            except OSError as e:
                print(f"Error accessing directory {entry.path}: {e}")
    return total_size
 def format_size(size):
    # Convert size to a readable format
    for unit in ['B', 'KB', 'MB', 'GB', 'TB']:
        if size < 1024:
            return f"{size:.2f} {unit}"
        size /= 1024
 def calculate_disk_space():
    sizes = {}
    for name, path in folders.items():
        if path.exists():
            folder_size = get_folder_size_scandir(path)
            # print(f"Folder: {name}, Total Size: {folder_size} bytes")
            sizes[name] = format_size(folder_size)
        else:
            sizes[name] = "Not present"
    return sizes['models'], sizes['finetune'], sizes['voices'], sizes['outputs'], sizes['alltalk_environment']
 def disk_space_page():
    # Define the layout
    with gr.Tab("Disk Space Usage"):
        with gr.Row():
            models_space = gr.Textbox(label="Models Folder", interactive=False)
            finetune_space = gr.Textbox(label="Finetune Folder", interactive=False)
            voices_space = gr.Textbox(label="Voices Folder", interactive=False)
            outputs_space = gr.Textbox(label="Outputs Folder", interactive=False)
            alltalk_env_space = gr.Textbox(label="Alltalk Python Environment", interactive=False)
        calculate_button = gr.Button("Calculate Disk Space Usage")
        # Define actions for buttons
        calculate_button.click(calculate_disk_space, outputs=[models_space, finetune_space, voices_space, outputs_space, alltalk_env_space])
 def get_disk_interface():
    return disk_space_page
 # Example usage in the main script
 if __name__ == "__main__":
    app = gr.Blocks()
    with app:
        disk_space_page()
    app.launch()
@@ -0,0 +1,114 @@
 import os
 import json
 import sys
 import importlib
 # Hard-coded paths
 this_folder = os.path.abspath(os.path.join(os.path.dirname(__file__), "..", "..", ".."))
 config_file = os.path.join(this_folder, "confignew.json")
 gradio_pages_folder = os.path.join(this_folder, "system", "gradio_pages", "themes")
 theme_list_file = os.path.join(gradio_pages_folder, "theme_list.json")
 themes_file_path = os.path.join(gradio_pages_folder, "themes.py")
 # Add gradio_pages folder to the system path
 sys.path.append(gradio_pages_folder)
 def get_class(filename):
    with open(filename, "r", encoding="utf8") as file:
        for line in file:
            if "class " in line:
                return line.split("class ")[1].split(":")[0].split("(")[0].strip()
    return None
 def get_list():
    themes_from_files = [
        os.path.splitext(name)[0]
        for name in os.listdir(gradio_pages_folder)
        if name.endswith(".py")
    ]
    try:
        with open(theme_list_file, "r", encoding="utf8") as json_file:
            themes_from_url = [item["id"] for item in json.load(json_file)]
    except FileNotFoundError:
        themes_from_url = []
    combined_themes = sorted(set(themes_from_files + themes_from_url))
    return list(combined_themes)
 def select_theme(name):
    selected_file = name + ".py"
    full_path = os.path.join(gradio_pages_folder, selected_file)
    try:
        with open(config_file, "r", encoding="utf8") as json_file:
            config_data = json.load(json_file)
        if not os.path.exists(full_path):
            # If the theme is from theme_list.json, update only the class field
            config_data["theme"]["file"] = None
            config_data["theme"]["class"] = name
            message = f"Theme {name} successfully selected, restart AllTalk."
        else:
            # If the theme file exists locally, update both file and class fields
            class_found = get_class(full_path)
            if class_found:
                config_data["theme"]["file"] = selected_file
                config_data["theme"]["class"] = class_found
                message = f"Theme {name} successfully selected, restart AllTalk."
            else:
                message = f"Theme {name} was not found."
        with open(config_file, "w", encoding="utf8") as json_file:
            json.dump(config_data, json_file, indent=2)
        print(message)
        return message
    except Exception as e:
        print(f"[AllTalk TTS] Error selecting theme {name}: {str(e)}")
        return message
 def read_json():
    try:
        with open(config_file, "r", encoding="utf8") as json_file:
            data = json.load(json_file)
            selected_file = data["theme"]["file"]
            class_name = data["theme"]["class"]
            if selected_file and class_name:
                return class_name
            elif not selected_file and class_name:
                return class_name
            else:
                return "gradio/base"
    except Exception as e:
        print(f"[AllTalk TTS] Error reading config.json: {e}")
        return "gradio/base"
 def load_json():
    try:
        with open(config_file, "r", encoding="utf8") as json_file:
            data = json.load(json_file)
            selected_file = data["theme"]["file"]
            class_name = data["theme"]["class"]
            if selected_file and class_name:
                module_name = os.path.splitext(selected_file)[0]
                module_spec = importlib.util.spec_from_file_location(module_name, themes_file_path)
                module = importlib.util.module_from_spec(module_spec)
                module_spec.loader.exec_module(module)
                obtained_class = getattr(module, class_name)
                instance = obtained_class()
                print(f"Theme Loaded: {class_name}")
                return instance
            elif not selected_file and class_name:
                return class_name
            else:
                print("The theme is incorrect.")
                return None
    except Exception as e:
        print(f"Error Loading: {str(e)}")
        return None
@@ -0,0 +1,81 @@
 [
  {"id": "freddyaboulton/dracula_revamped"},
  {"id": "freddyaboulton/bad-theme-space"},
  {"id": "gradio/dracula_revamped"},
  {"id": "abidlabs/dracula_revamped"},
  {"id": "gradio/dracula_test"},
  {"id": "abidlabs/dracula_test"},
  {"id": "gradio/seafoam"},
  {"id": "gradio/glass"},
  {"id": "gradio/monochrome"},
  {"id": "gradio/soft"},
  {"id": "gradio/default"},
  {"id": "gradio/base"},
  {"id": "abidlabs/pakistan"},
  {"id": "dawood/microsoft_windows"},
  {"id": "ysharma/steampunk"},
  {"id": "ysharma/huggingface"},
  {"id": "gstaff/xkcd"},
  {"id": "JohnSmith9982/small_and_pretty"},
  {"id": "abidlabs/Lime"},
  {"id": "freddyaboulton/this-theme-does-not-exist-2"},
  {"id": "aliabid94/new-theme"},
  {"id": "aliabid94/test2"},
  {"id": "aliabid94/test3"},
  {"id": "aliabid94/test4"},
  {"id": "abidlabs/banana"},
  {"id": "freddyaboulton/test-blue"},
  {"id": "gstaff/sketch"},
  {"id": "gstaff/whiteboard"},
  {"id": "ysharma/llamas"},
  {"id": "abidlabs/font-test"},
  {"id": "YenLai/Superhuman"},
  {"id": "bethecloud/storj_theme"},
  {"id": "sudeepshouche/minimalist"},
  {"id": "knotdgaf/gradiotest"},
  {"id": "ParityError/Interstellar"},
  {"id": "ParityError/Anime"},
  {"id": "Ajaxon6255/Emerald_Isle"},
  {"id": "ParityError/LimeFace"},
  {"id": "finlaymacklon/smooth_slate"},
  {"id": "finlaymacklon/boxy_violet"},
  {"id": "derekzen/stardust"},
  {"id": "EveryPizza/Cartoony-Gradio-Theme"},
  {"id": "Ifeanyi/Cyanister"},
  {"id": "Tshackelton/IBMPlex-DenseReadable"},
  {"id": "snehilsanyal/scikit-learn"},
  {"id": "Himhimhim/xkcd"},
  {"id": "shivi/calm_seafoam"},
  {"id": "nota-ai/theme"},
  {"id": "rawrsor1/Everforest"},
  {"id": "SebastianBravo/simci_css"},
  {"id": "rottenlittlecreature/Moon_Goblin"},
  {"id": "abidlabs/test-yellow"},
  {"id": "abidlabs/test-yellow3"},
  {"id": "idspicQstitho/dracula_revamped"},
  {"id": "kfahn/AnimalPose"},
  {"id": "HaleyCH/HaleyCH_Theme"},
  {"id": "simulKitke/dracula_test"},
  {"id": "braintacles/CrimsonNight"},
  {"id": "wentaohe/whiteboardv2"},
  {"id": "reilnuud/polite"},
  {"id": "remilia/Ghostly"},
  {"id": "Franklisi/darkmode"},
  {"id": "coding-alt/soft"},
  {"id": "xiaobaiyuan/theme_land"},
  {"id": "step-3-profit/Midnight-Deep"},
  {"id": "xiaobaiyuan/theme_demo"},
  {"id": "Taithrah/Minimal"},
  {"id": "Insuz/SimpleIndigo"},
  {"id": "zkunn/Alipay_Gradio_theme"},
  {"id": "Insuz/Mocha"},
  {"id": "xiaobaiyuan/theme_brief"},
  {"id": "Ama434/434-base-Barlow"},
  {"id": "Ama434/def_barlow"},
  {"id": "Ama434/neutral-barlow"},
  {"id": "dawood/dracula_test"},
  {"id": "nuttea/Softblue"},
  {"id": "BlueDancer/Alien_Diffusion"},
  {"id": "naughtondale/monochrome"},
  {"id": "Dagfinn1962/standard"}
 ]
@@ -0,0 +1,284 @@
 from __future__ import annotations
 from typing import Iterable
 import gradio as gr
 # gr.themes.builder()
 from gradio.themes.base import Base
 from gradio.themes.utils import colors, fonts, sizes
 import time
 class alltalk(Base):
    def __init__(
        self,
        *,
        primary_hue: colors.Color | str = colors.neutral,
        secondary_hue: colors.Color | str = colors.neutral,
        neutral_hue: colors.Color | str = colors.neutral,
        spacing_size: sizes.Size | str = sizes.spacing_md,
        radius_size: sizes.Size | str = sizes.radius_md,
        text_size: sizes.Size | str = sizes.text_lg,
        font: fonts.Font | str | Iterable[fonts.Font | str] = (
            "Syne V",
            fonts.GoogleFont("Syne"),
            "ui-sans-serif",
            "system-ui",
        ),
        font_mono: fonts.Font | str | Iterable[fonts.Font | str] = (
            "ui-monospace",
            fonts.GoogleFont("Nunito Sans"),
        ),
    ):
        super().__init__(
            primary_hue=primary_hue,
            secondary_hue=secondary_hue,
            neutral_hue=neutral_hue,
            spacing_size=spacing_size,
            radius_size=radius_size,
            text_size=text_size,
            font=font,
            font_mono=font_mono,
        )
        self.name = ("alltalk",)
        self.secondary_100 = ("#dbeafe",)
        self.secondary_200 = ("#bfdbfe",)
        self.secondary_300 = ("#93c5fd",)
        self.secondary_400 = ("#60a5fa",)
        self.secondary_50 = ("#eff6ff",)
        self.secondary_500 = ("#3b82f6",)
        self.secondary_600 = ("#2563eb",)
        self.secondary_700 = ("#1d4ed8",)
        self.secondary_800 = ("#1e40af",)
        self.secondary_900 = ("#1e3a8a",)
        self.secondary_950 = ("#1d3660",)
        super().set(
            # Blaise
            background_fill_primary="#110F0F",
            background_fill_primary_dark="#110F0F",
            background_fill_secondary="#110F0F",
            background_fill_secondary_dark="#110F0F",
            block_background_fill="*neutral_800",
            block_background_fill_dark="*neutral_800",
            block_border_color="*border_color_primary",
            block_border_color_dark="*border_color_primary",
            block_border_width="1px",
            block_border_width_dark="1px",
            block_info_text_color="*body_text_color_subdued",
            block_info_text_color_dark="*body_text_color_subdued",
            block_info_text_size="*text_sm",
            block_info_text_weight="400",
            block_label_background_fill="*background_fill_primary",
            block_label_background_fill_dark="*background_fill_secondary",
            block_label_border_color="*border_color_primary",
            block_label_border_color_dark="*border_color_primary",
            block_label_border_width="1px",
            block_label_border_width_dark="1px",
            block_label_margin="0",
            block_label_padding="*spacing_sm *spacing_lg",
            block_label_radius="calc(*radius_lg - 1px) 0 calc(*radius_lg - 1px) 0",
            block_label_right_radius="0 calc(*radius_lg - 1px) 0 calc(*radius_lg - 1px)",
            block_label_shadow="*block_shadow",
            block_label_text_color="*#110F0F",
            block_label_text_color_dark="*#110F0F",
            block_label_text_weight="400",
            block_padding="*spacing_xl",
            block_radius="*radius_md",
            block_shadow="none",
            block_shadow_dark="none",
            block_title_background_fill="rgb(255,255,255)",
            block_title_background_fill_dark="rgb(255,255,255)",
            block_title_border_color="none",
            block_title_border_color_dark="none",
            block_title_border_width="0px",
            block_title_padding="*block_label_padding",
            block_title_radius="*block_label_radius",
            block_title_text_color="#110F0F",
            block_title_text_color_dark="#110F0F",
            block_title_text_size="*text_md",
            block_title_text_weight="600",
            body_background_fill="#110F0F",
            body_background_fill_dark="#110F0F",
            body_text_color="white",
            body_text_color_dark="white",
            body_text_color_subdued="*neutral_400",
            body_text_color_subdued_dark="*neutral_400",
            body_text_size="*text_md",
            body_text_weight="400",
            border_color_accent="*neutral_600",
            border_color_accent_dark="*neutral_600",
            border_color_primary="*neutral_800",
            border_color_primary_dark="*neutral_800",
            button_border_width="*input_border_width",
            button_border_width_dark="*input_border_width",
            button_cancel_background_fill="*button_secondary_background_fill",
            button_cancel_background_fill_dark="*button_secondary_background_fill",
            button_cancel_background_fill_hover="*button_cancel_background_fill",
            button_cancel_background_fill_hover_dark="*button_cancel_background_fill",
            button_cancel_border_color="*button_secondary_border_color",
            button_cancel_border_color_dark="*button_secondary_border_color",
            button_cancel_border_color_hover="*button_cancel_border_color",
            button_cancel_border_color_hover_dark="*button_cancel_border_color",
            button_cancel_text_color="#110F0F",
            button_cancel_text_color_dark="#110F0F",
            button_cancel_text_color_hover="#110F0F",
            button_cancel_text_color_hover_dark="#110F0F",
            button_large_padding="*spacing_lg calc(2 * *spacing_lg)",
            button_large_radius="*radius_lg",
            button_large_text_size="*text_lg",
            button_large_text_weight="600",
            button_primary_background_fill="*primary_600",
            button_primary_background_fill_dark="*primary_600",
            button_primary_background_fill_hover="*primary_500",
            button_primary_background_fill_hover_dark="*primary_500",
            button_primary_border_color="*primary_500",
            button_primary_border_color_dark="*primary_500",
            button_primary_border_color_hover="*primary_400",
            button_primary_border_color_hover_dark="*primary_400",
            button_primary_text_color="white",
            button_primary_text_color_dark="white",
            button_primary_text_color_hover="#110F0F",
            button_primary_text_color_hover_dark="#110F0F",
            button_secondary_background_fill="transparent",
            button_secondary_background_fill_dark="transparent",
            button_secondary_background_fill_hover="*neutral_800",
            button_secondary_background_fill_hover_dark="*neutral_800",
            button_secondary_border_color="*neutral_700",
            button_secondary_border_color_dark="*neutral_700",
            button_secondary_border_color_hover="*neutral_600",
            button_secondary_border_color_hover_dark="*neutral_600",
            button_secondary_text_color="white",
            button_secondary_text_color_dark="white",
            button_secondary_text_color_hover="*button_secondary_text_color",
            button_secondary_text_color_hover_dark="*button_secondary_text_color",
            button_shadow="none",
            button_shadow_active="*shadow_inset",
            button_shadow_hover="none",
            button_small_padding="*spacing_sm calc(2 * *spacing_sm)",
            button_small_radius="*radius_lg",
            button_small_text_size="*text_md",
            button_small_text_weight="400",
            button_transition="0.3s ease all",
            checkbox_background_color="*neutral_700",
            checkbox_background_color_dark="*neutral_700",
            checkbox_background_color_focus="*checkbox_background_color",
            checkbox_background_color_focus_dark="*checkbox_background_color",
            checkbox_background_color_hover="*checkbox_background_color",
            checkbox_background_color_hover_dark="*checkbox_background_color",
            checkbox_background_color_selected="*secondary_600",
            checkbox_background_color_selected_dark="*secondary_600",
            checkbox_border_color="*neutral_700",
            checkbox_border_color_dark="*neutral_700",
            checkbox_border_color_focus="*secondary_500",
            checkbox_border_color_focus_dark="*secondary_500",
            checkbox_border_color_hover="*neutral_600",
            checkbox_border_color_hover_dark="*neutral_600",
            checkbox_border_color_selected="*secondary_600",
            checkbox_border_color_selected_dark="*secondary_600",
            checkbox_border_radius="*radius_sm",
            checkbox_border_width="*input_border_width",
            checkbox_border_width_dark="*input_border_width",
            checkbox_check="url(\"data:image/svg+xml,%3csvg viewBox='0 0 16 16' fill='white' xmlns='http://www.w3.org/2000/svg'%3e%3cpath d='M12.207 4.793a1 1 0 010 1.414l-5 5a1 1 0 01-1.414 0l-2-2a1 1 0 011.414-1.414L6.5 9.086l4.293-4.293a1 1 0 011.414 0z'/%3e%3c/svg%3e\")",
            checkbox_label_background_fill="transparent",
            checkbox_label_background_fill_dark="transparent",
            checkbox_label_background_fill_hover="transparent",
            checkbox_label_background_fill_hover_dark="transparent",
            checkbox_label_background_fill_selected="transparent",
            checkbox_label_background_fill_selected_dark="transparent",
            checkbox_label_border_color="transparent",
            checkbox_label_border_color_dark="transparent",
            checkbox_label_border_color_hover="transparent",
            checkbox_label_border_color_hover_dark="transparent",
            checkbox_label_border_width="transparent",
            checkbox_label_border_width_dark="transparent",
            checkbox_label_gap="*spacing_lg",
            checkbox_label_padding="*spacing_md calc(2 * *spacing_md)",
            checkbox_label_shadow="none",
            checkbox_label_text_color="*body_text_color",
            checkbox_label_text_color_dark="*body_text_color",
            checkbox_label_text_color_selected="*checkbox_label_text_color",
            checkbox_label_text_color_selected_dark="*checkbox_label_text_color",
            checkbox_label_text_size="*text_md",
            checkbox_label_text_weight="400",
            checkbox_shadow="*input_shadow",
            color_accent="*primary_500",
            color_accent_soft="*primary_50",
            color_accent_soft_dark="*neutral_700",
            container_radius="*radius_xl",
            embed_radius="*radius_lg",
            error_background_fill="*background_fill_primary",
            error_background_fill_dark="*background_fill_primary",
            error_border_color="*border_color_primary",
            error_border_color_dark="*border_color_primary",
            error_border_width="1px",
            error_border_width_dark="1px",
            error_text_color="#ef4444",
            error_text_color_dark="#ef4444",
            form_gap_width="0px",
            input_background_fill="*neutral_900",
            input_background_fill_dark="*neutral_900",
            input_background_fill_focus="*secondary_600",
            input_background_fill_focus_dark="*secondary_600",
            input_background_fill_hover="*input_background_fill",
            input_background_fill_hover_dark="*input_background_fill",
            input_border_color="*neutral_700",
            input_border_color_dark="*neutral_700",
            input_border_color_focus="*secondary_600",
            input_border_color_focus_dark="*primary_600",
            input_border_color_hover="*input_border_color",
            input_border_color_hover_dark="*input_border_color",
            input_border_width="1px",
            input_border_width_dark="1px",
            input_padding="*spacing_xl",
            input_placeholder_color="*neutral_500",
            input_placeholder_color_dark="*neutral_500",
            input_radius="*radius_lg",
            input_shadow="none",
            input_shadow_dark="none",
            input_shadow_focus="*input_shadow",
            input_shadow_focus_dark="*input_shadow",
            input_text_size="*text_md",
            input_text_weight="400",
            layout_gap="*spacing_xxl",
            link_text_color="*secondary_500",
            link_text_color_active="*secondary_500",
            link_text_color_active_dark="*secondary_500",
            link_text_color_dark="*secondary_500",
            link_text_color_hover="*secondary_400",
            link_text_color_hover_dark="*secondary_400",
            link_text_color_visited="*secondary_600",
            link_text_color_visited_dark="*secondary_600",
            loader_color="*color_accent",
            loader_color_dark="*color_accent",
            panel_background_fill="*background_fill_secondary",
            panel_background_fill_dark="*background_fill_secondary",
            panel_border_color="*border_color_primary",
            panel_border_color_dark="*border_color_primary",
            panel_border_width="1px",
            panel_border_width_dark="1px",
            prose_header_text_weight="600",
            prose_text_size="*text_md",
            prose_text_weight="400",
            radio_circle="url(\"data:image/svg+xml,%3csvg viewBox='0 0 16 16' fill='white' xmlns='http://www.w3.org/2000/svg'%3e%3ccircle cx='8' cy='8' r='3'/%3e%3c/svg%3e\")",
            section_header_text_size="*text_md",
            section_header_text_weight="400",
            shadow_drop="rgba(0,0,0,0.05) 0px 1px 2px 0px",
            shadow_drop_lg="0 1px 3px 0 rgb(0 0 0 / 0.1), 0 1px 2px -1px rgb(0 0 0 / 0.1)",
            shadow_inset="rgba(0,0,0,0.05) 0px 2px 4px 0px inset",
            shadow_spread="3px",
            shadow_spread_dark="1px",
            slider_color="#9E9E9E",
            slider_color_dark="#9E9E9E",
            stat_background_fill="*primary_500",
            stat_background_fill_dark="*primary_500",
            table_border_color="*neutral_700",
            table_border_color_dark="*neutral_700",
            table_even_background_fill="*neutral_950",
            table_even_background_fill_dark="*neutral_950",
            table_odd_background_fill="*neutral_900",
            table_odd_background_fill_dark="*neutral_900",
            table_radius="*radius_lg",
            table_row_focus="*color_accent_soft",
            table_row_focus_dark="*color_accent_soft",
        )
@@ -0,0 +1,196 @@
 <!DOCTYPE html>
 <html lang="en">
 <head>
    <meta charset="UTF-8">
    <meta name="viewport" content="width=device-width, initial-scale=1.0">
    <title>OpenAI API/AllTalk TTS API Test</title>
    <style>
        body {
            font-family: Arial, sans-serif;
            background-color: #1e1e1e;
            color: #c7c7c7;
            display: flex;
            justify-content: center;
            align-items: center;
            height: 100vh;
            margin: 0;
        }
        h1 {
            color: #ffffff;
            text-align: center;
            margin-bottom: 20px;
        }
        .container {
            background-color: #2b2b2b;
            padding: 20px;
            border-radius: 8px;
            box-shadow: 0 0 10px rgba(0, 0, 0, 0.5);
            max-width: 800px;
            width: 100%;
        }
        label {
            font-weight: bold;
            display: block;
            margin: 10px 0 5px;
            color: #ffffff;
        }
        input[type="text"], input[type="number"], textarea, select {
            width: calc(100% - 20px);
            padding: 10px;
            margin-bottom: 20px;
            border: 1px solid #555;
            border-radius: 4px;
            background-color: #3b3b3b;
            color: #ffffff;
        }
        input[type="submit"] {
            background-color: #4CAF50;
            color: white;
            padding: 10px 20px;
            border: none;
            border-radius: 4px;
            cursor: pointer;
        }
        input[type="submit"]:hover {
            background-color: #45a049;
        }
        .config, .response {
            background-color: #3b3b3b;
            padding: 20px;
            border-radius: 8px;
            box-shadow: 0 0 10px rgba(0, 0, 0, 0.5);
            margin-bottom: 20px;
        }
        .form-group {
            display: flex;
            justify-content: space-between;
        }
        .form-group > div {
            flex: 1;
            margin-right: 10px;
        }
        .form-group > div:last-child {
            margin-right: 0;
        }
    </style>
 </head>
 <body>
    <div class="container">
        <h1>OpenAI API/AllTalk TTS API Test</h1>
        <div class="config form-group">
            <div>
                <label for="ip">IP Address:</label>
                <input type="text" id="ip" name="ip" value="localhost" required>
            </div>
            <div>
                <label for="port">Port Number:</label>
                <input type="number" id="port" name="port" value="7851" required>
            </div>
        </div>
        <form id="ttsForm">
            <label for="model">Model:</label>
            <input type="text" id="model" name="model" value="some-model" required><br>
            <label for="input">Input Text:</label>
            <textarea id="input" name="input" rows="8" required>Hello, this is a test.</textarea><br>
            <p id="charCount">Character count: 21</p>
            <div class="form-group">
                <div>
                    <label for="voice">Voice:</label>
                    <select id="voice" name="voice" required>
                        <option value="alloy">Alloy</option>
                        <option value="echo">Echo</option>
                        <option value="fable">Fable</option>
                        <option value="nova">Nova</option>
                        <option value="onyx">Onyx</option>
                        <option value="shimmer">Shimmer</option>
                    </select>
                </div>
                <div>
                    <label for="response_format">Response Format:</label>
                    <select id="response_format" name="response_format" required>
                        <option value="wav">WAV</option>
                        <option value="mp3">MP3</option>
                        <option value="ogg">OGG</option>
                        <option value="flac">FLAC</option>
                        <option value="aac">AAC</option>
                        <option value="m4a">M4A</option>
                    </select>
                </div>
                <div>
                    <label for="speed">Speed:</label>
                    <input type="number" id="speed" name="speed" value="1.0" step="0.1" required><br>
                </div>
            </div>
            <label for="autoplay">Auto-play:</label>
            <input type="checkbox" id="autoplay" name="autoplay"><br><br>
            <input type="submit" value="Submit"><br><br>
        </form>
        <div id="response" class="response"></div>
    </div>
    <script>
        // Real-time character count
        const inputField = document.getElementById('input');
        const charCountDisplay = document.getElementById('charCount');
        inputField.addEventListener('input', () => {
            const trimmedInput = inputField.value.trim();
            charCountDisplay.textContent = `Character count: ${trimmedInput.length}`;
        });
        document.getElementById('ttsForm').addEventListener('submit', async function (event) {
            event.preventDefault();
            const formData = new FormData(event.target);
            const jsonData = {};
            // Sanitize input: trim spaces
            formData.forEach((value, key) => {
                if (key === 'input') {
                    value = value.trim().replace(/\s+/g, ' '); // Trim and normalize spaces
                }
                jsonData[key] = key === 'speed' ? parseFloat(value) : value;
            });
            const ip = document.getElementById('ip').value;
            const port = document.getElementById('port').value;
            const url = `http://${ip}:${port}/v1/audio/speech`;
            const responseDiv = document.getElementById('response');
            responseDiv.innerHTML = 'Sending request...';
            try {
                const response = await fetch(url, {
                    method: 'POST',
                    headers: {
                        'Content-Type': 'application/json',
                    },
                    body: JSON.stringify(jsonData),
                });
                if (response.ok) {
                    const blob = await response.blob();
                    const url = URL.createObjectURL(blob);
                    const audio = new Audio(url);
                    audio.controls = true;
                    if (document.getElementById('autoplay').checked) {
                        audio.autoplay = true;
                    }
                    responseDiv.innerHTML = '';
                    responseDiv.appendChild(audio);
                } else {
                    const errorText = await response.text();
                    responseDiv.innerHTML = 'Error: ' + errorText;
                }
            } catch (error) {
                responseDiv.innerHTML = 'Error: ' + error.message;
            }
        });
    </script>
 </body>
 </html>
@@ -0,0 +1,51 @@
 from pathlib import Path
 import argparse
 from OpenSSL import crypto
 import sys
 def generate_certificate(cert_name: str, output_dir: Path, cn: str = "localhost", days: int = 365):
    k = crypto.PKey()
    k.generate_key(crypto.TYPE_RSA, 2048)
    cert = crypto.X509()
    cert.get_subject().CN = cn
    cert.set_serial_number(1000)
    cert.gmtime_adj_notBefore(0)
    cert.gmtime_adj_notAfter(days*24*60*60)
    cert.set_issuer(cert.get_subject())
    cert.set_pubkey(k)
    cert.sign(k, 'sha256')
    output_dir.mkdir(parents=True, exist_ok=True)
    with open(output_dir / f"{cert_name}_cert.pem", "wb") as f:
        f.write(crypto.dump_certificate(crypto.FILETYPE_PEM, cert))
    with open(output_dir / f"{cert_name}_key.pem", "wb") as f:
        f.write(crypto.dump_privatekey(crypto.FILETYPE_PEM, k))
 def main():
    print("Certificate Generator")
    print("-" * 20)
    # Get default paths
    default_path = Path(__file__).parent / "certificates"
    # Get user input with defaults
    cert_name = input(f"Certificate name (default: test_cert): ") or "test_cert"
    cn = input(f"Common Name (default: localhost): ") or "localhost"
    days = input(f"Valid days (default: 365): ") or "365"
    output_dir = input(f"Output directory (default: {default_path}): ") or str(default_path)
    try:
        days = int(days)
        output_path = Path(output_dir)
        generate_certificate(cert_name, output_path, cn, days)
        print(f"\nSuccess! Certificate files created:")
        print(f"- {output_path / f'{cert_name}_cert.pem'}")
        print(f"- {output_path / f'{cert_name}_key.pem'}")
    except Exception as e:
        print(f"\nError: {str(e)}")
        sys.exit(1)
 if __name__ == "__main__":
    main()
@@ -0,0 +1,120 @@
 import logging
 import os
 from pathlib import Path
 from typing import Dict, Any, Optional
 from datetime import datetime
 import ssl
 import OpenSSL
 from dataclasses import dataclass
@dataclass
 class CertificateInfo:
    subject: str
    issuer: str
    valid_from: datetime
    valid_until: datetime
    serial_number: str
    is_valid: bool
 class CertificateManager:
    def __init__(self, proxy_manager):
        self.proxy_manager = proxy_manager
        self.logger = logging.getLogger("ProxyCertManager")
        self.certs_path = proxy_manager.certs_path
    def validate_certificate(self, cert_path: Path) -> Optional[CertificateInfo]:
        """Validate and get information about a certificate"""
        try:
            with open(cert_path, 'rb') as cert_file:
                cert_data = cert_file.read()
                x509 = OpenSSL.crypto.load_certificate(OpenSSL.crypto.FILETYPE_PEM, cert_data)
                valid_from = datetime.strptime(x509.get_notBefore().decode(), '%Y%m%d%H%M%SZ')
                valid_until = datetime.strptime(x509.get_notAfter().decode(), '%Y%m%d%H%M%SZ')
                now = datetime.now()
                return CertificateInfo(
                    subject=x509.get_subject().CN,
                    issuer=x509.get_issuer().CN,
                    valid_from=valid_from,
                    valid_until=valid_until,
                    serial_number=str(x509.get_serial_number()),
                    is_valid=valid_from <= now <= valid_until
                )
        except Exception as e:
            self.logger.error(f"Certificate validation error: {e}")
            return None
    def install_certificate(self, cert_path: Path, key_path: Path, name: str) -> bool:
        try:
            import shutil
            # Validate certificate first
            cert_info = self.validate_certificate(cert_path)
            if not cert_info or not cert_info.is_valid:
                self.logger.error("Invalid or corrupt certificate")
                return False
            cert_dest = self.certs_path / f"{name}_cert.pem"
            key_dest = self.certs_path / f"{name}_key.pem"
            shutil.copy2(cert_path, cert_dest)
            shutil.copy2(key_path, key_dest)
            # Update config
            config = self.proxy_manager.config_manager.get_instance()
            config.proxy_settings.gradio_endpoint.cert_name = name
            config.proxy_settings.api_endpoint.cert_name = name
            config.save()
            self.logger.info(f"Certificate '{name}' installed successfully")
            return True
        except Exception as e:
            self.logger.error(f"Certificate installation error: {e}")
            return False
    def remove_certificate(self, name: str) -> bool:
        """Remove a certificate"""
        try:
            cert_path = self.certs_path / f"{name}_cert.pem"
            key_path = self.certs_path / f"{name}_key.pem"
            if cert_path.exists():
                cert_path.unlink()
            if key_path.exists():
                key_path.unlink()
            self.logger.info(f"Certificate '{name}' removed successfully")
            return True
        except Exception as e:
            self.logger.error(f"Certificate removal error: {e}")
            return False
    def get_certificates_status(self) -> Dict[str, Any]:
        """Get status of all installed certificates"""
        certificates = {}
        for cert_file in self.certs_path.glob("*_cert.pem"):
            name = cert_file.name.replace("_cert.pem", "")
            cert_info = self.validate_certificate(cert_file)
            if cert_info:
                certificates[name] = {
                    "subject": cert_info.subject,
                    "issuer": cert_info.issuer,
                    "valid_from": cert_info.valid_from.strftime('%Y-%m-%d %H:%M:%S'),
                    "valid_until": cert_info.valid_until.strftime('%Y-%m-%d %H:%M:%S'),
                    "is_valid": cert_info.is_valid,
                    "days_until_expiry": (cert_info.valid_until - datetime.now()).days
                }
        return certificates
    def check_expiring_certificates(self, days_warning: int = 30) -> Dict[str, int]:
        """Check for certificates nearing expiration"""
        expiring = {}
        for cert_file in self.certs_path.glob("*_cert.pem"):
            name = cert_file.name.replace("_cert.pem", "")
            cert_info = self.validate_certificate(cert_file)
            if cert_info:
                days_left = (cert_info.valid_until - datetime.now()).days
                if days_left < days_warning:
                    expiring[name] = days_left
        return expiring
@@ -0,0 +1,127 @@
 import logging
 import threading
 import time
 from typing import Dict, Any
 import requests
 from dataclasses import dataclass, field
 from datetime import datetime
@dataclass
 class HealthMetrics:
    last_check: float = 0
    uptime_start: float = 0
    gradio_endpoint_up: bool = False
    api_endpoint_up: bool = False
    consecutive_failures: int = 0
    total_restarts: int = 0
    response_times: Dict[str, float] = field(default_factory=dict)
 class HealthMonitor:
    def __init__(self, proxy_manager):
        self.proxy_manager = proxy_manager
        self.logger = logging.getLogger("ProxyHealthMonitor")
        self.metrics = HealthMetrics()
        self.monitoring = False
        self.monitor_thread = None
        self.check_interval = 30  # seconds
    def start_monitoring(self):
        """Start health monitoring thread"""
        if self.monitoring:
            return
        self.monitoring = True
        self.metrics.uptime_start = time.time()
        self.monitor_thread = threading.Thread(target=self._monitor_loop, daemon=True)  # Make thread daemon
        self.monitor_thread.start()
        self.logger.info("Health monitoring started")
    def stop_monitoring(self):
        """Stop health monitoring thread"""
        if not self.monitoring:
            return
        self.monitoring = False
        if self.monitor_thread and self.monitor_thread.is_alive():
            try:
                # Give the thread a chance to finish naturally
                self.monitor_thread.join(timeout=5)
            except Exception as e:
                self.logger.warning(f"Could not gracefully stop monitoring thread: {e}")
            finally:
                self.monitor_thread = None
        self.logger.info("Health monitoring stopped")
    def _monitor_loop(self):
        """Main monitoring loop"""
        while self.monitoring:
            self._check_health()
            time.sleep(self.check_interval)
    def _check_health(self):
        """Check health of proxy endpoints"""
        config = self.proxy_manager.config_manager.get_instance()
        proxy_settings = config.proxy_settings
        self.metrics.last_check = time.time()
        try:
            # Only check endpoints that are enabled and running
            if proxy_settings.gradio_endpoint.enabled == "Enabled":
                try:
                    start_time = time.time()
                    response = requests.get(
                        f"http://127.0.0.1:{config.gradio_port_number}/",  # Use internal port
                        timeout=5
                    )
                    self.metrics.response_times['gradio'] = time.time() - start_time
                    self.metrics.gradio_endpoint_up = response.status_code == 200
                except Exception as e:
                    self.logger.debug(f"Gradio endpoint check failed: {e}")
                    self.metrics.gradio_endpoint_up = False
            if proxy_settings.api_endpoint.enabled == "Enabled":
                try:
                    start_time = time.time()
                    response = requests.get(
                        f"http://127.0.0.1:{config.api_def.api_port_number}/",  # Use internal port
                        timeout=5
                    )
                    self.metrics.response_times['api'] = time.time() - start_time
                    self.metrics.api_endpoint_up = response.status_code == 200
                except Exception as e:
                    self.logger.debug(f"API endpoint check failed: {e}")
                    self.metrics.api_endpoint_up = False
            if not self.metrics.gradio_endpoint_up and not self.metrics.api_endpoint_up:
                self.metrics.consecutive_failures += 1
            else:
                self.metrics.consecutive_failures = 0
        except Exception as e:
            self.logger.error(f"Health check failed: {str(e)}")
            self.metrics.consecutive_failures += 1
        # Only attempt restart if we have multiple complete failures
        if self.metrics.consecutive_failures >= 3:
            self.logger.warning("Too many consecutive failures, attempting restart")
            try:
                if self.proxy_manager.stop_proxy() and self.proxy_manager.start_proxy():
                    self.metrics.total_restarts += 1
                    self.metrics.consecutive_failures = 0
            except Exception as e:
                self.logger.error(f"Failed to restart proxy: {e}")
    def get_health_status(self) -> Dict[str, Any]:
        """Get current health metrics"""
        uptime = time.time() - self.metrics.uptime_start if self.metrics.uptime_start else 0
        return {
            "status": "healthy" if not self.metrics.consecutive_failures else "degraded",
            "uptime": f"{int(uptime/3600)}h {int((uptime%3600)/60)}m",
            "last_check": datetime.fromtimestamp(self.metrics.last_check).strftime('%Y-%m-%d %H:%M:%S') 
                         if self.metrics.last_check else "Never",
            "gradio_endpoint": "up" if self.metrics.gradio_endpoint_up else "down",
            "api_endpoint": "up" if self.metrics.api_endpoint_up else "down",
            "response_times": self.metrics.response_times,
            "total_restarts": self.metrics.total_restarts
        }
@@ -0,0 +1,174 @@
 import gradio as gr
 import tempfile
 from pathlib import Path
 from typing import TYPE_CHECKING
 if TYPE_CHECKING:
    from .proxy_manager import ProxyManager
 def create_proxy_interface(proxy_manager: 'ProxyManager'):
    """Create the proxy management interface"""
    config = proxy_manager.config_manager.get_instance()
    def validate_ports(gradio_port, api_port):
        """Validate port configurations"""
        if gradio_port == api_port and gradio_port != 0 and api_port != 0:
            return False, "Gradio and API ports cannot be the same!"
        if not (0 <= gradio_port <= 65535) or not (0 <= api_port <= 65535):
            return False, "Ports must be between 0 and 65535"
        return True, "Valid port configuration"
    def handle_save_config(proxy_enabled, gradio_enabled, api_enabled, 
                         gradio_port, api_port, startup, external_ip):
        # Validate ports first
        valid, message = validate_ports(gradio_port, api_port)
        if not valid:
            return message
        config.proxy_settings.proxy_enabled = proxy_enabled
        config.proxy_settings.start_on_startup = startup
        # Update Gradio endpoint
        config.proxy_settings.gradio_endpoint.enabled = gradio_enabled
        config.proxy_settings.gradio_endpoint.external_port = int(gradio_port) if gradio_enabled else 0
        config.proxy_settings.gradio_endpoint.external_ip = external_ip
        # Update API endpoint
        config.proxy_settings.api_endpoint.enabled = api_enabled
        config.proxy_settings.api_endpoint.external_port = int(api_port) if api_enabled else 0
        config.proxy_settings.api_endpoint.external_ip = external_ip
        config.save()
        return "Configuration saved successfully"
    def update_control_states(proxy_enabled):
        """Update enabled/disabled state of controls based on master switch"""
        return {
            start_button: gr.update(interactive=proxy_enabled),
            stop_button: gr.update(interactive=proxy_enabled),
            gradio_enabled: gr.update(interactive=proxy_enabled),
            api_enabled: gr.update(interactive=proxy_enabled),
            start_on_startup: gr.update(interactive=proxy_enabled)
        }
    def delete_all_certificates():
        """Delete all certificates and reset config"""
        try:
            # Delete certificate files
            for cert_file in proxy_manager.certs_path.glob("*.pem"):
                cert_file.unlink()
            # Reset config
            config = proxy_manager.config_manager.get_instance()
            config.proxy_settings.gradio_endpoint.cert_name = ""
            config.proxy_settings.api_endpoint.cert_name = ""
            config.save()
            return "All certificates deleted and config reset"
        except Exception as e:
            return f"Error deleting certificates: {str(e)}"
    with gr.Tab("Proxy Settings"):
        with gr.Column():
            with gr.Row():
                with gr.Group():
                    proxy_enabled = gr.Checkbox(
                        label="Enable Proxy System", 
                        value=config.proxy_settings.proxy_enabled,
                        info="Master switch/safety lockout for proxy functionality"
                    )
                    start_on_startup = gr.Checkbox(
                        label="Start Proxy Automatically on Startup", 
                        value=config.proxy_settings.start_on_startup,
                        info="Automatically start proxy when app launches"
                    )
                    external_ip = gr.Textbox(
                        label="External IP",
                        value=config.proxy_settings.gradio_endpoint.external_ip,
                        info="Use 0.0.0.0 to bind to all interfaces"
                    )
                with gr.Group():
                    gr.Markdown("API interface Proxy Management")
                    api_enabled = gr.Dropdown(
                        choices=["Enabled", "Disabled"],
                        value="Enabled" if config.proxy_settings.api_endpoint.enabled == "Enabled" else "Disabled",
                        label="API (TTS Generation) Proxy",
                        info="Enable or Disable Proxying the API (TTS Generation)"
                    )
                    api_port = gr.Number(
                        value=config.proxy_settings.api_endpoint.external_port,
                        label="API Port",
                        info="External port you want to use for the API (TTS Generation)"
                    )                                      
                with gr.Group():
                    gr.Markdown("Gradio interface Proxy Management")
                    gradio_enabled = gr.Dropdown(
                        choices=["Enabled", "Disabled"],
                        value="Enabled" if config.proxy_settings.gradio_endpoint.enabled == "Enabled" else "Disabled",
                        label="Gradio Interface Proxy",
                        info="Enable or Disable Proxying the Gradio interface"
                    )
                    gradio_port = gr.Number(
                        value=config.proxy_settings.gradio_endpoint.external_port,
                        label="Gradio Interface Port",
                        info="External port you want to use for the Gradio interface"
                    )
            # Status and control section
            with gr.Row():
                with gr.Column(scale=2):
                    status_output = gr.Textbox(
                        label="Status",
                        interactive=False,
                        lines=4
                    )
                with gr.Column(scale=1):
                    save_button = gr.Button("Save Configuration", variant="primary")                    
                    with gr.Row():
                        start_button = gr.Button("Start Service")
                        stop_button = gr.Button("Stop Service")
                        status_button = gr.Button("Check Status")
            # Certificate section
            with gr.Row():
                gr.Markdown("### Certificate Management")
            with gr.Row():
                with gr.Column(scale=1):
                    cert_file = gr.File(label="Certificate (.crt/.pem)")
                with gr.Column(scale=1):
                    key_file = gr.File(label="Key (.key)")
                with gr.Column(scale=1):
                    cert_name = gr.Textbox(label="Certificate Name", placeholder="e.g., gradio_cert")
                    upload_button = gr.Button("Upload Certificate", variant="primary")
                    delete_button = gr.Button("Delete Certificates", variant="secondary")
            # Connect everything
            save_button.click(
                fn=handle_save_config,
                inputs=[
                    proxy_enabled,
                    gradio_enabled,
                    api_enabled,
                    gradio_port,
                    api_port,
                    start_on_startup,
                    external_ip
                ],
                outputs=status_output
            )
            proxy_enabled.change(
                fn=update_control_states,
                inputs=[proxy_enabled],
                outputs=[start_button, stop_button, gradio_enabled, api_enabled, start_on_startup]
            )
            start_button.click(fn=proxy_manager.start_proxy, outputs=status_output)
            stop_button.click(fn=proxy_manager.stop_proxy, outputs=status_output)
            status_button.click(fn=proxy_manager.get_status, outputs=status_output)
            upload_button.click(
                fn=proxy_manager.handle_cert_upload,
                inputs=[cert_file, key_file, cert_name],
                outputs=status_output
            )
            delete_button.click(fn=delete_all_certificates, outputs=status_output)
    return proxy_enabled, start_on_startup, gradio_port, api_port
@@ -0,0 +1,94 @@
 import logging
 import threading
 import time
 from typing import Dict, Any
 from collections import defaultdict
 from dataclasses import dataclass, field
 from datetime import datetime, timedelta
@dataclass
 class ProxyMetrics:
    start_time: float = field(default_factory=time.time)
    total_requests: int = 0
    active_connections: int = 0
    bytes_transferred: int = 0
    request_times: Dict[str, float] = field(default_factory=lambda: defaultdict(float))
    requests_per_minute: Dict[str, int] = field(default_factory=lambda: defaultdict(int))
    status_codes: Dict[str, int] = field(default_factory=lambda: defaultdict(int))
 class MetricsCollector:
    def __init__(self, proxy_manager):
        self.proxy_manager = proxy_manager
        self.logger = logging.getLogger("ProxyMetrics")
        self.metrics = ProxyMetrics()
        self.collecting = False
        self.collector_thread = None
        self.collect_interval = 60  # seconds
        self._lock = threading.Lock()
    def start_collecting(self):
        """Start metrics collection"""
        if self.collecting:
            return
        self.collecting = True
        self.metrics = ProxyMetrics()  # Reset metrics
        self.collector_thread = threading.Thread(target=self._collect_loop)
        self.collector_thread.daemon = True
        self.collector_thread.start()
        self.logger.info("Metrics collection started")
    def stop_collecting(self):
        """Stop metrics collection"""
        self.collecting = False
        if self.collector_thread:
            self.collector_thread.join()
        self.logger.info("Metrics collection stopped")
    def _collect_loop(self):
        """Main collection loop"""
        while self.collecting:
            self._update_metrics()
            time.sleep(self.collect_interval)
    def _update_metrics(self):
        """Update periodic metrics"""
        with self._lock:
            # Reset per-minute counters
            self.metrics.requests_per_minute.clear()
    def record_request(self, endpoint: str, status_code: int, response_time: float, bytes_sent: int):
        """Record metrics for a single request"""
        with self._lock:
            self.metrics.total_requests += 1
            self.metrics.bytes_transferred += bytes_sent
            self.metrics.request_times[endpoint] = response_time
            self.metrics.status_codes[str(status_code)] += 1
            self.metrics.requests_per_minute[endpoint] += 1
    def get_current_metrics(self) -> Dict[str, Any]:
        """Get current metrics"""
        with self._lock:
            uptime = time.time() - self.metrics.start_time
            return {
                "uptime": str(timedelta(seconds=int(uptime))),
                "total_requests": self.metrics.total_requests,
                "active_connections": self.metrics.active_connections,
                "bytes_transferred": f"{self.metrics.bytes_transferred / 1024 / 1024:.2f} MB",
                "average_response_times": {
                    endpoint: f"{time:.2f}ms" 
                    for endpoint, time in self.metrics.request_times.items()
                },
                "requests_per_minute": dict(self.metrics.requests_per_minute),
                "status_codes": dict(self.metrics.status_codes)
            }
    def increment_connections(self):
        """Increment active connection count"""
        with self._lock:
            self.metrics.active_connections += 1
    def decrement_connections(self):
        """Decrement active connection count"""
        with self._lock:
            self.metrics.active_connections = max(0, self.metrics.active_connections - 1)
@@ -0,0 +1,235 @@
 import os
 import logging
 import subprocess
 import threading
 from pathlib import Path
 from typing import Optional, Dict, Any
 from dataclasses import asdict
 import psutil
 from logging.handlers import RotatingFileHandler
 import logging
 import os
 from .security import SecurityManager
 from .health_monitor import HealthMonitor
 from .cert_manager import CertificateManager
 from .metrics import MetricsCollector
 class ProxyManager:
    def __init__(self, config_manager, base_path: str = None):
        self.config_manager = config_manager
        # Set up base paths first
        self.base_path = Path(base_path) if base_path else Path(__file__).parent
        # Create required directories
        self.certs_path = self.base_path / "certificates"
        self.logs_path = self.base_path / "logs"
        self.config_path = self.base_path / "config"
        for path in [self.certs_path, self.logs_path, self.config_path]:
            path.mkdir(parents=True, exist_ok=True)
        # Setup logging before managers
        self.setup_logging()
        # Initialize managers after paths are created
        self.security = SecurityManager(self)
        self.health_monitor = HealthMonitor(self)
        self.cert_manager = CertificateManager(self)
        self.metrics = MetricsCollector(self)
        self.proxy_process = None
        self.logger.info("Proxy manager initialized successfully")
    def setup_logging(self):
        """Setup logging configuration with customized format"""
        config = self.config_manager.get_instance()
        log_level = getattr(logging, config.proxy_settings.log_level, logging.INFO)
        # Configure rotating file handler
        log_file = self.logs_path / "proxy.log"
        file_handler = RotatingFileHandler(
            log_file,
            maxBytes=5*1024*1024,
            backupCount=5,
            encoding='utf-8'
        )
        # Configure logging formats
        file_format = logging.Formatter(
            '%(asctime)s - %(name)s - %(levelname)s - %(message)s'
        )
        console_format = logging.Formatter(
            '[AllTalk PRX] %(message)s'
        )
        file_handler.setFormatter(file_format)
        # Setup logger
        self.logger = logging.getLogger("ProxyManager")
        self.logger.setLevel(log_level)
        self.logger.addHandler(file_handler)
        # Only add console handler if proxy debugging is enabled
        if config.debugging.debug_proxy:
            console_handler = logging.StreamHandler()
            console_handler.setFormatter(console_format)
            self.logger.addHandler(console_handler)
            self.logger.info("Proxy debug logging enabled")
    def _monitor_output(self, pipe, name):
        """Monitor subprocess output and redirect to logger"""
        try:
            for line in iter(pipe.readline, ''):
                line = line.strip()
                if line:
                    if name == "stderr":
                        self.logger.error(line)
                    else:
                        # Pass through with our logging format
                        print(line)  # Direct print for formatted output
        except Exception as e:
            self.logger.error(f"Error monitoring {name}: {e}")
        finally:
            pipe.close()
    def start_proxy(self) -> bool:
        """Start the proxy server with current configuration"""
        if self.proxy_process and self.proxy_process.poll() is None:
            self.logger.warning("Proxy already running")
            return False
        config = self.config_manager.get_instance()
        proxy_settings = config.proxy_settings
        try:
            cmd = [
                "python", "-m", "twisted_server",
                "--external_ip_a", proxy_settings.api_endpoint.external_ip if proxy_settings.api_endpoint.enabled == "Enabled" else "0",
                "--internal_ip_a", "127.0.0.1",
                "--external_ip_b", proxy_settings.gradio_endpoint.external_ip if proxy_settings.gradio_endpoint.enabled == "Enabled" else "0",
                "--internal_ip_b", "127.0.0.1",
                "--cert_file_a", proxy_settings.api_endpoint.cert_name or "0",
                "--cert_file_b", proxy_settings.gradio_endpoint.cert_name or "0"
            ]            
            # Start process with pipe for output
            self.proxy_process = subprocess.Popen(
                cmd,
                stdout=subprocess.PIPE,
                stderr=subprocess.PIPE,
                cwd=str(self.base_path),
                text=True,
                bufsize=1,
                env={**os.environ, 'PYTHONUNBUFFERED': '1'},  # Force unbuffered output
                universal_newlines=True
            )
            # Start output monitoring threads
            threading.Thread(target=self._monitor_output, 
                           args=(self.proxy_process.stdout, "stdout"), 
                           daemon=True).start()
            threading.Thread(target=self._monitor_output, 
                           args=(self.proxy_process.stderr, "stderr"), 
                           daemon=True).start()
            if proxy_settings.proxy_enabled:
                self.health_monitor.start_monitoring()
                self.metrics.start_collecting()
            self.logger.info("Proxy server started successfully")
            return True
        except Exception as e:
            self.logger.error(f"Failed to start proxy: {e}")
            return False
    def stop_proxy(self) -> bool:
        """Stop the proxy server and all monitoring"""
        if not self.proxy_process:
            self.logger.warning("No proxy process running")
            return False
        try:
            # Stop all monitoring first
            self.health_monitor.stop_monitoring()
            self.metrics.stop_collecting()
            # Stop the main process
            process = psutil.Process(self.proxy_process.pid)
            for proc in process.children(recursive=True):
                proc.terminate()
            process.terminate()
            try:
                process.wait(timeout=5)
            except psutil.TimeoutExpired:
                process.kill()
            self.proxy_process = None
            self.logger.info("Proxy server stopped successfully")
            return True
        except Exception as e:
            self.logger.error(f"Failed to stop proxy: {e}")
            return False
    def get_status(self) -> Dict[str, Any]:
        """Get comprehensive status of the proxy system"""
        config = self.config_manager.get_instance()
        proxy_config = config.proxy_settings
        status = {
            "running": self.proxy_process is not None and self.proxy_process.poll() is None,
            "config": {
                "gradio_endpoint": asdict(proxy_config.gradio_endpoint),
                "api_endpoint": asdict(proxy_config.api_endpoint),
                "start_on_startup": proxy_config.start_on_startup
            },
            "certificates": self.cert_manager.get_certificates_status(),
            "health": self.health_monitor.get_health_status(),
            "metrics": self.metrics.get_current_metrics(),
            "security": self.security.get_security_status()
        }
        return status
    def handle_cert_upload(self, cert_file, key_file, name):
        """Handle certificate upload from the interface"""
        if not cert_file or not key_file or not name:
            return "Please provide both certificate and key files, and a name"
        cert_path = Path(cert_file.name)
        key_path = Path(key_file.name)
        success = self.cert_manager.install_certificate(cert_path, key_path, name)
        return "Certificate uploaded successfully" if success else "Failed to upload certificate"
    def handle_start(self):
        """Handle proxy start from the interface"""
        success = self.start_proxy()
        return "Proxy Server: Starting..." if success else "Failed to start proxy server"
    def handle_stop(self):
        """Handle proxy stop from the interface"""
        success = self.stop_proxy()
        return "Proxy Server: Stopped" if success else "Failed to stop proxy server"
    def handle_status(self):
        """Handle status request from the interface"""
        status = self.get_status()
        if status['running']:
            return "Proxy Server: Running\n" + \
                f"API Endpoint: {'Active' if status['config']['api_endpoint']['enabled'] == 'Enabled' else 'Disabled'}\n" + \
                f"Gradio Endpoint: {'Active' if status['config']['gradio_endpoint']['enabled'] == 'Enabled' else 'Disabled'}\n" + \
                f"Certificates: {len(status['certificates'])} configured"
        else:
            return "Proxy Server: Stopped"
    def handle_security_event(self, event_type: str, details: Dict[str, Any]):
        """Handle security events from the SecurityManager"""
        self.logger.warning(f"Security event: {event_type} - {details}")
        # Implement security response logic here
    def cleanup(self):
        """Cleanup resources before shutdown"""
        self.stop_proxy()
        self.logger.info("Proxy manager cleanup completed")
@@ -0,0 +1,61 @@
 import logging
 from typing import Dict, Any, Set
 import time
 from collections import defaultdict
 from dataclasses import dataclass
 from threading import Lock
@dataclass
 class SecurityMetrics:
    total_requests: int = 0
    blocked_requests: int = 0
    last_incident: float = 0
    blacklisted_ips: Set[str] = None
    def __post_init__(self):
        if self.blacklisted_ips is None:
            self.blacklisted_ips = set()
 class SecurityManager:
    def __init__(self, proxy_manager):
        self.proxy_manager = proxy_manager
        self.logger = logging.getLogger("ProxySecurityManager")
        self.metrics = SecurityMetrics()
        self.rate_limits = defaultdict(lambda: {"count": 0, "reset_time": 0})
        self.rate_limit_lock = Lock()
    def check_rate_limit(self, ip: str, limit: int = 100, window: int = 60) -> bool:
        """Check if IP has exceeded rate limit"""
        with self.rate_limit_lock:
            current_time = time.time()
            if self.rate_limits[ip]["reset_time"] < current_time:
                self.rate_limits[ip] = {"count": 0, "reset_time": current_time + window}
            self.rate_limits[ip]["count"] += 1
            return self.rate_limits[ip]["count"] <= limit
    def is_blacklisted(self, ip: str) -> bool:
        """Check if IP is blacklisted"""
        return ip in self.metrics.blacklisted_ips
    def add_to_blacklist(self, ip: str, reason: str):
        """Add IP to blacklist"""
        self.metrics.blacklisted_ips.add(ip)
        self.logger.warning(f"IP {ip} blacklisted: {reason}")
        self.metrics.last_incident = time.time()
    def remove_from_blacklist(self, ip: str):
        """Remove IP from blacklist"""
        self.metrics.blacklisted_ips.discard(ip)
        self.logger.info(f"IP {ip} removed from blacklist")
    def get_security_status(self) -> Dict[str, Any]:
        """Get current security metrics"""
        return {
            "total_requests": self.metrics.total_requests,
            "blocked_requests": self.metrics.blocked_requests,
            "blacklisted_ips": len(self.metrics.blacklisted_ips),
            "last_incident": time.strftime('%Y-%m-%d %H:%M:%S', 
                                         time.localtime(self.metrics.last_incident)) 
                            if self.metrics.last_incident else "None"
        }
		`@@ -1 +0,0 @@`
			{"activate": true, "autoplay": true, "branding": "AllTalk ", "narrator_enabled": false, "deepspeed_activate": false, "delete_output_wavs": "Disabled", "ip_address": "127.0.0.1", "language": "English", "low_vram": false, "local_temperature": "0.7", "local_repetition_penalty": "10.0", "model_loaded": true, "model_name": "tts_models/multilingual/multi-dataset/xtts_v2", "narrator_voice": "female_01.wav", "output_folder_wav": "extensions/alltalk_tts/outputs/", "port_number": "7851", "remove_trailing_dots": false, "show_text": true, "tts_method_api_local": false, "tts_method_api_tts": false, "tts_method_xtts_local": true, "voice": "female_01.wav"}
		`@@ -0,0 +1 @@`
							Place the `alltalk.js` in your `\SillyTavern\public\scripts\extensions\tts\` folder, replacing the existing file.