Is there an in-built queue system to handle different requests to different loaded engines?
A: MEM incorporates a built-in queue system to manage multiple TTS requests across loaded engine instances:
All TTS requests are received through the API port (default: 7401).
The queue system distributes incoming requests among available TTS engine instances.
If all engines are busy, new requests are held in a queue until an engine becomes available.
The system continuously checks for available engines to process waiting requests.
If a request cannot be processed within the allocated time, it will be marked as failed.
This queue system aims to balance the load across all running engines efficiently. Advanced features for queue management and dynamic timeout calculations are available and can be configured in the MEM settings.
Note: As this is a research and testing implementation, its performance in high-load or production environments is not guaranteed. For load testing, you can use the provided mem_load_test.py script to simulate multiple simultaneous requests.
To use the load testing tool: `python mem_load_test.py --requests [number_of_requests] --length [text_length] --url "http://127.0.0.1:7501/api/tts-generate"`
AllTalk MEM (Multi Engine Manager) is a research tool designed to manage and test multiple instances of different Text-to-Speech (TTS) engines being loaded simultaneously, with a view to a centralised engine being able to handle multiple requests simultaneously. AllTalk MEM is currently in a demonstration, testing, and experimental phase.
⚠️ MEM is not intended for production use at this time and there is NO support being offered on MEM. ⚠️
Added an additional note to the first run splash screen to explain additional models/engines are managed in the gradio interface.
Doubled the effort to mark first time run has been completed.
🟧 Using Single Voice Samples
Voice samples are stored in `/alltalk_tts/voices/` and should be named using the following format `name.wav`. These files will be listed as `name.wav` in the available voices list.
🟧 Using Multiple Voice Samples
If you have multiple voice samples for a single voice, you can organize them into subfolders within the `/alltalk_tts/voices/` directory. Each subfolder should be named according to the voice it contains, up to 5 voice samples will be randomly selected for use.
• Each subfolder should reflect the name or type of the voice it contains (e.g., female_voice, male_voice).
• The voice samples inside each subfolder should follow the standard .wav format.
• An example folder path would be `/alltalk_tts/voices/mynewvoice/` and this would be listed in the available voices list as `mynewvoice/`.
This organization allows for easy selection and management of multiple voice samples while ensuring the system can correctly identify and utilize each voice. Manual CURL API requests would send the folder in the format `mynewvoice/`.
Attempt to resolve suspicion that some OS/Language character sets may be using a comma for decimal separation, rather than a period e.g. `10,0` vs `10.0`
This could be affecting the repetition penalty setting with XTTS when called from a command line/terminal request where the period `.` isnt being specifed.
Can call /api/openai-voicemap with
curl -X PUT "http://localhost:7851/api/openai-voicemap" -H "Content-Type: application/json" -d "{\"alloy\":\"female_01.wav\",\"echo\":\"female_01.wav\",\"fable\":\"female_01.wav\",\"nova\":\"female_01.wav\",\"onyx\":\"male_01.wav\",\"shimmer\":\"male_02.wav\"}"
Caveats:
- You have to update all the voices in one go, not just one voice.
- Its up to you to ensure you have the correct voices for the TTS engine that's loaded in which you can find currently available on /api/voices
- The in-memory mappings will update, but Gradio will have cached what was originally in the settings, so it wont display the changes in the Gradio interface without a restart of AllTalk/Gradio
erew123