Is there an in-built queue system to handle different requests to different loaded engines?
A: MEM incorporates a built-in queue system to manage multiple TTS requests across loaded engine instances:
All TTS requests are received through the API port (default: 7401).
The queue system distributes incoming requests among available TTS engine instances.
If all engines are busy, new requests are held in a queue until an engine becomes available.
The system continuously checks for available engines to process waiting requests.
If a request cannot be processed within the allocated time, it will be marked as failed.
This queue system aims to balance the load across all running engines efficiently. Advanced features for queue management and dynamic timeout calculations are available and can be configured in the MEM settings.
Note: As this is a research and testing implementation, its performance in high-load or production environments is not guaranteed. For load testing, you can use the provided mem_load_test.py script to simulate multiple simultaneous requests.
To use the load testing tool: `python mem_load_test.py --requests [number_of_requests] --length [text_length] --url "http://127.0.0.1:7501/api/tts-generate"`
AllTalk MEM (Multi Engine Manager) is a research tool designed to manage and test multiple instances of different Text-to-Speech (TTS) engines being loaded simultaneously, with a view to a centralised engine being able to handle multiple requests simultaneously. AllTalk MEM is currently in a demonstration, testing, and experimental phase.
⚠️ MEM is not intended for production use at this time and there is NO support being offered on MEM. ⚠️
erew123