Lightweight continuous batching OpenAI compatibility using HuggingFace Transformers include T5 and Whisper. - View it on GitHub