llamapool is a distributed worker pool that exposes the chat/completions API of multiple LLM workers. - View it on GitHub