Aphrodite-Engine
Visit ToolAphrodite Engine is a large-scale LLM inference engine that optimizes the serving of HuggingFace-compatible models. It delivers high-performance model inference for multiple concurrent users, built on vLLM's Paged Attention technology.
At a glance
Trending