Shypd.aiThe largest open index of AI toolsAphrodite Engine is a large-scale LLM inference engine that optimizes the serving of HuggingFace-compatible models. It delivers high-performance model inference for multiple concurrent users, built on vLLM's Paged Attention technology.
Trending
About
Aphrodite Engine is an inference engine designed to optimize the serving of HuggingFace-compatible large language models (LLMs) at scale. Leveraging vLLM's Paged Attention technology, it provides high-performance model inference for multiple concurrent users. Developed through a collaboration between PygmalionAI and Ruliad, Aphrodite serves as the backend engine powering their chat platforms and API infrastructure. Key features include continuous batching, efficient K/V management, optimized CUDA kernels, and extensive quantization support (AQLM, AWQ, GPTQ, etc.). It also offers distributed inference, 8-bit KV Cache, modern sampler support, speculative decoding, and multimodal capabilities. The engine supports Linux and Windows (WSL2) with Python 3.9 to 3.12, and requires CUDA >= 12, supporting a wide range of GPUs including AMD, Intel, Google TPU, and AWS Inferentia.
Capabilities
Pricing & Plans
Open Source
Free
FAQs
Trending
