Shypd.aiThe largest open index of AI toolsXinference is an AI Frameworks & Infra tool that allows users to swap GPT for any LLM by changing a single line of code. It enables running open-source, speech, and multimodal models on cloud, on-prem, or a laptop through one unified, production-ready inference API.
Trending
Also listed in
About
Xinference, also known as Xorbits Inference, is a powerful and versatile library designed to serve language, speech recognition, and multimodal models. It simplifies the deployment and serving of both custom and state-of-the-art built-in models with a single command, making it accessible for researchers, developers, and data scientists. Key features include agent-native serving, automatic request batching for improved throughput, and distributed inference across workers. Xinference supports a wide range of models, including MiniMax-M2.7, GLM-5.1, Qwen3.6, and Gemma-4, and integrates seamlessly with popular third-party libraries like LangChain, LlamaIndex, Dify, and Chatbox. It offers flexible APIs, including OpenAI-compatible RESTful API, RPC, CLI, and WebUI, and intelligently utilizes heterogeneous hardware like GPUs and CPUs for accelerated inference.
Capabilities
Pricing & Plans
Open Source · Enterprise
Free
FAQs
Trending
