Shypd.aiThe largest open index of AI toolsTileRT is a Coding & Development tool that provides a tile-based runtime for ultra-low-latency LLM inference. It significantly reduces token generation latency for large language models, enabling millisecond-level time per output token.
Trending
About
TileRT is an open-source, tile-based runtime engineered for ultra-low-latency Large Language Model (LLM) inference. It aims to push the boundaries of LLM latency without compromising model size or quality, allowing models with hundreds of billions of parameters to achieve millisecond-level time per output token (TPOT). Unlike traditional inference systems optimized for high-throughput batch processing, TileRT prioritizes responsiveness, making it ideal for applications like high-frequency trading, interactive AI, real-time decision-making, and AI-assisted coding. It achieves this by decomposing LLM operators into fine-grained tile-level tasks and dynamically rescheduling computation, I/O, and communication across multiple devices to minimize idle time and improve hardware utilization. TileRT currently supports models like GLM-5 and DeepSeek-V3.2 and offers Multi-Token Prediction (MTP) for efficient longer output generation.
Capabilities
Pricing & Plans
Open Source
Free
FAQs
Trending
