Shypd.aiThe largest open index of AI toolsAdan is an open-source deep learning optimizer that implements the Adaptive Nesterov Momentum Algorithm for faster optimization of deep models. It is available as a PyTorch implementation and supports various deep learning tasks.
Trending
About
Adan is an open-source PyTorch implementation of the Adaptive Nesterov Momentum Algorithm, designed to accelerate the optimization of deep learning models. It offers significant improvements in training speed and efficiency across a wide range of applications, including large language models (LLMs) like MoE and GPT2, vision tasks such as ViTs and ResNets, and NLP tasks like BERT and Transformer-XL. Adan is integrated into popular frameworks like NVIDIA's NeMo, Huggingface's Timm, and OpenMMLab's MMClassification. It also serves as the default optimizer for projects like High-Fidelity Text-to-3D Generation and Masked Diffusion Transformer V2. The tool provides a 'FusedAdan' version for reduced memory footprint and offers detailed instructions and experimental results for reproducibility.
Capabilities
Pricing & Plans
Open Source
Free
FAQs
Trending
