Shypd.aiThe largest open index of AI toolskeras-transformer is a Python library for building Transformer models using Keras. It provides components for creating BERT and GPT models, supporting positional encoding, attention masking, and memory-compressed attention.
Trending
Also listed in
About
Keras-transformer is a Python library designed to facilitate the construction of (Universal) Transformer models within the Keras framework. It offers essential building blocks such as positional encoding, embeddings, attention masking, and memory-compressed attention. The library also supports Adaptive Computation Time (ACT) and provides a general implementation for BERT models, making it highly relevant for Natural Language Processing (NLP) tasks. Developers can flexibly piece together multi-step Transformer models using its Keras layers, or customize existing components like self-attention and activation functions. The repository includes practical examples demonstrating its application in language modeling with BERT and GPT on datasets like WikiText-2.
Capabilities
Pricing & Plans
Open Source
Free
FAQs
Trending
