Bert4Keras is a light, readable Keras reimplementation of Transformer models — “transformers for humans” — created by Chinese NLP researcher bojone (Jianlin Su). Starting with BERT, it grew into a compact library covering a wide family of pretrained models, and it has earned more than 5,400 stars on GitHub, with its latest updates in 2024.
Where large frameworks optimize for scale, bert4keras optimizes for clarity: the core is a set of small, composable building blocks (embeddings, attention, feed-forward layers) from which BERT, ALBERT, ELECTRA, GPT-2, T5, XLNet and many variants are assembled in a few lines. It can load official pretrained checkpoints, so you can fine-tune real models for classification, sequence labeling, text generation and more, and its examples — especially for Chinese NLP tasks — are extensive and battle-tested. The codebase is famously readable, which has made it a favorite teaching resource for understanding how Transformer architectures actually fit together, as well as a practical fine-tuning toolkit. Development has slowed in recent years as the ecosystem moved on, but the library remains a widely referenced implementation. The author’s companion blog posts, explaining the math and code in detail, are themselves a widely read tutorial series, and many competition and production solutions for Chinese text classification, named-entity recognition and reading comprehension were built on bert4keras.
Bert4Keras is completely free and open source under the Apache-2.0 license.
Bert4Keras is for NLP practitioners and students — particularly in the Chinese NLP community — who want to fine-tune BERT-family models in Keras or understand Transformer implementations without framework magic.