ru-gpts is a family of open-source GPT-style language models trained specifically for Russian, developed by the AI Forever (formerly SberDevices) team. The repository collects ruGPT3XL, ruGPT3Large, ruGPT3Medium and ruGPT3Small, along with a Russian ruGPT2Large, all autoregressive transformers trained on a large Russian-language corpus with sequence lengths up to 2048 tokens using a mix of sparse and dense attention. The project has around 2,100 GitHub stars, and although the code was last updated in 2022, the models remain a reference point for Russian NLP.
Before multilingual LLMs became common, ruGPT models were the practical way to generate and fine-tune for Russian text. The repo provides model checkpoints, generation examples and Colab notebooks for each size, so you can test text generation immediately or fine-tune a model for tasks such as summarization, dialogue, classification or content drafting in Russian. The XL model was notable at release for bringing GPT-3-scale modeling to a non-English language with openly downloadable weights.
Free. All models and code are released under the Apache-2.0 license; you only supply the compute to run or fine-tune them.
NLP engineers and researchers working with Russian-language text who want a dedicated, openly licensed model family for generation or fine-tuning, and students studying how GPT-3-class models were adapted to a specific language.
The ruGPT models were developed by the SberDevices AI team (now AI Forever) and were widely used in Russian-language products and research before large multilingual models became available. The repository also documents the training setup and model cards for each size, making it a useful historical and practical reference for anyone studying language-specific pretraining, and the smaller checkpoints remain handy for low-resource experimentation on modest hardware.