NeMo AutoModel is NVIDIA’s open-source, PyTorch-native distributed training library for large language models and vision-language models. Published on GitHub under the Apache-2.0 license with about 1,033 stars, it is part of the NeMo ecosystem and is under very active development in 2026. Its defining idea is day-0 integration with Hugging Face: new open models can be fine-tuned or pretrained as soon as they land.
AutoModel automates the hard parts of large-scale training. It is built on PyTorch DTensor-native SPMD parallelism, so models shard cleanly across many GPUs, and it ships optimized recipes and custom kernels for popular model families. The supported roster tracks the frontier closely โ Llama, Qwen, Gemma, Mistral, DeepSeek, GLM, GPT-OSS, and multimodal VLM architectures among them. Instead of writing bespoke training loops for every new architecture, teams get out-of-the-box Hugging Face support with production-grade distributed performance, which shortens the path from a model release to a fine-tuned checkpoint.
Training regimes covered include pretraining, supervised fine-tuning (SFT), and parameter-efficient fine-tuning (PEFT) for both LLMs and VLMs, with shared configs and utilities across regimes. That breadth matters in practice: a team can prototype with a light PEFT run, then scale to full fine-tuning or pretraining on the same stack when the experiment graduates. Recipes are maintained for day-0 support of new architectures, so the library tracks the fast-moving open-model ecosystem closely.
NeMo AutoModel is completely free and open source under the Apache-2.0 license.
ML engineers and research teams who fine-tune or pretrain large open models at scale and want Hugging Face compatibility without giving up distributed-training performance.