LLaMA2-Accessory

LLaMA2-Accessory is an open-source toolkit for pretraining, fine-tuning and deploying large language models and multimodal LLMs, published on GitHub with about 2,800 stars. Developed by the Alpha-VLLM team, it extends the LLaMA-Adapter line of work with more advanced training features.

Overview

The toolkit covers the full LLM development workflow: continued pretraining, instruction fine-tuning with parameter-efficient methods, alignment, and deployment of both text-only and vision-language models. Its flagship model, SPHINX, is a versatile multimodal LLM that mixes diverse training tasks, data domains and visual embeddings, and the repo also hosts Large-DiT experiments that apply the same machinery to diffusion transformers for image generation. Pretrained checkpoints, training recipes and demos are included for the released models. The codebase was last updated in early 2025. Detailed documentation on ReadTheDocs walks through data preparation, distributed training configurations and evaluation for both the language and multimodal model tracks. Checkpoints for SPHINX variants and the Large-DiT models are hosted on Hugging Face, and the repository documents their benchmark results on multimodal understanding suites.

Key Features

• Pretraining and fine-tuning pipelines for LLaMA-family LLMs
• Multimodal LLM support with the SPHINX model family
• Parameter-efficient tuning inherited from LLaMA-Adapter
• Data processing and instruction-tuning recipes included
• Pretrained checkpoints and interactive demos
• Deployment code for serving trained models

💳 Pricing

LLaMA2-Accessory is completely free and open source. There are no paid tiers or hosted versions; you train and serve models on your own hardware using the released code and checkpoints.

Who Is It For

LLaMA2-Accessory is for researchers and engineers who want a complete, hackable toolkit for training and customizing open LLMs and multimodal models.