LLM2Vec is an open-source method and codebase from McGill NLP that turns decoder-only large language models into powerful text encoders. It has 1,729 stars on GitHub and is under active development in 2026. The project accompanies the paper ‘LLM2Vec: Large Language Models Are Secretly Powerful Text Encoders’.
Decoder-only LLMs are trained to generate text left to right, which makes their raw embeddings surprisingly weak for search and classification. LLM2Vec fixes that with a three-step recipe: enable bidirectional attention so tokens can see the whole sequence, train with masked next-token prediction to adapt the model to the new attention pattern, and apply unsupervised contrastive learning. The resulting encoders can be used directly or fine-tuned further, reaching state-of-the-art results on embedding benchmarks such as MTEB. Official converted models are published on Hugging Face, and a follow-up project, LLM2Vec-Gen, extends the idea to generative embeddings. The repository includes training scripts for each stage of the recipe, evaluation code, and instructions for converting specific model families. Because converted models keep the scale and much of the knowledge of the base LLM, they tend to outperform older encoder-only models on retrieval-heavy tasks.
LLM2Vec is free and open source under the MIT license, including the code and the released model checkpoints. Training runs use your own compute.
LLM2Vec is for NLP researchers and search engineers who want embeddings from modern open LLMs, and for teams building retrieval systems that need stronger encoders than classic BERT-style models. The project is actively maintained, with support added as new open model families are released.