LLM-Blender is an ensembling framework from researchers at USC and AI2 that gets consistently better answers out of open-source LLMs by combining them: first rank the candidate outputs, then fuse the best ones into a single response. Published at ACL 2023, the project is Apache-2.0 licensed and has earned just under 1,000 stars on GitHub.
The framework works in two stages. PairRanker — powered by the PairRM reward model — compares outputs from multiple LLMs pairwise and selects the strongest candidates; notably, PairRM approaches GPT-4-level agreement with human preferences at a tiny 0.4B model size. GenFuser then merges the top candidates, integrating their strengths rather than merely picking a winner. The approach plays to the reality that different open models excel at different prompts, so ensembling smooths out individual weaknesses. Checkpoints load directly from Hugging Face through provided wrappers, and the repository includes the code and data recipes behind the paper’s experiments, making the method straightforward to reproduce or adapt.
The framework is also a useful lens on evaluation itself: PairRM’s strength shows how much signal a small, purpose-trained reward model can extract about answer quality, and the fusion stage demonstrates that generation models can act as editors that reconcile several drafts into a better final answer. Teams experimenting with multi-model routing or council-style setups will find LLM-Blender a concrete, published reference point — with ablations and benchmarks — rather than a hand-wavy design pattern.
LLM-Blender is free and open source under the Apache-2.0 license; you supply the compute for the models you ensemble.
It is for researchers and ML engineers who run several open models and want a principled way to combine them into one stronger system.