JetMoE is MyShell’s open-source mixture-of-experts language model project, best known for JetMoE-8B โ a model that reached LLaMA2-7B-level performance while costing less than $0.1 million to train. Published on GitHub under the Apache-2.0 license with about 989 stars, the project is a statement about efficiency: competitive open models do not require multi-billion-dollar training budgets. The repository was last updated in 2024, and the model weights are available on Hugging Face.
JetMoE-8B is a sparsely activated model: although it holds 8 billion parameters in total, only about 2.2 billion are active during inference, which drastically lowers serving cost compared with dense models of similar quality. It was trained exclusively on public datasets with fully open-sourced code, making it academia-friendly and reproducible, and it can be fine-tuned on a limited compute budget that most labs can afford. MyShell also published a chat variant, and an online demo was hosted through Lepton AI’s playground, so anyone can try the model without local hardware.
On public benchmarks evaluated with the Open LLM Leaderboard methodology, JetMoE-8B’s chat variant posts competitive scores across ARC, HellaSwag, MMLU, TruthfulQA, WinoGrande, GSM8K, and code benchmarks such as MBPP and HumanEval โ respectable company for a model whose entire training run cost less than many organizations spend on a single experiment. For the efficiency-obsessed, it remains one of the clearest demonstrations that mixture-of-experts architectures can cut both training and inference costs dramatically.
JetMoE is completely free and open source under the Apache-2.0 license, including the model weights.
Researchers and small labs interested in efficient model training, and developers who want a capable open model whose inference cost stays low thanks to sparse activation.