Engram (DeepSeek) is the official implementation of DeepSeek AI’s research on conditional memory for large language models, presented in the paper Conditional Memory via Scalable Lookup. The repository has earned more than 4,700 stars on GitHub since its 2026 release. Note that this listing covers DeepSeek’s Engram module specifically; it is a distinct project from other products named Engram listed on Tooloso.
Mixture-of-Experts scales model capacity through conditional computation, but Transformers have no native primitive for simply looking up stored knowledge. Engram modernises classic N-gram embeddings into a memory module with constant-time lookup, adding conditional memory as a complementary axis of sparsity alongside MoE. The paper formulates how to split capacity between neural computation and static memory, identifying a U-shaped scaling law, and verifies under strict iso-parameter and iso-FLOPs conditions that an Engram-27B model consistently improves over MoE baselines on knowledge, reasoning, code and mathematics benchmarks. Its deterministic addressing also lets very large embedding tables be offloaded to host memory with minimal overhead.
The authors argue that separating lookup from computation mirrors how memory and reasoning are organised in cognition, and the mechanistic analysis in the paper suggests the module relieves early layers from reconstructing static patterns, preserving effective depth for complex reasoning. The released code lets other teams test the module inside their own training stacks and reproduce the paper’s controlled comparisons against MoE baselines.
Engram is free and open source under the Apache-2.0 license. The implementation and paper are available on GitHub at no cost for research and experimentation.
Engram is for LLM researchers and model architects exploring alternatives to pure MoE scaling. Teams training large models can study the design and code to understand how native memory lookup affects capability, efficiency and system design in next-generation architectures.