Acme is DeepMind’s library of reinforcement-learning components and agents, built to make RL research more modular and reproducible. Rather than monolithic algorithm scripts, Acme breaks agents into reusable actors, learners, replay buffers and networks that you can recombine for experiments. The project has about 4,100 stars on GitHub.
Acme provides reference implementations of influential agents — DQN and its variants, MPO, SAC, TD3, IMPALA and others — sharing common infrastructure for environment loops, distributed acting and learning (via Launchpad), and logging. Algorithms are implemented in both JAX and TensorFlow flavours across the codebase’s history, with datasets handled through Reverb replay. The design philosophy is that research code should read like the paper: each agent directory contains a compact, hackable implementation whose pieces (networks, losses, behaviour policies) can be swapped to test an idea. DeepMind used Acme across many published results, and it remains a reference for how to structure serious RL codebases — particularly valuable for researchers who need distributed training without building the plumbing themselves. The repository continues to be maintained in 2026.
Acme’s agent implementations were written to be read and modified: the core files are deliberately compact, with the algorithmic idea kept visible and the distributed machinery factored into reusable adders, builders and environment loops. It pairs with DeepMind’s wider open ecosystem — Launchpad for orchestration, Reverb for experience replay, dm-env for environment specs and Haiku/Sonnet for networks — and the repository includes examples for Atari, DeepMind Control Suite and multi-agent settings. For a lab spinning up RL research, it removes months of infrastructure work.
Acme is free and open source under the Apache-2.0 license.
DeepMind Acme is for RL researchers and engineers who want well-structured, distributed-ready agent implementations to build experiments on.