Video PreTraining (VPT) is an open-source research project from OpenAI showing that agents can learn to act by watching unlabeled online videos. The repository has 1,753 stars on GitHub and was last updated in 2025. It contains the code and pretrained models behind OpenAI’s well-known Minecraft-playing agents.
Labeling agent behavior by hand is expensive, but the internet is full of videos of people doing tasks. VPT bridges that gap: first, a small labeled dataset trains an inverse dynamics model that can infer the actions behind recorded gameplay; that model then labels a huge corpus of online Minecraft videos, and a foundation agent is pretrained on the pseudo-labeled data. The pretrained agent can play Minecraft for extended sessions, craft items, and, after fine-tuning with reinforcement learning, reach diamond tools (a long-standing challenge task) far more reliably than agents trained from scratch. The approach mattered well beyond Minecraft: it demonstrated that the web’s video, paired with a small amount of labeled interaction data, could pretrain generally capable embodied agents, an idea that later work on robotics and game agents has repeatedly built on. The repository also documents the exact environment versions needed to reproduce the original results.
VPT is free and open source under the MIT license, including the pretrained model files. You supply your own compute for running or fine-tuning the agents.
VPT is for reinforcement learning and imitation learning researchers studying how to pretrain agents from passive video, and for anyone reproducing a landmark result in embodied AI.