AppAgent is an open-source, LLM-based multimodal agent framework that operates smartphone apps the way a human user does — by looking at screenshots and tapping, swiping and typing on the screen. It comes from Tencent’s QQGYLab, has earned about 6,900 stars on GitHub, is MIT licensed, and was developed through 2025.
Most app automation needs APIs or fragile scripts tied to an app’s internals. AppAgent takes the opposite approach: give a multimodal model such as GPT-4V eyes and a finger. It runs in two phases. During exploration, the agent (or a human demonstrator) interacts with an app while AppAgent records UI elements and their functions into a knowledge document. During deployment, the agent uses that document plus live screenshots to plan and execute multi-step tasks across arbitrary Android apps — no backend access, no app-specific code. Because it works purely through the visible interface, the same framework generalizes across apps and tasks, from social media to maps to shopping. The approach also degrades gracefully: when an app updates its layout, the agent can re-explore rather than breaking like a coordinate-based script. The project is a widely cited reference design for GUI agents and phone-use agents.
Free. AppAgent is open source under the MIT license; you only pay for the multimodal model API calls it makes while running tasks.
Researchers and developers working on GUI agents, mobile automation, app testing or accessibility tooling who want a proven open framework for agents that use phones like people do.