Open Interface is an open-source application that lets a large language model control any computer through natural-language instructions. The GitHub project has collected around 2,730 stars and was last actively updated in 2026. Inspired by the vision of an LLM operating system, it watches the screen with a vision model, plans the steps needed, and drives the mouse and keyboard to complete tasks in whatever app is open.
Give Open Interface a goal in plain English, such as finding a file, filling a form, or checking information across apps, and it screenshots the desktop, asks a multimodal model like GPT-4V what to do next, and executes the clicks and keystrokes with PyAutoGUI. It works across Windows, macOS, and Linux, and because it operates at the GUI level, it can automate software that has no API at all. A simple interface shows the plan and progress while the agent works.
The project is a compact demonstration of the computer-use pattern that larger agent products later popularized: perceive the screen, reason about the next action, act, and repeat while verifying progress. It keeps a visible log of the model’s instructions so users understand what the agent is doing, and it supports connecting different multimodal backends as they improve. As an open codebase, it is also a practical starting point for building your own desktop automation agent.
Open Interface is free and open source under the GPL-3.0 license. The only cost is usage of the vision-capable model API you connect it to.
Developers and power users experimenting with computer-use agents, accessibility use cases, and anyone who wants to delegate repetitive desktop tasks to an LLM.