InternGPT (iGPT) is an open-source interactive multimodal platform from OpenGVLab (Shanghai AI Laboratory) that lets you communicate with foundation models by pointing, clicking and dragging as well as typing. Built on GitHub, the project has gathered around 3,200 stars and was last actively updated in 2024. It combines large language models with vision models so users can direct the system at regions of an image or video instead of describing everything in words.
InternGPT wires together a controller LLM with a toolbox of specialist models: Segment Anything (SAM) for interactive segmentation, DragGAN for point-based image editing, ImageBind for cross-modal understanding, and InternImage-based perception models. A Husky-based conversational core coordinates these tools, so a user can click on an object, ask questions about it, remove or replace it, generate captions, answer visual questions, and even produce video, all in one session. The hosted demo at igpt.opengvlab.com showcases the interactions directly in the browser.
The project accompanies the team’s research on interacting with ChatGPT beyond language, and the codebase is organised so new vision models can be registered as tools the controller can call. Example workflows include circling an object to remove it from a photo, generating an image from a rough sketch plus a text instruction, describing a video clip, and segmenting whatever the user points at. Because the platform is modular and the demo is public, InternGPT became a frequently cited reference for how multimodal agents can expose specialist models through a single conversational interface.
InternGPT is free and open source under the Apache-2.0 license. You can run the demo online or self-host the platform; the only costs are the compute and any external model APIs you choose to connect.
Researchers and developers exploring human-computer interaction with multimodal models, and anyone who wants a hands-on demo of what pointing-driven vision-language systems can do.