LlamaGym is an open-source Python library for fine-tuning LLM agents with online reinforcement learning. Released under the MIT license, it has collected about 1,300 stars on GitHub and is available on PyPI as the llamagym package.
LLM agents usually do not learn from experience: they act, but their weights never change. LlamaGym brings the classic Gym workflow from reinforcement learning to language-model agents. Dropping an LLM agent into a Gym-style environment normally means hand-writing a lot of plumbing: conversation context across episodes, batching, reward assignment and PPO setup. LlamaGym packages all of that into a single Agent abstract class. You implement a few methods describing your agent and environment, and the library handles the training loop, so you can iterate quickly on prompts and hyperparameters while the agent genuinely improves through trial, error and reward.
LlamaGym is aimed at closing the loop between prompting and training: instead of endlessly hand-tuning prompts, developers let reward signals shape the agent’s behavior over many episodes. Model calls are abstracted so different LLM providers can be swapped without rewriting the training code, and example setups show agents improving at Gym-style tasks through repeated interaction. It grew out of the open-source agent community around Reworkd, the team behind AgentGPT.
LlamaGym is completely free and open source under the MIT license. Your only costs are the compute and LLM API usage your own training runs consume.
LlamaGym is for researchers and agent developers who want LLM agents that learn from feedback instead of staying frozen, and for RL practitioners curious about training language models in Gym environments.