Cortex is an open-source platform for deploying machine learning models in production on your own AWS account. Instead of hand-wiring EC2 instances, load balancers and autoscaling groups for every model, you describe your deployment in a simple configuration file and Cortex handles the infrastructure. The project has earned about 8,000 stars on GitHub; the codebase was last updated in 2024 and remains a widely referenced approach to self-hosted model serving.
Cortex turns trained models — from TensorFlow, PyTorch, ONNX or any Python-based framework — into autoscaling inference APIs running on Kubernetes-grade infrastructure inside your AWS account. It provisions GPU and CPU instances, configures autoscaling so capacity follows traffic, supports rolling updates so new model versions deploy without downtime, and can run multiple models behind a single endpoint to cut costs. Because everything runs in your own cloud account, you keep full control of data, networking and spend — there is no external control plane and no per-request markup. Its declarative cortex.yaml workflow made “infrastructure as configuration” accessible to data science teams that wanted production serving without becoming full-time platform engineers.
Cortex is free and open source under the Apache 2.0 license. You pay only for the AWS infrastructure it provisions in your account. (The old cortexlabs.com domain is no longer the project’s home; the GitHub repository is the canonical source.)
Cortex is for ML teams that want production-grade, autoscaling model serving on AWS without paying for a managed platform or building serving infrastructure from scratch.