torch2trt

torch2trt is an open-source converter from NVIDIA’s AI IoT team that turns PyTorch models into TensorRT engines with a single function call, unlocking fast GPU inference on Jetson devices and desktop GPUs. The GitHub repository has just under 4,900 stars and is a standard tool in the edge AI community for squeezing real-time performance out of trained models.

Overview

TensorRT delivers big speedups through layer fusion, precision calibration and kernel auto-tuning, but building engines by hand is tedious. torch2trt traces your PyTorch module, maps its operators to TensorRT layers, and returns a drop-in replacement module with the same interface. It supports FP16 and INT8 execution, dynamic shapes, and a plugin mechanism for operators TensorRT does not cover natively, and the converted model keeps living inside your PyTorch code. That drop-in design is the project’s signature advantage: optimization becomes an experiment you can try in an afternoon and measure, rather than a rewrite, and if a layer is unsupported you can fall back to PyTorch execution for just that part of the network.

Key Features

• One-line conversion of PyTorch modules to TensorRT engines
• FP16 and INT8 (calibrated) precision support for maximum Jetson throughput
• Drop-in module replacement: converted models keep the original call signature
• Custom converter registry and plugin support for unsupported operators
• Benchmarks and examples for popular vision models on Jetson Nano, TX2 and Xavier
• Dynamic input shape handling for variable-size workloads

💳 Pricing

Free and open source under the MIT license. TensorRT itself is part of NVIDIA’s free SDK stack; there are no paid tiers for this converter.

Who Is It For

Edge AI and robotics developers deploying PyTorch vision models on NVIDIA Jetson hardware who need real-time frame rates without rewriting models in a new framework.