Hiera

Hiera is a hierarchical vision transformer from Meta AI (FAIR) that deliberately strips away the specialized modules other vision architectures rely on — and still comes out faster and more accurate. It is the official implementation of the ICML 2023 oral paper, released under the Apache-2.0 license, and its GitHub repository has drawn over 1,000 stars from the computer-vision community.

Overview

Hiera starts from the Masked Autoencoder (MAE) insight that models learn strong visual features when pre-trained to reconstruct masked inputs. By removing hand-crafted components and other bells and whistles, Hiera ends up simpler to implement, easier to train, and state-of-the-art across image and video recognition benchmarks at the time of release. The repository provides pre-trained checkpoints, training and fine-tuning code, and ready-made configurations, so researchers can reproduce the paper’s results or adapt the backbone to downstream tasks such as object detection and segmentation within standard frameworks. It is a good example of a recurring lesson in deep learning: strong pre-training plus a simple architecture often beats architectural complexity.

Key Features

• Hierarchical vision transformer with a deliberately minimal design
• MAE-style pre-training for strong, transferable visual features
• State-of-the-art results on image and video benchmarks at release
• Pre-trained checkpoints plus training and fine-tuning code
• Official ICML 2023 implementation from Meta AI researchers
• Apache-2.0 licensed for research and commercial use

The repository is organized for reproduction and reuse: configuration files describe each released model variant, and the training code follows the conventions of the MAE codebase that many vision researchers already know. Released checkpoints scale from compact models suitable for quick experiments to larger variants aimed at maximum accuracy, and the same backbone design has been adopted in follow-up work on video understanding, where its efficiency over long frame sequences is particularly valuable. For teams evaluating vision backbones, Hiera’s combination of speed, accuracy, and implementation simplicity makes it an easy baseline to include.

💳 Pricing

Hiera is free and open source under the Apache-2.0 license, including the code and the released model checkpoints.

Who Is It For

Hiera is for computer-vision researchers and engineers who need a strong, modern image or video backbone and prefer an implementation that stays close to the paper it came from.