Daft

Daft is a high-performance data engine built for AI and multimodal workloads, developed by Eventual and open-sourced on GitHub under the Apache-2.0 license. It processes images, audio, video, text and structured data at any scale — from a laptop to petabyte-scale distributed clusters — using a familiar DataFrame and SQL interface. The project has earned more than 5,800 stars on GitHub and is actively developed, with its latest updates in 2026, and it is used in production at companies such as Amazon and Anthropic.

Overview

Daft is written in Rust on top of Apache Arrow and is designed specifically for the data pipelines that feed modern AI: preparing training datasets, running batch inference and building search indexes. Unlike general-purpose engines, it treats multimodal data as first-class — images, video frames, audio and embeddings are native column types you can decode, filter and transform like any other column. Model operations such as LLM extraction and text embedding are built in as native operators, with a managed UDF runtime that handles batching, retries and error handling for you.

Key Features

• Multimodal column types for images, video, audio, text and embeddings
• Native model operators: LLM extraction, embeddings and structured outputs as first-class operations
• Familiar Python DataFrame and SQL APIs — if you know Pandas or Spark, you know Daft
• Rust core with zero-copy Arrow execution; runs the same code locally or distributed on Ray
• Managed UDF runtime with automatic batching, retries and error handling
• Connects to Parquet, Iceberg, Delta Lake, Hugging Face datasets and vector databases

💳 Pricing

Daft is free and open source under the Apache-2.0 license. You run it on your own infrastructure and only pay for the compute and any model APIs you plug in.

Who Is It For

Daft is for data and ML engineers building multimodal data pipelines — dataset preparation, batch inference, embedding generation and AI search — who want Spark-class scale without Spark-class complexity.