Data formats and dataframes
Libraries for dataframes and for columnar and array file formats.
Start here
The Featured projects with the most downloads.
-
arrow-rs
Rust implementation of Apache Arrow
11M downloads (30 days) Latest release: 60.1.0, 28 Oct 2026
-
datafusion
DataFusion is an in-memory query engine that uses Apache Arrow as the memory model
3M downloads (30 days) Latest release: 55.2.0, 7 Oct 2026
-
polars
DataFrame library based on Apache Arrow
1.7M downloads (30 days) Latest release: 0.55.2, 6 Aug 2026
All projects
Order: strongest tier first, then most downloads, then name, with unmeasured values last. A library counts downloads over 30 days. An app counts binary downloads per release. An app with none counts its crate downloads over 30 days.
What do the tiers mean?
- Featured
- Shown first. It passes every Featured gate.
- Listed
- Shown in the normal listing. No other tier applies.
- New
- A project whose first release is recent.
- Archived
- No longer maintained, or ended by its author.
Tiers come from published rules over raw signals. Stars alone never qualify a project.
Active dependents: crates from other owners that depend on this project in a current release. One owner counts at most once.
Movement needs more history.
| Project | Downloads | Dependents | Stars | Latest release | Runs on (GPU) | Native code | License |
|---|---|---|---|---|---|---|---|
| Featured 11 projects | |||||||
|
arrow-rs Featured
Rust implementation of Apache Arrow Python bindings |
10,746,755 downloads over 30 days | 2,126 | 3,664 | 60.1.0 | GPU support unknown | Binds native code: ring (feature encryption), liblzma-sys (feature default), zstd-sys (feature default) | Apache-2.0 |
|
datafusion Featured
DataFusion is an in-memory query engine that uses Apache Arrow as the memory model |
2,992,524 downloads over 30 days | 484 | 9,578 | 55.2.0 | GPU support unknown | Binds native code: libmimalloc-sys, liblzma-sys (feature compression), zstd-sys (feature compression) | Apache-2.0 |
|
polars Featured
DataFrame library based on Apache Arrow Python bindings |
1,729,523 downloads over 30 days | 639 | 40,285 | 0.55.2 | GPU support unknown | Binds native code: zstd-sys (feature decompress) | MIT |
|
lance Featured
A columnar data format that is 100x faster than Parquet for random access. |
915,526 downloads over 30 days | 48 | 7,268 | 14.0.2 | GPU support unknown | Binds native code: ring (feature credential-vendor-gcp), lz4-sys (feature lz4), zstd-sys (feature zstd). Build uses a native toolchain: cc | Apache-2.0 |
|
ndarray-npy Featured
.npy and .npz file format support for ndarray |
560,950 downloads over 30 days | 99 | 71 | 0.10.0 | GPU support unknown | No native code found | Apache-2.0 OR MIT |
|
npyz Featured
NumPy file format (de-)serialization. Fork of outdated npy-rs. |
467,634 downloads over 30 days | 36 | 33 | 0.9.1 | GPU support unknown | No native code found | MIT |
|
delta-rs Featured
Native Delta Lake implementation in Rust |
463,085 downloads over 30 days | 33 | 3,356 | 1.1.4 | GPU support unknown | Binds native code: lz4-sys (feature delta-cache), zstd-sys (feature delta-cache) | Apache-2.0 |
|
hdf5-rust Featured
Thread-safe Rust bindings for the HDF5 library. |
253,180 downloads over 30 days | 70 | 85 | 0.15.1 | GPU support unknown | Binds native code: hdf5-metno-sys, blosc-src (feature blosc), lzf-sys (feature lzf), mpi-sys (feature mpio), zfp-sys (feature zfp) | Apache-2.0 OR MIT |
|
zarrs Featured
A library for the Zarr storage format for multidimensional arrays and metadata |
75,052 downloads over 30 days | 51 | 285 | 0.23.14 | GPU support unknown | Binds native code: blosc-src (feature blosc), gdeflate-sys (feature gdeflate), zfp-sys (feature zfp), zstd-sys (feature zstd) | Apache-2.0 OR MIT |
|
hdf5-pure Featured
Pure-Rust HDF5 library: read, write, and edit files in place (WASM-compatible, no C dependencies) |
64,279 downloads over 30 days | 18 | 42 | 0.47.0 | GPU support unknown | No native code found | Apache-2.0 OR MIT |
|
netcdf Featured
High-level NetCDF bindings for Rust |
22,481 downloads over 30 days | 29 | 106 | 0.12.1 | GPU support unknown | Binds native code: netcdf-sys, hdf5-metno-sys, libz-sys, mpi-sys (feature mpi) | Apache-2.0 OR MIT |
| Listed 11 projects | |||||||
|
vortex Listed
An extensible, state-of-the-art framework for columnar compression, and the fastest FOSS columnar file format. Formerly at @spiraldb, now an Incubation Stage project at LFAI&Data, part of the Linux Foundation. |
98,129 downloads over 30 days | 32 | 3,310 | 0.93.0 | GPU support unknown | Binds native code: zstd-sys | Apache-2.0 |
|
dlpark Listed
Pure Rust DLPack implementation for zero-copy tensor interoperability Python bindings |
68,157 downloads over 30 days | 5 | 66 | 0.10.0 | CUDA | No native code found | Apache-2.0 |
|
netcdf-rust Listed
Pure-Rust NetCDF-3 classic and NetCDF-4 (HDF5-backed) file reader |
36,012 downloads over 30 days | 10 | 11 | 0.12.1 | GPU support unknown | No native code found | Apache-2.0 OR MIT |
|
ndarray-csv Listed
Easily read and write homogeneous CSV data to and from 2D ndarrays |
23,808 downloads over 30 days | 7 | 20 | 0.5.4 | GPU support unknown | No native code found | Apache-2.0 OR MIT |
|
MNIST data set parser. |
7,083 downloads over 30 days | 1 | 23 | 0.6.0 | GPU support unknown | Binds native code: curl-sys (feature download), libz-sys (feature download) | MIT |
|
connector-x Listed
Load data from databases to dataframes, the fastest way. |
4,951 downloads over 30 days | 5 | 2,659 | 0.4.6 | GPU support unknown | Binds native code: libsqlite3-sys (feature src_sqlite), odpic-sys (feature src_oracle), openssl-sys (feature src_postgres) | MIT |
|
Data Agent Ready Warehouse : One for Analytics, Search, AI, Python Sandbox. — rebuilt from scratch. Unified architecture on your S3. |
48 binary downloads per release | Not applicable | 9,493 | Not measured | Not measured | Apache-2.0 | |
|
🔨 🍇 💻 🚀 GraphScope: A One-Stop Large-Scale Graph Computing System from Alibaba | 一站式图计算系统 |
48 binary downloads per release | Not applicable | 3,563 | Not measured | Not measured | Apache-2.0 | |
|
elusion Listed
Elusion is a modern DataFrame / Data Engineering / Data Analysis library that combines the familiarity of DataFrame operations (like those in PySpark, Pandas, and Polars) with the power of SQL query building. |
27 downloads over 30 days | 0 | 144 | 9.0.0 | GPU support unknown | Binds native code: ring, openssl-sys | MIT |
|
The pipeline for the OSCAR corpus. |
1 downloads over 30 days | 0 | 178 | 2.0.0 | GPU support unknown | Binds native code: zstd-sys | Apache-2.0 |
|
Drop-in Apache Spark replacement written in Rust, unifying batch processing, stream processing, and compute-intensive AI workloads. |
0 binary downloads per release | Not applicable | 3,491 | Not measured | Not measured | Apache-2.0 | |
| New 1 project | |||||||
|
hdf5-pure-rs New
Pure Rust implementation of the HDF5 file format |
801 downloads over 30 days | 5 | 5 | 0.3.10 | GPU support unknown | No native code found | BSD-3-Clause |
| Archived 2 projects | |||||||
|
black-jack Archived
DataFrame / Series / general data processing in Rust |
1 downloads over 30 days | 0 | 31 | 0.1.0 | GPU support unknown | Binds native code: GSL-sys | GPL-3.0 |
|
A Rust DataFrame implementation, built on Apache Arrow |
Not measured | Not applicable | 280 | Not measured | Not measured | Not measured | Apache-2.0 |