Datasets and model formats

Load and save datasets and model files in the formats you need.

37 projects. Checked .

Start here

The Featured projects with the most downloads.

All projects

Order: strongest tier first, then most downloads, then name, with unmeasured values last. A library counts downloads over 30 days. An app counts binary downloads per release. An app with none counts its crate downloads over 30 days.

What do the tiers mean?
Featured
Shown first. It passes every Featured gate.
Listed
Shown in the normal listing. No other tier applies.
New
A project whose first release is recent.
Archived
No longer maintained, or ended by its author.

Tiers come from published rules over raw signals. Stars alone never qualify a project.

Active dependents: crates from other owners that depend on this project in a current release. One owner counts at most once.

Movement needs more history.

Datasets and model formats
ProjectDownloadsDependentsStarsLatest releaseRuns on (GPU)Native codeLicense
arrow-rs Featured

Rust implementation of Apache Arrow

Python bindings

10,746,755 downloads over 30 days 2,126 3,664 60.1.0 GPU support unknown Binds native code: ring (feature encryption), liblzma-sys (feature default), zstd-sys (feature default) Apache-2.0
safetensors Featured

Provides functions to read and write safetensors which aim to be safer than their PyTorch counterpart.

5,486,837 downloads over 30 days 531 3,943 0.8.0 GPU support unknown No native code found Apache-2.0
hf-hub Featured

Rust client for the Hugging Face Hub API

3,442,405 downloads over 30 days 578 343 1.0.0 GPU support unknown No native code found Apache-2.0
datafusion Featured

DataFusion is an in-memory query engine that uses Apache Arrow as the memory model

2,992,524 downloads over 30 days 484 9,578 55.2.0 GPU support unknown Binds native code: libmimalloc-sys, liblzma-sys (feature compression), zstd-sys (feature compression) Apache-2.0
polars Featured

DataFrame library based on Apache Arrow

Python bindings

1,729,523 downloads over 30 days 639 40,285 0.55.2 GPU support unknown Binds native code: zstd-sys (feature decompress) MIT
lance Featured

A columnar data format that is 100x faster than Parquet for random access.

915,526 downloads over 30 days 48 7,268 14.0.2 GPU support unknown Binds native code: ring (feature credential-vendor-gcp), lz4-sys (feature lz4), zstd-sys (feature zstd). Build uses a native toolchain: cc Apache-2.0
ndarray-npy Featured

.npy and .npz file format support for ndarray

560,950 downloads over 30 days 99 71 0.10.0 GPU support unknown No native code found Apache-2.0 OR MIT
npyz Featured

NumPy file format (de-)serialization. Fork of outdated npy-rs.

467,634 downloads over 30 days 36 33 0.9.1 GPU support unknown No native code found MIT
delta-rs Featured

Native Delta Lake implementation in Rust

463,085 downloads over 30 days 33 3,356 1.1.4 GPU support unknown Binds native code: lz4-sys (feature delta-cache), zstd-sys (feature delta-cache) Apache-2.0
hdf5-rust Featured

Thread-safe Rust bindings for the HDF5 library.

253,180 downloads over 30 days 70 85 0.15.1 GPU support unknown Binds native code: hdf5-metno-sys, blosc-src (feature blosc), lzf-sys (feature lzf), mpi-sys (feature mpio), zfp-sys (feature zfp) Apache-2.0 OR MIT
zarrs Featured

A library for the Zarr storage format for multidimensional arrays and metadata

75,052 downloads over 30 days 51 285 0.23.14 GPU support unknown Binds native code: blosc-src (feature blosc), gdeflate-sys (feature gdeflate), zfp-sys (feature zfp), zstd-sys (feature zstd) Apache-2.0 OR MIT
hdf5-pure Featured

Pure-Rust HDF5 library: read, write, and edit files in place (WASM-compatible, no C dependencies)

64,279 downloads over 30 days 18 42 0.47.0 GPU support unknown No native code found Apache-2.0 OR MIT
netcdf Featured

High-level NetCDF bindings for Rust

22,481 downloads over 30 days 29 106 0.12.1 GPU support unknown Binds native code: netcdf-sys, hdf5-metno-sys, libz-sys, mpi-sys (feature mpi) Apache-2.0 OR MIT
Listed 20 projects
xet-core Listed

Async runtime, configuration, logging, and utility infrastructure for the Hugging Face Xet client tools.

Python bindings

467,103 downloads over 30 days 9 625 1.8.0 GPU support unknown No native code found Apache-2.0
vortex Listed

An extensible, state-of-the-art framework for columnar compression, and the fastest FOSS columnar file format. Formerly at @spiraldb, now an Incubation Stage project at LFAI&Data, part of the Linux Foundation.

98,129 downloads over 30 days 32 3,310 0.93.0 GPU support unknown Binds native code: zstd-sys Apache-2.0
dlpark Listed

Pure Rust DLPack implementation for zero-copy tensor interoperability

Python bindings

68,157 downloads over 30 days 5 66 0.10.0 CUDA No native code found Apache-2.0

Pure-Rust NetCDF-3 classic and NetCDF-4 (HDF5-backed) file reader

36,012 downloads over 30 days 10 11 0.12.1 GPU support unknown No native code found Apache-2.0 OR MIT

Easily read and write homogeneous CSV data to and from 2D ndarrays

23,808 downloads over 30 days 7 20 0.5.4 GPU support unknown No native code found Apache-2.0 OR MIT
burn-onnx Listed

Import and export ONNX models with the Burn framework

14,530 downloads over 30 days 23 141 0.22.0 GPU support unknown No native code found Apache-2.0 OR MIT
matfile Listed Dormant

Matfile is a library for reading and writing Matlab ".mat" data files.

10,737 downloads over 30 days 5 14 0.5.0 GPU support unknown No native code found MIT
mnist Listed Dormant

MNIST data set parser.

7,083 downloads over 30 days 1 23 0.6.0 GPU support unknown Binds native code: curl-sys (feature download), libz-sys (feature download) MIT

Load data from databases to dataframes, the fastest way.

4,951 downloads over 30 days 5 2,659 0.4.6 GPU support unknown Binds native code: libsqlite3-sys (feature src_sqlite), odpic-sys (feature src_oracle), openssl-sys (feature src_postgres) MIT

Minimal ONNX model loader for extracting weights, tensors, operations, and graph structure

3,975 downloads over 30 days 4 6 0.5.2 GPU support unknown No native code found MIT
gguf Listed

GGUF file parser

2,472 downloads over 30 days 4 25 0.1.8 GPU support unknown No native code found MIT
onnx-rs Listed

Zero-dependency ONNX model parser in Rust

453 downloads over 30 days 5 6 0.1.3 GPU support unknown No native code found MIT
oxen App Listed

Oxen server is a fast data version control backend, supporting local disk and S3. Self host your repositories on your own storage, or use the hosted platform on Oxen.ai. Stores, syncs, and serves versioned datasets, model checkpoints, game assets, studio media, and any large data.

126 binary downloads per release Not applicable 1,208 0.67.0 GPU support unknown Binds native code: librocksdb-sys, lmdb-master-sys, ffmpeg-sys-next (feature ffmpeg), libduckdb-sys (feature docs) Apache-2.0
databend App Listed

Data Agent Ready Warehouse : One for Analytics, Search, AI, Python Sandbox. — rebuilt from scratch. Unified architecture on your S3.

48 binary downloads per release Not applicable 9,493 Not measured Not measured Apache-2.0
GraphScope App Listed

🔨 🍇 💻 🚀 GraphScope: A One-Stop Large-Scale Graph Computing System from Alibaba | 一站式图计算系统

48 binary downloads per release Not applicable 3,563 Not measured Not measured Apache-2.0
elusion Listed

Elusion is a modern DataFrame / Data Engineering / Data Analysis library that combines the familiarity of DataFrame operations (like those in PySpark, Pandas, and Polars) with the power of SQL query building.

27 downloads over 30 days 0 144 9.0.0 GPU support unknown Binds native code: ring, openssl-sys MIT
cmf App Listed

One backbone, many specialists. CMF is a self-describing, memory-mappable single-file format and a dependency-free runtime for quantized LLMs — zero-copy execution on CPU or GPU (Vulkan · Metal · DX12) that overlays task-specialized skills onto one shared model, with no per-model duplication.

24 binary downloads per release Not applicable 52 Not measured Not measured Apache-2.0
ungoliant Listed Dormant

The pipeline for the OSCAR corpus.

1 downloads over 30 days 0 178 2.0.0 GPU support unknown Binds native code: zstd-sys Apache-2.0
sail App Listed

Drop-in Apache Spark replacement written in Rust, unifying batch processing, stream processing, and compute-intensive AI workloads.

0 binary downloads per release Not applicable 3,491 Not measured Not measured Apache-2.0
nebula App Listed Dormant

A Package Manager for Machine Learning Datasets and Models.

Not measured Not applicable 85 Not measured Not measured Not measured BSD-3-Clause
New 2 projects

Pure Rust implementation of the HDF5 file format

801 downloads over 30 days 5 5 0.3.10 GPU support unknown No native code found BSD-3-Clause
compressr App New

Model conversion and quantization with no Python: reads SafeTensors, GGUF, TCF and PyTorch state dicts, writes SafeTensors, GGUF and TCF, and quantizes to Q2_K through Q8_0 and IQ4 formats, including Q4_K_M mixes. GPTQ, AWQ, NF4, FP8, EXL2 and ONNX input are not implemented.

Python bindings

23 downloads over 30 days Not applicable 0 0.2.1 GPU support unknown No native code found Apache-2.0
Archived 2 projects
black-jack Archived

DataFrame / Series / general data processing in Rust

1 downloads over 30 days 0 31 0.1.0 GPU support unknown Binds native code: GSL-sys GPL-3.0
rust-dataframe App Archived

A Rust DataFrame implementation, built on Apache Arrow

Not measured Not applicable 280 Not measured Not measured Not measured Apache-2.0