frostem

Library New

Pre-built Snowball stemmers for Rust (tracks snowball main)

Checked · Sources

Downloads
504,939
Dependents
4
Stars
4
Forks
1
Contributors
2
Commits
26
Latest release
2.20261016.1 ·

About

From the project README ·

Pre-built Snowball stemmers for Rust.

frostem tracks upstream snowballstem/snowball main and publishes the generated Rust stemmers behind a small, stable facade. You do not need the Snowball compiler at build time.

Install

[dependencies]
frostem = "1"

All algorithms are enabled by default. To shrink the dependency, disable defaults and pick languages:

[dependencies]
frostem = { version = "1", default-features = false, features = ["english", "german"] }

Usage

use frostem::{Algorithm, Stemmer};

fn main() {
    let stemmer = Stemmer::new(Algorithm::English);
    assert_eq!(stemmer.stem("fruitlessly"), "fruitless");

    // Names and aliases from Snowball modules.txt (case-insensitive)
    let de = Stemmer::try_from_name("de").unwrap();
    assert_eq!(de.stem("automaten"), "automat");
}

Inputs should already be lowercased when that is meaningful for the language.

Upstream provenance (git commit / time) is recorded in upstream-pin.toml in this repository, not exposed as Rust API. Consumers should rely on the crate version (major.YYYYMMDD.patch).

Read the full README

Versioning

ComponentMeaning
majorfrostem public Rust API
minorUTC date (YYYYMMDD) of the upstream commit that last changed algorithms/
patchReleases with no algorithms/ change (codegen/runtime-only upstream sync, frostem-only fixes, collisions)

Example: 1.20260729.0 — facade API v1; last upstream algorithm change on 2026-07-29 UTC.

  • API breaks are detected with cargo-semver-checks and bump major.
  • Daily automation publishes whenever upstream main HEAD advances.
  • minor advances only when upstream algorithms/ differs from the previous pin (stem definitions / new or removed .sbl files). The minor number is that commit’s UTC YYYYMMDD.
  • Upstream movement that does not touch algorithms/ (compiler, runtime, docs, other backends, …) keeps the previous minor and bumps patch.
  • Tag or crates.io version collisions also bump patch.

Algorithm is #[non_exhaustive]: new upstream languages are a minor change, not a major break.

Algorithms

Every Snowball algorithm under algorithms/*.sbl is included (including curiosities such as lovins). Feature names match the algorithm ids (e.g. english, dutch_porter, earlymodernenglish).

Use Algorithm::all() for the set enabled in a given build.

Maintenance

Regenerate from upstream (requires a C toolchain and make):

python3 scripts/sync_from_snowball.py
cargo test

Options:

--snowball-dir DIR   use an existing checkout instead of .snowball-src
--major N            override major (default: keep current)
--minor YYYYMMDD     override minor (default: algorithms/ gate)
--patch N            override patch (default: algorithms/ gate)

scripts/sync_from_snowball.py applies the algorithms gate locally and in CI: if upstream algorithms/ changed vs upstream-pin.toml, minor becomes the commit’s UTC YYYYMMDD and patch is 0; otherwise minor is kept and patch increments. Daily CI only adds major bumps from cargo-semver-checks and extra patch bumps for tag/crates.io collisions.

CI runs a daily sync against main, publishes to crates.io when the upstream commit changes (Trusted Publishing / OIDC), and creates a GitHub Release whose notes summarize Snowball upstream commits and frostem changes since the previous tag.

Trusted Publishing

Publishing from CI uses Trusted Publishing (OIDC via rust-lang/crates-io-auth-action). No CARGO_REGISTRY_TOKEN secret is required for routine releases.

Configured for this repo as:

cargo install cargo-trustpub
cargo trustpub add --publisher github --owner Xuanwo --repo frostem --pipeline daily.yml
cargo trustpub status

(daily.yml must match .github/workflows/daily.yml. No GitHub Environment is used.)

After a successful CI publish, you may optionally require Trusted Publishing for all future versions:

cargo trustpub set --trustpub-only true

If you rename the workflow file or add a GitHub Environment, update the config with cargo trustpub (or the crates.io UI) so it matches exactly.

License

BSD-3-Clause. Stemming algorithms and the Snowball runtime are from the Snowball project; see LICENSE and LICENSE-SNOWBALL.

Runs on
GPU support unknown
What shows it
  • no dependency or feature matches a backend row
Native code
None found
  • No native code found
Python bindings
No Python binding crate found
License
BSD-3-Clause

Compare with similar projects

Order: strongest tier first, then most downloads, then name, with unmeasured values last. A library counts downloads over 30 days. An app counts binary downloads per release. An app with none counts its crate downloads over 30 days. ML Rust runs this catalogue, so its closest project always gets a row.

frostem beside the projects that share a topic with it
ProjectDownloadsDependentsStarsLatest releaseRuns on (GPU)Native codeLicense
frostem This page New

Pre-built Snowball stemmers for Rust (tracks snowball main)

504,939 downloads over 30 days 4 4 2.20261016.1 GPU support unknown No native code found BSD-3-Clause
tiktoken-rs Featured

Library for encoding and decoding with the tiktoken library in Rust

3,939,182 downloads over 30 days 449 410 0.12.1 GPU support unknown No native code found MIT
whatlang-rs Featured

Fast and lightweight language identification library for Rust.

882,019 downloads over 30 days 74 1,087 0.18.0 GPU support unknown No native code found MIT
rust-stemmers Listed Dormant

A rust implementation of some popular snowball stemming algorithms

5,399,112 downloads over 30 days 128 136 1.2.0 GPU support unknown No native code found BSD-3-Clause OR MIT
whichlang Listed

A blazingly fast and lightweight language detection library for Rust.

52,880 downloads over 30 days 11 464 0.1.1 GPU support unknown No native code found MIT
snowball Listed

Rust implementation of Snowball stemming algorithms for 33 languages

21,654 downloads over 30 days 3 891 0.1.0 GPU support unknown No native code found BSD-3-Clause
splintr Listed

Tokenizer for Rust and Python: pure Rust by default, with 18 bundled vocabularies and four backends covering BPE, SentencePiece and WordPiece. Its output matches Hugging Face tokenizers id for id on 14 model families, including GPT-2, Qwen, Whisper, T5 and BERT.

Python bindings

8,204 downloads over 30 days 7 63 0.22.2 GPU support unknown Binds native code: pcre2-sys (feature pcre2) MIT
Why NewIts first release on 4 Aug 2026 is under 180 days old.

The check tests the Featured gates once the first release passes this age limit.

What else the check found

  • The last activity is on 17 Oct 2026, 23 days ago. The rules expect an update within 600 days. Dormancy rule

Exact rules: New. The check on applied thresholds version 9.

Every rule outcome, as the check records it
  • expected-interval: version 2.20261016.1 base 300, doubled for 504939 downloads per month: E = 600 days
  • dormancy: last activity 2026-10-17, 23 days ago, E = 600: active
  • new: new: first release 2026-08-04 is under 180 days old

Measurements and sources

Each signal row shows the raw value, its source and the fetch date. The Sources list gathers every distinct source on this page. Movement needs more history.

Sources6 sources
Adoption4 active dependents, 4 new dependents in 365 days1 not measured
Adoption measurements
SignalValueSourceFetchedCounts
Active dependents4crates.io database dumpYes
New dependents (365 days)4crates.io database dumpYes
Peak active dependents3.5crates.io database dumpYes
Active dependents as a share of the peak1crates.io database dumpYes
Binary downloads per releaseNot measuredYes
MaintenanceLatest human commit: 3 months ago3 not measured
Maintenance measurements
SignalValueSourceFetchedCounts
Latest human commit2026-08-05GitHub repositoryYes
Latest release2026-10-17crates.io database dumpYes
First release2026-08-04crates.io database dumpYes
Latest version2.20261016.1crates.io database dumpYes
Days the oldest outside issue waits for a reply0GitHub repositoryYes
Maintainer replies over 365 days0GitHub repositoryYes
Repository archivedfalseGitHub repositoryYes
Repository gonefalseGitHub repositoryYes
Release commit or manifest in the repotruecrates.io packageYes
Published docstruecrates.io database dumpYes
Latest release fails cargo checkNot measuredYes
Build check dateNot measuredYes
Build check toolchainNot measuredYes
SecurityUnpatched vulnerabilities: none
Security measurements
SignalValueSourceFetchedCounts
Unpatched vulnerabilities past the grace windownoneRustSec advisory databaseYes
RustSec unmaintained advisorynoneRustSec advisory databaseYes
Deprecation noticenonecrates.io database dumpYes
Deprecated requirementnonecrates.io database dumpYes
Distribution17 releases, 504,939 downloads over 30 days5 not measured
Distribution measurements
SignalValueSourceFetchedCounts
Releases17crates.io database dumpYes
Downloads over 30 days504939crates.io database dumpYes
Binary downloads over the last 3 releasesNot measuredYes
Homebrew installs on request over 365 daysNot measuredYes
Repology repository familiesNot measuredYes
Docker pull growth over 90 daysNot measuredYes
Docker pulls at the growth baselineNot measuredYes
Every version a prereleasefalsecrates.io database dumpYes
Every version 0.0.xfalsecrates.io database dumpYes
Every version yankedfalsecrates.io database dumpYes
Crate deleted from crates.iofalsecrates.io database dumpYes
Repository4 stars, 1 forks, 2 contributors, 26 commits. Stars alone never qualify a project.
Repository measurements
SignalValueSourceFetchedCounts
GitHub stars4GitHub repositoryYes — one part of the repository route
GitHub forks1GitHub repositoryYes — one part of the repository route
Contributors with a GitHub account2GitHub contributorsYes — one part of the repository route
Commits on the default branch26GitHub repositoryNo
Repository's main crate left out of the entrynonecrates.io database dumpYes
Facts that never count
Facts that never count measurements
SignalValueSourceFetchedCounts
LicenseBSD-3-Clausecrates.io database dumpNo
Runs on, from listed crates and their direct dependenciesGPU support unknowncrates.io database dumpNo
What shows each backendno dependency or feature matches a backend rowcrates.io database dumpNo
Native code, from listed crates and their dependenciesNo native code foundcrates.io database dumpNo
Python bindings, from direct dependencies of listed cratesNo Python binding crate foundcrates.io database dumpNo
Crates that depend on a listed crate directly4crates.io database dumpNo

Corrections and review history

Report a correction

No correction applies to this entry.

Judgments

A judgment enters only through an approved issue and a merged pull request.

No judgment applies to this entry.