rust-stop-words

Language is the preferred language enum; the original LANGUAGE enum is deprecated since 0.10.1 and remains available for compatibility, including direct variant imports. Replace LANGUAGE with Language to migrate. They are distinct types accepted by both lookup functions. The existing get function is retained and panics for unknown or disabled languages. Call available_languages() to discover the codes enabled in a particular build.

For ISO languages, a lookup code is usually a two-letter ISO 639-1 code. Fictional constructed languages use three-letter codes (dot, dov, nav, qya, sjn, tlh, val), and NLTK-specific hinglish is supported when the nltk feature is enabled.

You can find a complete example of reading a text file and removing stop words here.

Features

Features are additive. Enabling constructed, for example, does not remove ISO or NLTK enum variants. When both iso and nltk provide a language, the NLTK list takes precedence, preserving the crate's existing behavior.

The crate continues to use the Rust 2021 edition.

Membership and customization

The crate returns static slices without allocating. For repeated membership checks or domain-specific changes, collect the slice into the set type your application needs:

use std::collections::HashSet;
use stop_words::lookup;

let Some(english) = lookup("en") else {
    return;
};
let mut words: HashSet<&str> = english.iter().copied().collect();
words.remove("computer"); // Preserve a meaningful domain term.
words.insert("project");  // Add an application-specific stop word.

assert!(!words.contains("computer"));
assert!(words.contains("project"));

Entries are returned as supplied by their source dataset. Normalize input and list entries consistently for case-insensitive matching; the crate does not impose language-dependent normalization.

Natural Language Availability

This crate supports all languages from Stopwords ISO and also from NLTK. Expand the table below to see a comprehensive description.

Language Coverage Table

NLTK also includes a non-ISO list for hinglish (lookup key: "hinglish") when the nltk feature is enabled.

Fictional Constructed Language Availability

The optional constructed feature provides small experimental lists for seven fictional languages. These lists were initially generated with ChatGPT in March 2023 rather than derived from authoritative corpora. Treat them as approximate starting points, not linguistically validated datasets. Source-backed corrections and corpus-derived replacements are welcome.

Language Coverage Table