instant-segment
App Listed
Fast English word segmentation in Rust
- Downloads
- 0
- Dependents
- Not applicable
- Stars
- 107
- Forks
- 8
- Contributors
- 9
- Commits
- 226
- Latest release
About
Instant Segment: fast English word segmentation in Rust
Instant Segment is a fast Apache-2.0 library for English word segmentation. It is based on the Python wordsegment project written by Grant Jenks, which is in turn based on code from Peter Norvig's chapter Natural Language Corpus Data from the book Beautiful Data (Segaran and Hammerbacher, 2009).
For the microbenchmark included in this repository, Instant Segment is ~500x faster than the Python implementation. The API was carefully constructed so that multiple segmentations can share the underlying state to allow parallel usage.
How it works
Instant Segment works by segmenting a string into words by selecting the splits with the highest probability given a corpus of words and their occurrences.
For instance, provided that choose and spain occur more frequently than
chooses and pain, and that the pair choose spain occurs more frequently
than chooses pain, Instant Segment can help identify the domain
choosespain.com as ChooseSpain.com which more likely matches user intent.
Read about how we built and improved Instant Segment for use in production at Instant Domain Search to help our users find relevant domains they can register.
Read the full README
Using the library
Python (>= 3.9)
pip install instant-segment
Rust
[dependencies]
instant-segment = "0.8.1"
Examples
The following examples expect unigrams and bigrams to exist. See the
examples (Rust,
Python) to see how to construct
these objects.
import instant_segment
segmenter = instant_segment.Segmenter(unigrams, bigrams)
search = instant_segment.Search()
segmenter.segment("instantdomainsearch", search)
print([word for word in search])
--> ['instant', 'domain', 'search']
use instant_segment::{Search, Segmenter};
use std::collections::HashMap;
let segmenter = Segmenter::new(unigrams, bigrams);
let mut search = Search::default();
let words = segmenter
.segment("instantdomainsearch", &mut search)
.unwrap();
println!("{:?}", words.collect::<Vec<&str>>())
--> ["instant", "domain", "search"]
Check out the tests for more thorough examples: Rust, Python
Testing
To run the tests run the following:
cargo t -p instant-segment --all-features
You can also test the Python bindings with:
make test-python
Compatibility
- License
- Apache-2.0
- Crate facts
- Not measured
- Runs on, Native code and Python bindings read crates on crates.io. This entry lists no crate.
Compare with similar projects
Order: strongest tier first, then most downloads, then name, with unmeasured values last. A library counts downloads over 30 days. An app counts binary downloads per release. An app with none counts its crate downloads over 30 days. ML Rust runs this catalogue, so its closest project always gets a row.
| Project | Downloads | Dependents | Stars | Latest release | Runs on (GPU) | Native code | License |
|---|---|---|---|---|---|---|---|
|
Fast English word segmentation in Rust |
0 binary downloads per release | Not applicable | 107 | Not measured | Not measured | Apache-2.0 | |
|
jieba-rs Featured
Jieba Chinese Word Segmentation in Rust |
947,384 downloads over 30 days | 60 | 992 | 0.11.0 | GPU support unknown | No native code found | MIT |
|
whatlang-rs Featured
Fast and lightweight language identification library for Rust. |
882,019 downloads over 30 days | 74 | 1,087 | 0.18.0 | GPU support unknown | No native code found | MIT |
|
whichlang Listed
A blazingly fast and lightweight language detection library for Rust. |
52,880 downloads over 30 days | 11 | 464 | 0.1.1 | GPU support unknown | No native code found | MIT |
|
splintr Listed
Tokenizer for Rust and Python: pure Rust by default, with 18 bundled vocabularies and four backends covering BPE, SentencePiece and WordPiece. Its output matches Hugging Face tokenizers id for id on 14 model families, including GPT-2, Qwen, Whisper, T5 and BERT. Python bindings |
8,204 downloads over 30 days | 7 | 63 | 0.22.2 | GPU support unknown | Binds native code: pcre2-sys (feature pcre2) | MIT |
|
A fast and lightweight Rust library for splitting text into semantically meaningful chunks. |
864 downloads over 30 days | 3 | 4 | 0.1.1 | GPU support unknown | Binds native code: torch-sys (feature rust-bert) | MIT |
|
textstat New
Readability metrics for English text: Flesch Reading Ease, Flesch-Kincaid, Gunning Fog, SMOG, ARI, Coleman-Liau, plus word/sentence/syllable counts. Zero-dependency, no_std. |
95 downloads over 30 days | 2 | 0 | 0.1.1 | GPU support unknown | No native code found | Apache-2.0 OR MIT |
Why ListedNo archive, New or Featured rule places it, so it is Listed.
What it lacks for Featured
- The check holds no value for Homebrew installs on request over 365 days, Repology repository families, Docker pull growth over 90 days. A gate that lacks its signal does not pass.
What else the check found
- The last activity is on 26 Oct 2026, 14 days ago. The rules expect an update within 365 days. Dormancy rule
Not measured: First release, Homebrew installs on request over 365 days, Repology repository families, Docker pull growth over 90 days.
Exact rules: Listed, Featured app. The check on applied thresholds version 9.
Every rule outcome, as the check records it
expected-interval: app: E = 365 daysdormancy: last activity 2026-10-26, 14 days ago, E = 365: activefeatured: not featured: unknown homebrew_on_request_365d, repology_families, docker_pull_growth_90dlisted: listed: no archive, new or featured rule applies
Measurements and sources
Each signal row shows the raw value, its source and the fetch date. The Sources list gathers every distinct source on this page. Movement needs more history.
Sources4 sources
- Description: the GitHub repository description
- GitHub releases, fetched
- GitHub repository, fetched
- GitHub contributors, fetched
Adoption0 binary downloads per release4 not measured
| Signal | Value | Source | Fetched | Counts |
|---|---|---|---|---|
| Active dependents | Not measured | Yes | ||
| New dependents (365 days) | Not measured | Yes | ||
| Peak active dependents | Not measured | Yes | ||
| Active dependents as a share of the peak | Not measured | Yes | ||
| Binary downloads per release | 0 | GitHub releases | Yes |
MaintenanceLatest human commit: 14 days ago7 not measured
| Signal | Value | Source | Fetched | Counts |
|---|---|---|---|---|
| Latest human commit | 2026-10-26 | GitHub repository | Yes | |
| Latest release | 2026-10-26 | GitHub releases | Yes | |
| First release | Not measured | Yes | ||
| Latest version | Not measured | Yes | ||
| Days the oldest outside issue waits for a reply | 0 | GitHub repository | Yes | |
| Maintainer replies over 365 days | 4 | GitHub repository | Yes | |
| Repository archived | false | GitHub repository | Yes | |
| Repository gone | false | GitHub repository | Yes | |
| Release commit or manifest in the repo | Not measured | Yes | ||
| Published docs | Not measured | Yes | ||
| Latest release fails cargo check | Not measured | Yes | ||
| Build check date | Not measured | Yes | ||
| Build check toolchain | Not measured | Yes |
SecurityNot measured
| Signal | Value | Source | Fetched | Counts |
|---|---|---|---|---|
| Unpatched vulnerabilities past the grace window | Not measured | Yes | ||
| RustSec unmaintained advisory | Not measured | Yes | ||
| Deprecation notice | Not measured | Yes | ||
| Deprecated requirement | Not measured | Yes |
Distribution0 binary downloads over the last 3 releases10 not measured
| Signal | Value | Source | Fetched | Counts |
|---|---|---|---|---|
| Releases | Not measured | Yes | ||
| Downloads over 30 days | Not measured | Yes | ||
| Binary downloads over the last 3 releases | 0 | GitHub releases | Yes | |
| Homebrew installs on request over 365 days | Not measured | Yes | ||
| Repology repository families | Not measured | Yes | ||
| Docker pull growth over 90 days | Not measured | Yes | ||
| Docker pulls at the growth baseline | Not measured | Yes | ||
| Every version a prerelease | Not measured | Yes | ||
| Every version 0.0.x | Not measured | Yes | ||
| Every version yanked | Not measured | Yes | ||
| Crate deleted from crates.io | Not measured | Yes |
Repository107 stars, 8 forks, 9 contributors, 226 commits. Stars alone never qualify a project.1 not measured
| Signal | Value | Source | Fetched | Counts |
|---|---|---|---|---|
| GitHub stars | 107 | GitHub repository | Yes — one part of the repository route | |
| GitHub forks | 8 | GitHub repository | Yes — one part of the repository route | |
| Contributors with a GitHub account | 9 | GitHub contributors | Yes — one part of the repository route | |
| Commits on the default branch | 226 | GitHub repository | No | |
| Repository's main crate left out of the entry | Not measured | Yes |
Facts that never count
| Signal | Value | Source | Fetched | Counts |
|---|---|---|---|---|
| License | Apache-2.0 | GitHub repository | No | |
| Runs on, from listed crates and their direct dependencies | Not measured | No | ||
| What shows each backend | Not measured | No | ||
| Native code, from listed crates and their dependencies | Not measured | No | ||
| Python bindings, from direct dependencies of listed crates | Not measured | No | ||
| Crates that depend on a listed crate directly | Not measured | No |
Corrections and review history
No correction applies to this entry.
Judgments
A judgment enters only through an approved issue and a merged pull request.
No judgment applies to this entry.