keyword-extraction-rs

Default features: ["tf_idf", "rake", "text_rank"]. By default all algorithms apart from "co_occurrence" and "yake" are enabled.

NOTE: "parallel" feature is only recommended for large documents, it exchanges memory for computation resourses.

Examples

For the stop words, you can use the stop-words crate:

[dependencies]
stop-words = "0.8.0"

For example for english:

use stop_words::{get, LANGUAGE};

fn main() {
    let stop_words = get(LANGUAGE::English);
    let punctuation: Vec<String> =[
        ".", ",", ":", ";", "!", "?", "(", ")", "[", "]", "{", "}", "\"", "'",
    ].iter().map(|s| s.to_string()).collect();
    // ...
}

TF-IDF

Create a TfIdfParams enum which can be one of the following:

1. Unprocessed Documents: TfIdfParams::UnprocessedDocuments;
2. Processed Documents: TfIdfParams::ProcessedDocuments;
3. Single Unprocessed Document/Text block: TfIdfParams::TextBlock;

use keyword_extraction::tf_idf::{TfIdf, TfIdfParams};

fn main() {
    // ... stop_words & punctuation
    let documents: Vec<String> = vec![
        "This is a test document.".to_string(),
        "This is another test document.".to_string(),
        "This is a third test document.".to_string(),
    ];

    let params = TfIdfParams::UnprocessedDocuments(&documents, &stop_words, Some(&punctuation));

    let tf_idf = TfIdf::new(params);
    let ranked_keywords: Vec<String> = tf_idf.get_ranked_words(10);
    let ranked_keywords_scores: Vec<(String, f32)> = tf_idf.get_ranked_word_scores(10);

    // ...
}

RAKE

Create a RakeParams enum which can be one of the following:

1. With defaults: RakeParams::WithDefaults;
2. With defaults and phrase length (phrase window size limit): RakeParams::WithDefaultsAndPhraseLength;
3. All: RakeParams::All;

use keyword_extraction::rake::{Rake, RakeParams};

fn main() {
    // ... stop_words
    let text = r#"
        This is a test document.
        This is another test document.
        This is a third test document.
    "#;

    let rake = Rake::new(RakeParams::WithDefaults(text, &stop_words));
    let ranked_keywords: Vec<String> = rake.get_ranked_words(10);
    let ranked_keywords_scores: Vec<(String, f32)> = rake.get_ranked_word_scores(10);

    // ...
}

TextRank

Create a TextRankParams enum which can be one of the following:

1. With defaults: TextRankParams::WithDefaults;
2. With defaults and phrase length (phrase window size limit): TextRankParams::WithDefaultsAndPhraseLength;
3. All: TextRankParams::All;

use keyword_extraction::text_rank::{TextRank, TextRankParams};

fn main() {
    // ... stop_words
    let text = r#"
        This is a test document.
        This is another test document.
        This is a third test document.
    "#;

    let text_rank = TextRank::new(TextRankParams::WithDefaults(text, &stop_words));
    let ranked_keywords: Vec<String> = text_rank.get_ranked_words(10);
    let ranked_keywords_scores: Vec<(String, f32)> = text_rank.get_ranked_word_scores(10);
}

YAKE

Create a YakeParams enum which can be one of the following:

1. With defaults: YakeParams::WithDefaults;
2. All: YakeParams::All;

use keyword_extraction::yake::{Yake, YakeParams};

fn main() {
    // ... stop_words
    let text = r#"
        This is a test document.
        This is another test document.
        This is a third test document.
    "#;

    let yake = Yake::new(YakeParams::WithDefaults(text, &stop_words));
    let ranked_keywords: Vec<String> = yake.get_ranked_keywords(10);
    let ranked_keywords_scores: Vec<(String, f32)> = yake.get_ranked_keyword_scores(10);
    // ...
}

Contributing

I would love your input! I want to make contributing to this project as easy and transparent as possible, please read the CONTRIBUTING.md file for details.

License

This project is licensed under the GNU Lesser General Public License v3.0. See the Copying and Copying Lesser files for details.