Skip to content

A part-of-speech tagger NLP model implemented with the Viterbi algorithm with HMMs in Rust.

Notifications You must be signed in to change notification settings

nkaush/pos-tagging

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

18 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

pos-tagging

A Rust implementation of the the Viterbi algorithm for part-of-speech tagging.

Usage

The executable this project produces has the capability to...

  • train a model, saves it to a file for future use, and optionally evaluates the model on some data
  • evaluate a pre-trained model on some data
  • predict the POS tagging of some sentnces using a pre-trained model either from standard input or a file

These functionalities correspond to the train, evaluate, and predict subcommands, respectively. The following blocks indicate how to use each subcommand.

A Rust implementation of the the Viterbi algorithm for part-of-speech tagging.

Usage: pos-tagger <COMMAND>

Commands:
  train     Trains a model, saves it to a file for future use, and optionally evaluates the model on some data
  evaluate  Evaluate a pre-trained model on some data
  predict   Predict the POS tagging of some sentnces using a pre-trained model either from standard input or from a file
  help      Print this message or the help of the given subcommand(s)

Options:
  -h, --help     Print help
  -V, --version  Print version

Train

Trains a model, saves it to a file for future use, and optionally evaluates the model on some data

Usage: pos-tagger train [OPTIONS] -d <DATA_FILES> -o <OUT_FILE>

Options:
  -d <DATA_FILES>      Paths to all of the data files used to train the model
  -o <OUT_FILE>        The path to save the trained model to
  -e <EVAL_FILE>       The path to a data file to evaluate the model

Evaluate

Evaluate a pre-trained model on some data

Usage: pos-tagger evaluate -m <MODEL_FILE> -e <EVAL_FILE>

Options:
  -m <MODEL_FILE>      The path to the saved pre-trained model
  -e <EVAL_FILE>       The path to a data file to evaluate the model

Predict

Predict the POS tagging of some sentnces using a pre-trained model either from standard input or from a file

Usage: pos-tagger predict [OPTIONS] -m <MODEL_FILE>

Options:
  -m <MODEL_FILE>        The path to the saved pre-trained model
  -p <PREDICT_FILE>      The path to a data file of sentences to predict with. Defaults to STDIN if not specified

About

A part-of-speech tagger NLP model implemented with the Viterbi algorithm with HMMs in Rust.

Topics

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages