PyQt application to demonstrate the Kneser-Ney smoothing algorithm for bigram/word prediction.
-
Updated
Sep 29, 2017 - Python
PyQt application to demonstrate the Kneser-Ney smoothing algorithm for bigram/word prediction.
Language Modeling using ngrams and Kneser-Ney Smoothing
Interpolated Kneser-Ney smoothing with an out-of-vocabulary correction and discount estimated from training data
This repository implements N-gram language modeling with Kneser-Kney and Witten Bell smoothing techniques, including an in-house tokenizer. It also features a neural model with LSTM architecture and calculates perplexities for comparing language and neural models.
Add a description, image, and links to the kneser-ney-smoothing topic page so that developers can more easily learn about it.
To associate your repository with the kneser-ney-smoothing topic, visit your repo's landing page and select "manage topics."