Skip to content

Spell Checker with GRU and Attention mechanism using Pytorch

License

Notifications You must be signed in to change notification settings

gaushh/Deep-Spelling

Repository files navigation

Sequence To Sequence GRU for automatic Spelling Correction.

This repository contains code for an auto spell checker built using Pytorch as framework.

We've built encoder-decoder networks using GRU as the building block. Luong Attention mechanism is used to speeden up the training process and improve accuracy.

Characters (not words) were provided as input to the architecture while training and testing.
The model was trained on free cloud GPU available on google colab.

We used a billion word datasetreleased by Google. Artificial noise was injected generate spelling errors so as to train the model. The noise is the simulated spelling mistakes and the model tries to learn how to correct the input by comparing the output to the original text. The dataset can be found here

The trained weights file is available here

The model achieved 92% Test set accuracy on training on free Colab GPU within a few hours

Major Modifications

  • Used GRU instead of RNN as building block.
  • Introduced Attention Mechanism

Acknowledgment

This work is an extension to this repo by Tal Weiss.

Feel free to contact me in case of any query.

About

Spell Checker with GRU and Attention mechanism using Pytorch

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published