End-To-End-Data-Cleaning-Mechanisms

Breif History

In my experience, i have encountered a lot of time put into cleaning data especially with NLP related problems. So i thought to build a generalized data cleaning mechanisms with which we can directly plug-in the methods for preprocessing and directly building various models or we can select required methods, customize and use them in your own code.

Cleaning Mechanisms

The Mechanisms that are included in cleaningmechanism.py.

1. Word/Phrase Embeddings Cleaner
2. Named Entity Recognition Cleaner
3. Text Summarization Cleaner
4. Text Generation Cleaner

Future Scope

Pending Topics/Methods To Cover.

1. Spelling Correction
2. Join Split Words
3. Split Join Words
4. Grammer Correction
5. Normalize Slang Words