LifeHack

SecureGPT

Data sources used

1. https://www.kaggle.com/datasets/rmisra/news-category-dataset/data
2. https://pypi.org/project/bbc-news/
3. https://www.gdeltproject.org/

Final filtered unique count

• 1775

Content extractor

• https://apify.com/lukaskrivka/article-extractor-smart This was used as it would save time with the content pre-processing such as removing special characters and splitting words, which are both common operations in NLP but not withstanding this, the preprocessing pipeline for NLP still remains complex. For example: https://www.kaggle.com/code/gunesevitan/nlp-with-disaster-tweets-eda-cleaning-and-bert?scriptVersionId=28164619&cellId=30

Final scrapable content after context extraction

• 1491

References

https://medium.com/@maleeshadesilva21/preprocessing-steps-for-natural-language-processing-nlp-a-beginners-guide-d6d9bf7689c9