This project focuses on the use of big data platforms, specifically Spark (PySpark, SparkML, Spark NLP). We will use the comment text a user posted, categorize the sentiment and predict scores of each comment. Our objective is to understand the dynamics of the Reddit online community and how the way people communicate online leads to different r…

Twitter based stock market prediction using Pyspark, project for Big Data PW 2020L

Text summarization algorithms using PySpark

This repository contains application code for the Wizeline Data Engineering Bootcamp (DEB) 2023. It is one of two repositories for the DEB. The other houses the infrastructure code.

Web application to detect emotion in text

Completed professional certificates and specializations via Coursera

This repository provides examples of using pre-trained BERT models from SparkNLP with PySpark for Natural Language Processing task.

Media diversity shapes perspectives, yet biased news distorts reality, fostering misinformation. 'Political Ideologies Prediction in News Articles' aims to forecast bias using PySpark, NLP, and ML for adaptable, swift inference. Integrated with NYT API, it predicts bias in top political articles, fostering better understanding of subjective content

Drawing a Company Ecosystem Graph

Media diversity shapes perspectives, yet biased news distorts reality, fostering misinformation. 'Political Ideologies Prediction in News Articles' aims to forecast bias using PySpark, NLP, and ML for adaptable, swift inference. Integrated with NYT API, it predicts bias in top political articles, fostering better understanding of subjective content

Automated Extraction of Medical Risk Factors For Life Insurance Underwriting

Micro project on big data technologies via spark

To ensure ongoing drug safety, pharma companies need to monitor and report adverse drug events post-market launch. This accelerator extracts, processes and analyzes adverse drug events from real-world text data using NLP

Build a lakehouse for all your gamer data and use natural language processing techniques to flag questionable comments for moderation.

Generate oncology insights from real-world data using NLP. Once extracted, oncology data is enriched with useful information like ICD-10 codes and used to build powerful visualizations

Our joint Solution Accelerator with John Snow Labs automates the detection of sensitive information contained within unstructured data using NLP models for healthcare. Extracted data is stored within the Lakehouse, where teams can use the pre-trained models to easily remove, obfuscate or mask data for downstream analytics at massive scale.

My notebook on using Python with Jupyter Notebook, PySpark etc

Databricks and John Snow Labs Solution Accelerator for Medicare Risk Adjustment automates the extraction of undiagnosed member conditions from unstructured clinical notes with NLP models, improving downstream reimbursements.