This project involves ETL and a Machine Learning Pipeline implementation, that classifies messages related to disaster response, and covering multiple language disasters, with a disaster emergency response using a Flask Web App.
Motivation
The classifier model is built using Extract, Transform and Load process(ETL), natural language processing(NLP) and machine learning pipeline for classifying disaster messages. The project also includes a web app where an emergency worker can input a new message and get classification results in several categories. It can be useful to detect what messages actually need attention during the event of a disaster.
Dataset
The dataset contains around 30,000 messages drawn from events including an earthquake in Haiti in 2010, an earthquake in Chile in 2010, floods in Pakistan in 2010, super-storm Sandy in the U.S.A. in 2012, and news articles spanning a large number of years and 100s of different disasters.
The data has been encoded with 36 different categories related to disaster response and has been stripped of messages with sensitive information in their entirety.
Disaster response messages dataset consists of imbalanced category labels data. Some labels like aid-related, weather-related have much more examples as compared to other categories. This imbalance might affect the model training as the classes are not represented equally. It can be handled by resampling the dataset or by generating synthetic samples. Although I have not applied these methods for now but I am planning to do it in future.
✅Test Different Models and find the best model out of it
✅Train the model using Intel oneDAL to get better results and faster computation(Intel oneAPI Data Analytics Library (oneDAL))
What we learnt
✅Building application using intel oneDAL:The Intel oneAPI Data Analytics Library (oneDAL) contributes to the acceleration of big data analysis by providing highly optimised algorithmic building blocks for all phases of data analytics (preprocessing, transformation, analysis, modelling, validation, and decision making) in batch, online, and distributed processing modes of computation.The library optimizes data ingestion along with algorithmic computation to increase throughput and scalability.
✅Understanding of the data: You would have learned how to preprocess and clean the data, as well as how to handle missing values and categorical variables. You may also have conducted exploratory data analysis to gain insights into the relationships between the variables.
✅Selection of appropriate algorithms: You would have learned how to select appropriate machine learning algorithms for the given problem. For example, logistic regression may be useful for binary classification problems, while decision trees may be better suited for multiclass problems.
✅Machine Learning: I likely learned about different machine learning algorithms and how they can be applied to predict cardiovascular disease and make recommendations for patients.
✅Data Analysis: I likely gained experience in collecting and analyzing large amounts of data, including historical data, to train our machine learning models.
✅Comparison of model performance: You would have learned how to compare the performance of different models using appropriate statistical tests or visualizations. This can help you choose the best model for the given problem.
✅Collaboration: Building a project like this likely required collaboration with a team of experts in various fields, machine learning, and data analysis, and I likely learned the importance of working together to achieve common goals.