HydroSentry is a system designed to monitor the quality and condition of water pipe infrastructure, with a focus on identifying potential pipe failures before they occur.
The project utilizes data from the Water Point Data Exchange (WPdx), which can be accessed here: https://www.waterpointdata.org
Specifically, we use the following dataset: https://data.waterpointdata.org/dataset/Water-Point-Data-Exchange-Plus-WPdx-/eqje-vguj/about_data
HydroSentry incorporates a machine learning model trained on the above data to analyze patterns in water pipe conditions, usage, and failure history. Using these insights, the model predicts when pipes are likely to require maintenance or repair, which allows repairs to occur before pipes break and therefore take away access to water for some populations.
We first filtered through the several columns which the WPdx-Plus had (we used plus as it was precleaned to some extent) determining which ones could be important for a pipe's longevity. As a result we managed to narrow down the total of 73 columns to 10. From there we did even further narrowing, manipulating certain datasets such as report date and installment year to get years of service, and dropped report date and so on.
We also ended up cleaning the data by removing any NaNs, grouping together duplicates we thought were redundant, ensuring binary values for datapoints we needed the model to have its scope as (not functional and functional), etc.
Once the model was trained, it was saved with its encoder and integrated into the backend to make predictions on new data. When new water pipe data is provided as inputs on the frontend, the model then analyzes the input and predicts when the water point will need maintenance.
The application was implemented using Django. The backend is written in Python and is responsible for loading the trained model and handling prediction requests.
When a user submits data into the HTML frontend, the backend processes the input, runs it through the model, and returns a prediction to be displayed to the frontend.
We learned a lot about different Machine Learning Models and the pipeline of turning it into an actual product. One of us didn't know at all about ML before this whilst the other two knew some of it but not expansive knowledge. However, as a result we all ended up getting stuck once we realized how far out of scope this all was. We needed models different than what we were used to and using different code which we had to learn. Furthermore, even though we may have had some experience with either full stack or making a model somewhat, we never had experience in the full pipeline of training the model, as a result we ran into several issues; we didn't know how to actually get the model after it was done training, we didn't know how to then make that interact with the frontend as there was no obvious API to just use, we even ran into several issues trying to refactor the model and figuring out which model to use as the ones which we were familiar with using such as random forest or k nearest neighbours and so required a lot of research and failures. Nonetheless, we managed to go against the odds and learn how to do these things within the 36 hour period of bitcamp and produce a working model and product.
We are hoping to continue developing and refining the model and then perhaps making it useful enough for NGOs or other major coproations to use it and improve existing infrastructure making it such that communities are less impacted by water pipe infrastructure issues.