Skip to content

Web Scrapping data from worldometer website and cleaning the data using python libraries and storing them into RDBMS (SQL Server). Performing data analysis over the datasets to get insights about the data

Notifications You must be signed in to change notification settings

jayshilj/COVID-19_Web_Scrapping_and_Analysis

Repository files navigation

COVID-19_Web_Scrapping_and_Analysis

COVID-19_Analysis on World and US State Data

This project deals with 4 diffrent aspects on COVID-19 Analysis

  1. Web Scraping: Webscrapping was performed on the website https://www.worldometers.info/coronavirus/ Data for all the World Countries was extracted and cleaned Data for all the USA states was extracted and cleaned Data Visualizations were performed on the dataset.

  2. Using officially released data to perform analysis over course of the inception of the pandemic The data was collected and analysed using multiple visualization packages in python Dynamic dashboards were created and hosted on site in the process using the dstack.ai The link to the dashboards 'https://dstack.ai/jayshil97'

  3. Data Collection in Relational Database (RDBMS) The data which was web scrapped was collected and stored in relational database (SQL Server) A task sheduler was created to run the script daily to collect data

  4. Tableau Analysis: I have created tableau Analysis using the dataset link to the Tableau Public is:

https://public.tableau.com/profile/jayshil.jain#!/vizhome/COVIDDashboard_15922437933840/COVIDDashboard https://public.tableau.com/profile/jayshil.jain#!/vizhome/COVIDTimeSeries_15886643195770/COVID-19Analysis

File Details:

  1. Images - Stores the images generated in the project
  2. Analysis_for_COVID_19.html - HTML page to show all the dynamic visualizations generated by plotly
  3. Analysis_for_COVID_19.ipnb - The code for all the visualizations and dstack integration (Worldwide)
  4. COVID-19 World Data Web Scrapping.ipynb - The code for all the Web Scrapping and RDBMS integration of all the world countries
  5. COVID19_USASTATES_WebScraping.ipynb - The code for all Web Scrapping and RDBMS integration of all the US States

Some Analysis:

Refrence:

https://www.quandl.com

https://www.worldometers.info/coronavirus/

https://raw.githubusercontent.com/CSSEGISandData/

A special thanks to prof Nicholas Brown for his expert assistance and guidence.

https://github.com/nikbearbrown

About

Web Scrapping data from worldometer website and cleaning the data using python libraries and storing them into RDBMS (SQL Server). Performing data analysis over the datasets to get insights about the data

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published