The Internet hosts perhaps the greatest source of information—and misinformation—on the planet. Many disciplines, such as data science, business intelligence, and investigative reporting, can benefit enormously from collecting and analyzing data from websites.
Collecting data from websites using an automated process is known as web scraping. Some websites explicitly forbid users from scraping their data with automated tools like the ones you’ll create in this tutorial.
Important: Before using your Python skills for web scraping, you should always check your target website’s acceptable use policy to see if accessing the website with automated tools is a violation of its terms of use.
This code snippest demonstrates the use of scraping technique to extract and parse raw data from a HTTP request response received from google when we make a query for weather data of an specific city.
The following list describes the different sections of this code.
-
Define the end point url and header.
-
Make the request using HTTP GET method.
-
Print out Today weather data.
-
Using PrettyTable module to create a table.
-
Checking for exceptions.
-
Code output:
