You will create a function that takes a website address and a number of webpages as input arguments and then goes all over the website saving every news article on the page to a separate .txt file on your computer.
https://hyperskill.org/projects/145
scraper.py
Stage #1: Wanna Talk to the Internet?
Send an HTTP request, process the results, and learn about response status codes.
Stage #2: The Beautiful Soup Ingredients
Get familiar with the BeautifulSoup library. Learn how to parse simple data from a webpage.
Stage #3: Whats the File?
Learn to work with files.
Stage #4: The Soup is Real
Create a function that scrapes every article on a page with a for loop.
Stage #5: Soup, Sweet Soup
Improve the function by adding ‘webpage number’ and 'article type' parameters.