webscraper

I wanted to build my very own webscraper in Python using the Beautiful Soup library.

In this exercise, I wanted to:

• Be able to scrape from different sources.
• Have the flexibility to easily add and remove sources.
• Reuse code where possible, only writing custom parsing for each source.

Considerations:

• Note that some bot detections such as Imperva will detect certain scraping frameworks.