Web Scraping

The list of tools, programming libraries and APIs used in web-scraping.

• Python
• PHP
• Ruby
• JavaScript
• Golang
  • Feel free to add your favourite language. Use new_language_template.md as start point.
• Proxy Services
• Web Services
• Console tools
• Books

Other Awesome List Projects

• lists - List of useful, silly and awesome lists curated on GitHub
• HeadlessBrowsers - a list of (almost) all headless web browsers in existence

Contributing

Make this list better! Your contributions are always welcome! See contributing how-to

Credits

This list partially contains data from these sources:

• awesome-python by vinta / CC BY 4.0
• awesome-php by ziadoz
• awesome-ruby by markets
• ruby-nlp by diasks2
• awesome-javascript by sorrycc