Skip to content

jsbaartz/web-crawler

Repository files navigation

Web Crawler

A Python-based asynchronous web crawler that:

  • Crawls a website starting from a base URL
  • Extracts headings, paragraphs, links, and images
  • Uses asyncio + aiohttp for concurrency
  • Outputs results to a structured JSON file

Usage

uv run main.py <URL> <max_concurrency> <max_pages>

About

No description, website, or topics provided.

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

 
 
 

Contributors

Languages