A series of distributed components for Scrapy. Including RabbitMQ-based components, Kafka-based components, and RedisBloom-based components for Scrapy.
python
redis
crawler
kafka
spider
rabbitmq
scraping
crawling
scrapy
distributed-spider
redisbloom
rabbitmq-pipeline
-
Updated
Apr 23, 2023 - Python