issues Search Results · repo:rmax/scrapy-redis language:Python
Filter by
197 results
(55 ms)197 results
inrmax/scrapy-redis (press backspace or delete to remove)import asyncio
class RedisMixin:
...
async def start(self):
Asynchronous start method compatible with new Scrapy versions.
for req in await asyncio.to_thread(self._collect_start_requests): ...
lexxai
- Opened 28 days ago
- #305
Is Valkey support being considered as an alternative to Redis in the future?
abebus
- Opened on May 10
- #304
Problem:
File .../lib/python3.11/site-packages/pydispatch/robustapply.py , line 55, in robustApply
return receiver(*arguments, **named)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File .../lib/python3.11/site-packages/scrapy/extensions/logstats.py ...
lexxai
- Opened on Feb 10
- #303
Description
Today, I use web scraping and LLM models to extract data from URLs. If, for some reason, I encounter an error with the
LLM, my plan is to retry processing during the next scheduled run (I ...
matheus-rossi
- 5
- Opened on Nov 18, 2024
- #302
Description
AttributeError: BossSpider object has no attribute settings
scrapy v2.11.2 scrapy-redis v0.9.1 python v3.12
Step to Reproduce
`from scrapy import signals import scrapy from DrissionPage._pages.chromium_page ...
gaye746560359
- Opened on Sep 29, 2024
- #301
https://github.com/rmax/scrapy-redis/blob/48a7a8921ae064fe7b4202b130f1054ede9103d6/src/scrapy_redis/scheduler.py#L136
calls from_spider in a dupefilter class.
However, the from_spider ONLY implements ...
HairlessVillager
- Opened on Jun 22, 2024
- #293
Description
Let me introduce my python program: First, I get the data I need to crawl from MySQL and then process the data into a
URL through image Submit to redis and wait for crawling. If crawling fails, ...
Hao1617
- 7
- Opened on May 9, 2024
- #291
Description
I ve been using Scrapy for web scraping tasks and frequently pass additional information using response.meta for further
processing within the spider. My current project requires integrating ...
SteliosGiannatos
- 1
- Opened on Mar 19, 2024
- #290
I encountered a new problem。 Program error interrupt, I am using scrapy==2.11.1, scrapy-redis==0.7.3
image
In addition, since the latest package of scrapy-redis that is compatible with the new version ...
georgeJzzz
- 13
- Opened on Feb 23, 2024
- #289
I suggest adding a feature to Scrapy Redis for seamless integration with Kafka, enabling users to publish crawled items
directly to a Kafka topic. I m willing to implement this feature and welcome feedback ...
hamed21q
- 1
- Opened on Jan 18, 2024
- #288

Learn how you can use GitHub Issues to plan and track your work.
Save views for sprints, backlogs, teams, or releases. Rank, sort, and filter issues to suit the occasion. The possibilities are endless.Learn more about GitHub IssuesProTip!
Press the /
key to activate the search input again and adjust your query.
Learn how you can use GitHub Issues to plan and track your work.
Save views for sprints, backlogs, teams, or releases. Rank, sort, and filter issues to suit the occasion. The possibilities are endless.Learn more about GitHub IssuesProTip!
Press the /
key to activate the search input again and adjust your query.