issues Search Results · repo:rmax/scrapy-redis language:Python
Filter by
198 results
(95 ms)198 results
inrmax/scrapy-redis (press backspace or delete to remove)For example: In scrapy.Reques(),I can do this: scrapy.Request(url= https://exaple.com , errback=self.errback_close_page)
But in scrapyredis,it s not work { url : https://exaple.com , : meta : { allowed_depth ...
ynkmyz233
- Opened 7 days ago
- #306
import asyncio
class RedisMixin:
...
async def start(self):
Asynchronous start method compatible with new Scrapy versions.
for req in await asyncio.to_thread(self._collect_start_requests): ...
lexxai
- Opened on Jun 17
- #305
Is Valkey support being considered as an alternative to Redis in the future?
abebus
- Opened on May 10
- #304
Problem:
File .../lib/python3.11/site-packages/pydispatch/robustapply.py , line 55, in robustApply
return receiver(*arguments, **named)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File .../lib/python3.11/site-packages/scrapy/extensions/logstats.py ...
- Opened on Feb 10
- #303
Description
Today, I use web scraping and LLM models to extract data from URLs. If, for some reason, I encounter an error with the
LLM, my plan is to retry processing during the next scheduled run (I ...
matheus-rossi
- 5
- Opened on Nov 18, 2024
- #302
Description
AttributeError: BossSpider object has no attribute settings
scrapy v2.11.2 scrapy-redis v0.9.1 python v3.12
Step to Reproduce
`from scrapy import signals import scrapy from DrissionPage._pages.chromium_page ...
gaye746560359
- Opened on Sep 29, 2024
- #301
https://github.com/rmax/scrapy-redis/blob/48a7a8921ae064fe7b4202b130f1054ede9103d6/src/scrapy_redis/scheduler.py#L136
calls from_spider in a dupefilter class.
However, the from_spider ONLY implements ...
HairlessVillager
- Opened on Jun 22, 2024
- #293
Description
Let me introduce my python program: First, I get the data I need to crawl from MySQL and then process the data into a
URL through image Submit to redis and wait for crawling. If crawling fails, ...
Hao1617
- 7
- Opened on May 9, 2024
- #291
Description
I ve been using Scrapy for web scraping tasks and frequently pass additional information using response.meta for further
processing within the spider. My current project requires integrating ...
SteliosGiannatos
- 1
- Opened on Mar 19, 2024
- #290
I encountered a new problem。 Program error interrupt, I am using scrapy==2.11.1, scrapy-redis==0.7.3
image
In addition, since the latest package of scrapy-redis that is compatible with the new version ...
georgeJzzz
- 13
- Opened on Feb 23, 2024
- #289
Learn how you can use GitHub Issues to plan and track your work.
Save views for sprints, backlogs, teams, or releases. Rank, sort, and filter issues to suit the occasion. The possibilities are endless.Learn more about GitHub IssuesProTip!
Restrict your search to the title by using the in:title qualifier.Learn how you can use GitHub Issues to plan and track your work.
Save views for sprints, backlogs, teams, or releases. Rank, sort, and filter issues to suit the occasion. The possibilities are endless.Learn more about GitHub IssuesProTip!
Restrict your search to the title by using the in:title qualifier.