We read every piece of feedback, and take your input very seriously.
To see all available qualifiers, see our documentation.
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
When duplicate requests found, it only record the first duplicate url, no more message.
in file dupfilter.py function log, msg is overriden...
dupfilter.py
log
msg
def log(self, request, spider): """Logs given request. Parameters ---------- request : scrapy.http.Request spider : scrapy.spiders.Spider """ if self.debug: msg = "Filtered duplicate request: %(request)s" self.logger.debug(msg, {'request': request}, extra={'spider': spider}) elif self.logdupes: msg = ("Filtered duplicate request %(request)s" " - no more duplicates will be shown" " (see DUPEFILTER_DEBUG to show all duplicates)") msg = "Filtered duplicate request: %(request)s" self.logger.debug(msg, {'request': request}, extra={'spider': spider}) self.logdupes = False
The text was updated successfully, but these errors were encountered:
You are right!
Sorry, something went wrong.
e4a42dd
BUG Don't override message about duplicates not being displayed after…
fa203e8
…wards. Closes GH-75.
No branches or pull requests
When duplicate requests found, it only record the first duplicate url, no more message.
in file
dupfilter.py
functionlog
,msg
is overriden...The text was updated successfully, but these errors were encountered: