Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[MRG+1] Add message to IgnoreRequest exception in robotstxt.py #3113

Merged
merged 2 commits into from Feb 8, 2018

Conversation

@WenbinZhang
Copy link
Contributor

@WenbinZhang WenbinZhang commented Feb 7, 2018

Add message to IgnoreRequest exception so that it can be detectedin the errbak method of a spider

Add message to IgnoreRequest exception so that it can be detectedin the errbak method of a spider
@kmike kmike changed the title Update robotstxt.py [MRG+1] Update robotstxt.py Feb 8, 2018
@kmike
Copy link
Member

@kmike kmike commented Feb 8, 2018

Thanks @WenbinZhang, I think that's a good change.

@codecov
Copy link

@codecov codecov bot commented Feb 8, 2018

Codecov Report

Merging #3113 into master will not change coverage.
The diff coverage is 100%.

@@          Coverage Diff           @@
##           master   #3113   +/-   ##
======================================
  Coverage    82.1%   82.1%           
======================================
  Files         228     228           
  Lines        9586    9586           
  Branches     1384    1384           
======================================
  Hits         7871    7871           
  Misses       1456    1456           
  Partials      259     259
Impacted Files Coverage Δ
scrapy/downloadermiddlewares/robotstxt.py 100% <100%> (ø) ⬆️
@dangra dangra merged commit c56f7b3 into scrapy:master Feb 8, 2018
2 checks passed
2 checks passed
codecov/patch 100% of diff hit (target 82.1%)
Details
continuous-integration/travis-ci/pr The Travis CI build passed
Details
@kmike kmike added this to the v1.6 milestone Jul 4, 2018
@kmike kmike changed the title [MRG+1] Update robotstxt.py [MRG+1] Add message to IgnoreRequest exception in robotstxt.py Dec 26, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked issues

Successfully merging this pull request may close these issues.

None yet

4 participants