-
Notifications
You must be signed in to change notification settings - Fork 10.5k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[MRG+1] Two small fixes for when using the parse command and the '-r' flag (rules). #2225
Conversation
1. Use default "parse" as callback when the matching rule has no callback. 2. Log error and return when no rule matches the parsed url.
Current coverage is 83.34% (diff: 0.00%)@@ master #2225 diff @@
==========================================
Files 161 161
Lines 8699 8705 +6
Methods 0 0
Messages 0 0
Branches 1278 1279 +1
==========================================
+ Hits 7252 7255 +3
- Misses 1199 1202 +3
Partials 248 248
|
I'm not a |
Thanks @Tethik ! |
@redapple Sounds good, I'll take a look at the your pull with the tests when I next get the time. |
Add tests for crawl command non-default cases
… for logged errors.
Tests look good to me. I couldn't think of a missing case either. One minor change is that I moved them to a separate file instead (test_commands.py -> test_command_parse.py). I assume codecov doesn't detect them because the tests are generating the spider code on the fly? Anyhow, I don't have much more input on this. Feel free to merge if you think it looks ok. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm not sure why Codecov is complaining either.
LGTM.
@kmike , do you (still) agree with the change? |
Thanks @Tethik! |
The first error was encountered when running something similar to this:
scrapy parse http://www.example.com/blah/ -r
In a project with a CrawlSpider with two rules. The matching rule did not have a callback defined, resulting in the following error when the command tries to fetch the function.