Additional typing for scraper and a small code change. #5100
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Codecov Report
@@ Coverage Diff @@
## master #5100 +/- ##
==========================================
- Coverage 88.28% 88.25% -0.03%
==========================================
Files 162 162
Lines 10364 10374 +10
Branches 1504 1506 +2
==========================================
+ Hits 9150 9156 +6
- Misses 946 948 +2
- Partials 268 270 +2
|
Gallaecio
approved these changes
Apr 14, 2021
|
|
||
| def _itemproc_finished(self, output, item, response, spider): | ||
| def _itemproc_finished(self, output: Any, item: Any, response: Response, spider: Spider) -> None: |
There was a problem hiding this comment.
I don’t think it needs to be done in this pull request, but I’m thinking that it may be good to have a type alias for all itemadapter-supported item types, and use it in scenarios like this one instead of Any. It probably makes sense to implement that type alias in itemadapter, though.
There was a problem hiding this comment.
Yeah, assuming it's possible to express.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This was left out of the previous PR for several reasons.
This removes a check from
_log_download_errors, I believe this check can't be False.