Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Additional typing for scraper and a small code change. #5100

Merged
merged 2 commits into from
Apr 14, 2021

Conversation

wRAR
Copy link
Member

@wRAR wRAR commented Apr 14, 2021

This was left out of the previous PR for several reasons.

This removes a check from _log_download_errors, I believe this check can't be False.

@codecov
Copy link

codecov bot commented Apr 14, 2021

Codecov Report

Merging #5100 (ba0c394) into master (9bf9ab7) will decrease coverage by 0.02%.
The diff coverage is 90.00%.

❗ Current head ba0c394 differs from pull request most recent head 309a637. Consider uploading reports for the commit 309a637 to get more accurate results

@@            Coverage Diff             @@
##           master    #5100      +/-   ##
==========================================
- Coverage   88.28%   88.25%   -0.03%     
==========================================
  Files         162      162              
  Lines       10364    10374      +10     
  Branches     1504     1506       +2     
==========================================
+ Hits         9150     9156       +6     
- Misses        946      948       +2     
- Partials      268      270       +2     
Impacted Files Coverage Δ
scrapy/core/scraper.py 87.86% <90.00%> (-1.71%) ⬇️

scrapy/core/scraper.py Outdated Show resolved Hide resolved
scrapy/core/scraper.py Outdated Show resolved Hide resolved

def _itemproc_finished(self, output, item, response, spider):
def _itemproc_finished(self, output: Any, item: Any, response: Response, spider: Spider) -> None:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don’t think it needs to be done in this pull request, but I’m thinking that it may be good to have a type alias for all itemadapter-supported item types, and use it in scenarios like this one instead of Any. It probably makes sense to implement that type alias in itemadapter, though.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, assuming it's possible to express.

@wRAR wRAR merged commit 06f3d12 into scrapy:master Apr 14, 2021
@wRAR wRAR deleted the more-scraper-typing branch May 8, 2023 10:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants