Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Implement a close reason for robots.txt affecting all start requests #6164

Open
wants to merge 3 commits into
base: master
Choose a base branch
from

New signals: start_request_returned, start_requests_exhausted

e11a6fb
Select commit
Failed to load commit list.
Open

Implement a close reason for robots.txt affecting all start requests #6164

New signals: start_request_returned, start_requests_exhausted
e11a6fb
Select commit
Failed to load commit list.
Codecov / codecov/patch failed Dec 14, 2023 in 0s

34.04% of diff hit (target 88.59%)

View this Pull Request on Codecov

34.04% of diff hit (target 88.59%)

Annotations

Check warning on line 201 in scrapy/core/engine.py

See this annotation in the file changed.

@codecov codecov / codecov/patch

scrapy/core/engine.py#L201 

Added line #L201 was not covered by tests

Check warning on line 89 in scrapy/downloadermiddlewares/robotstxt.py

See this annotation in the file changed.

@codecov codecov / codecov/patch

scrapy/downloadermiddlewares/robotstxt.py#L88-L89 

Added lines #L88 - L89 were not covered by tests

Check warning on line 91 in scrapy/downloadermiddlewares/robotstxt.py

See this annotation in the file changed.

@codecov codecov / codecov/patch

scrapy/downloadermiddlewares/robotstxt.py#L91 

Added line #L91 was not covered by tests

Check warning on line 94 in scrapy/downloadermiddlewares/robotstxt.py

See this annotation in the file changed.

@codecov codecov / codecov/patch

scrapy/downloadermiddlewares/robotstxt.py#L94 

Added line #L94 was not covered by tests

Check warning on line 101 in scrapy/downloadermiddlewares/robotstxt.py

See this annotation in the file changed.

@codecov codecov / codecov/patch

scrapy/downloadermiddlewares/robotstxt.py#L99-L101 

Added lines #L99 - L101 were not covered by tests

Check warning on line 105 in scrapy/downloadermiddlewares/robotstxt.py

See this annotation in the file changed.

@codecov codecov / codecov/patch

scrapy/downloadermiddlewares/robotstxt.py#L104-L105 

Added lines #L104 - L105 were not covered by tests

Check warning on line 108 in scrapy/downloadermiddlewares/robotstxt.py

See this annotation in the file changed.

@codecov codecov / codecov/patch

scrapy/downloadermiddlewares/robotstxt.py#L108 

Added line #L108 was not covered by tests

Check warning on line 111 in scrapy/downloadermiddlewares/robotstxt.py

See this annotation in the file changed.

@codecov codecov / codecov/patch

scrapy/downloadermiddlewares/robotstxt.py#L110-L111 

Added lines #L110 - L111 were not covered by tests

Check warning on line 113 in scrapy/downloadermiddlewares/robotstxt.py

See this annotation in the file changed.

@codecov codecov / codecov/patch

scrapy/downloadermiddlewares/robotstxt.py#L113 

Added line #L113 was not covered by tests

Check warning on line 121 in scrapy/downloadermiddlewares/robotstxt.py

See this annotation in the file changed.

@codecov codecov / codecov/patch

scrapy/downloadermiddlewares/robotstxt.py#L121 

Added line #L121 was not covered by tests

Check warning on line 128 in scrapy/downloadermiddlewares/robotstxt.py

See this annotation in the file changed.

@codecov codecov / codecov/patch

scrapy/downloadermiddlewares/robotstxt.py#L128 

Added line #L128 was not covered by tests

Check warning on line 131 in scrapy/downloadermiddlewares/robotstxt.py

See this annotation in the file changed.

@codecov codecov / codecov/patch

scrapy/downloadermiddlewares/robotstxt.py#L130-L131 

Added lines #L130 - L131 were not covered by tests

Check warning on line 133 in scrapy/downloadermiddlewares/robotstxt.py

See this annotation in the file changed.

@codecov codecov / codecov/patch

scrapy/downloadermiddlewares/robotstxt.py#L133 

Added line #L133 was not covered by tests

Check warning on line 153 in scrapy/downloadermiddlewares/robotstxt.py

See this annotation in the file changed.

@codecov codecov / codecov/patch

scrapy/downloadermiddlewares/robotstxt.py#L152-L153 

Added lines #L152 - L153 were not covered by tests

Check warning on line 229 in scrapy/downloadermiddlewares/robotstxt.py

See this annotation in the file changed.

@codecov codecov / codecov/patch

scrapy/downloadermiddlewares/robotstxt.py#L229 

Added line #L229 was not covered by tests

Check warning on line 232 in scrapy/downloadermiddlewares/robotstxt.py

See this annotation in the file changed.

@codecov codecov / codecov/patch

scrapy/downloadermiddlewares/robotstxt.py#L231-L232 

Added lines #L231 - L232 were not covered by tests

Check warning on line 237 in scrapy/downloadermiddlewares/robotstxt.py

See this annotation in the file changed.

@codecov codecov / codecov/patch

scrapy/downloadermiddlewares/robotstxt.py#L237 

Added line #L237 was not covered by tests
View more details on Codecov