added custom scheduler breakage in news.rst #4274
Conversation
Codecov Report
@@ Coverage Diff @@
## master #4274 +/- ##
=========================================
- Coverage 84.06% 83.9% -0.17%
=========================================
Files 166 166
Lines 9730 9869 +139
Branches 1454 1469 +15
=========================================
+ Hits 8180 8281 +101
- Misses 1296 1334 +38
Partials 254 254
|
|
Hi there, thanks for the PR. Could you add a few words explaining why this happens? Thanks again! |
|
@elacuesta , sure I'll do that! |
|
hey @elacuesta , I'm still new to scrapy, can you explain what the changes are exactly doing, all I got was that we have set crawler==None in the init function |
docs/news.rst
Outdated
| @@ -288,6 +288,8 @@ Backward-incompatible changes | |||
| :class:`~scrapy.http.Request` objects instead of arbitrary Python data | |||
| structures. | |||
|
|
|||
| * This version of scrapy breaks custom schedulers | |||
There was a problem hiding this comment.
I know this is how I phrased it in the issue, but I think the actual documentation text should go more in the line of the entries above.
It should indicate that the __init__ method of the scrapy.core.scheduler.Scheduler class now received an additional crawler parameter, and that because of that custom Scheduler subclasses that don’t accept arbitrary parameters in their __init__ method will stop working.
Since we do not have API documentation of scrapy.core.scheduler.Scheduler at the moment that we can link, scrapy.core.scheduler.Scheduler should probably link to the documentation of the SCHEDULER setting.
There was a problem hiding this comment.
@Gallaecio , thanks for the help! Appreciate it!
docs/news.rst
Outdated
| @@ -288,6 +288,14 @@ Backward-incompatible changes | |||
| :class:`~scrapy.http.Request` objects instead of arbitrary Python data | |||
| structures. | |||
|
|
|||
| * An additional `crawler` paramter has been added to the `__init__` method of | |||
| the `scrapy.core.scheduler.Scheduler` class. Custom scheduler subclassses | |||
| don't accept binary parameters in their `__init__` method. This version of | |||
There was a problem hiding this comment.
@kmike my bad, I meant to write arbitrary.
This is not necessarily true. The note should say that custom schedulers might break, if they override the |
|
@elacuesta, okay will change it! Is there anything else that needs to be changed? |
|
Nothing substantial, just keep an eye open for typos, and change "Scrapy" to be uppercase. Thanks! |
|
@elacuesta @kmike @Gallaecio let me know if the latest changes are okay, I'll squash all commits into one. |
docs/news.rst
Outdated
| @@ -288,6 +288,14 @@ Backward-incompatible changes | |||
| :class:`~scrapy.http.Request` objects instead of arbitrary Python data | |||
| structures. | |||
|
|
|||
| * An additional `crawler` paramter has been added to the `__init__` method of | |||
| the `scrapy.core.scheduler.Scheduler` class. Custom scheduler subclassses | |||
There was a problem hiding this comment.
I think this is still not 100% accurate. What do you think of something like the following?
Custom scheduler subclasses which don't accept arbitrary parameters in their
__init__method might break because of this change.
There was a problem hiding this comment.
@elacuesta , you're right. Your sentence is clear & concise.
There was a problem hiding this comment.
I apologize if I wasn't clear, I didn't mean you should suppress the first sentence ("An additional crawler parameter has been added to the __init__ method of the scrapy.core.scheduler.Scheduler class"). That sentence is the explanation for the following one (currently the paragraph mentions "this change" without mentioning the specific change).
Also, I just realized that two accent characters are needed for Sphinx to render text as monospace, one means italic. You could check this by running make html from the docs directory.
Lastly, I think it would be good to refer to the Scheduler class using the class directive. There is no API reference page for the Scheduler at the moment, but doing so would make this part link to it if we add it in the future. What I mean is:
:class:`scrapy.core.scheduler.Scheduler`
Sorry for the back and forth, thanks for your patience.
There was a problem hiding this comment.
@elacuesta, I'm here to learn. Thanks for taking the time out to help a beginner like me!
I'll make the requested changes!
|
Build failures seem to be completely unrelated, but let's close and reopen just in case. Update: Intersphinx is failing because https://twistedmatrix.com is down. |
You're welcome! Is there anything else that needs to be done? |
@Gallaecio created an issue #4264, in which I had to add in the backward-incompatible changes that scrapy 1.7.0 version breaks custom schedulers.
I have added it to news.rst, please check it out and let me know if it needs to be more thorough.
I have ran all the tests using tox
(edit) Fixes #4264