-
Notifications
You must be signed in to change notification settings - Fork 10.7k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add feed_slot_closed and feed_exporter_closed signals to work with FeedExporter #5876
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why not keep the original deferred_list
list, store DeferredList(deferred_list)
in a variable, and add the feed_exporter_closed signal as a callback to that DeferredList(deferred_list)
?
Codecov Report
@@ Coverage Diff @@
## master #5876 +/- ##
==========================================
- Coverage 88.84% 88.78% -0.06%
==========================================
Files 162 162
Lines 11055 11175 +120
Branches 1800 1817 +17
==========================================
+ Hits 9822 9922 +100
- Misses 954 966 +12
- Partials 279 287 +8
|
@Gallaecio Something like
? |
Previously, the code assumed that only the slots in By getting rid of the previous approach and relying only on awaiting scrapy/scrapy/extensions/feedexport.py Lines 411 to 417 in a0b7ea9
The current commit centralices all the new logic inside
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
OK, I did not realize another method (item_scraped) was in the picture.
The approach looks good to me, then.
Next step: Please, check on the CI issues; we have some flaky tests, but at least some of the issues seem legit.
Jerrod from Codecov here --> I'm noticing you guys are seeing this error a lot on Codecov. Did you change the git behavior or CI uploading step at all recently?
|
Looking at https://github.com/scrapy/scrapy/blame/master/.github/workflows/tests-ubuntu.yml, I would say the last change was 3 years ago. |
…e, change in docs, add tests
Let me know if you had something else else in mind |
For deprecation using scrapy/scrapy/utils/deprecate.py Line 20 in 5a37af1
|
The original issue reported here, while minor, is still pending 😅 |
Thanks! |
The signals will be sent when a slot and the FeedExporter are closed respectively.
This allows for the use of extensions related to feeds (like email feeds or slack notifications) that may require them to be used without the need for a custom FeedExporter.