Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[MRG+1] Add a setting to use active mode in FTPFeedStorage #3829

Merged
merged 6 commits into from Jul 10, 2019

Conversation

@heylouiz
Copy link
Contributor

@heylouiz heylouiz commented Jun 14, 2019

FTP servers can be configured in active and passive mode, by default
Python and Scrapy use passive mode and there was no way to use active.

This commit addes a setting FEED_STORAGE_FTP_ACTIVE to allow a feed
to be exported to a FTP server configured in active mode.

FTP servers can be configured in active and passive mode, by default
Python and Scrapy use passive mode and there was no way to use active.

This commit addes a setting FEED_STORAGE_FTP_ACTIVE to allow a feed
to be exported to a FTP server configured in active mode.
@codecov
Copy link

@codecov codecov bot commented Jun 14, 2019

Codecov Report

Merging #3829 into master will decrease coverage by 0.04%.
The diff coverage is 42.85%.

@@            Coverage Diff             @@
##           master    #3829      +/-   ##
==========================================
- Coverage   85.43%   85.39%   -0.05%     
==========================================
  Files         169      169              
  Lines        9649     9655       +6     
  Branches     1437     1438       +1     
==========================================
+ Hits         8244     8245       +1     
- Misses       1157     1161       +4     
- Partials      248      249       +1
Impacted Files Coverage Δ
scrapy/settings/default_settings.py 98.65% <100%> (ø) ⬆️
scrapy/extensions/feedexport.py 83.41% <33.33%> (-1.5%) ⬇️
scrapy/utils/trackref.py 83.78% <0%> (-2.71%) ⬇️

@codecov
Copy link

@codecov codecov bot commented Jun 14, 2019

Codecov Report

Merging #3829 into master will increase coverage by 0.12%.
The diff coverage is 28.57%.

@@            Coverage Diff             @@
##           master    #3829      +/-   ##
==========================================
+ Coverage   85.43%   85.56%   +0.12%     
==========================================
  Files         169      164       -5     
  Lines        9649     9551      -98     
  Branches     1437     1431       -6     
==========================================
- Hits         8244     8172      -72     
+ Misses       1157     1132      -25     
+ Partials      248      247       -1
Impacted Files Coverage Δ
scrapy/settings/default_settings.py 98.66% <100%> (+0.01%) ⬆️
scrapy/extensions/feedexport.py 83.48% <16.66%> (-1.42%) ⬇️
scrapy/utils/decorators.py 80% <0%> (-8%) ⬇️
scrapy/crawler.py 89.94% <0%> (-2.46%) ⬇️
scrapy/pipelines/files.py 66.53% <0%> (-1.71%) ⬇️
scrapy/commands/parse.py 20.32% <0%> (-0.73%) ⬇️
scrapy/spiders/crawl.py 81.7% <0%> (-0.65%) ⬇️
scrapy/http/request/form.py 95.41% <0%> (-0.62%) ⬇️
scrapy/spiders/__init__.py 98.27% <0%> (-0.3%) ⬇️
... and 26 more

scrapy/extensions/feedexport.py Outdated Show resolved Hide resolved
scrapy/extensions/feedexport.py Outdated Show resolved Hide resolved
scrapy/extensions/feedexport.py Show resolved Hide resolved
@heylouiz
Copy link
Contributor Author

@heylouiz heylouiz commented Jul 9, 2019

Hey @dangra, I made the changes you requested, can you please review again?

I had to instantiate the test spider using from_crawler because the test was failing in https://github.com/scrapy/scrapy/pull/3829/files#diff-b40c5755a5dda2b02aaac6bf4c8e2ff8R50
AttributeError: 'Spider' object has no attribute 'crawler'

@kmike kmike changed the title Add a setting to use active mode in FTPFeedStorage [MRG+1] Add a setting to use active mode in FTPFeedStorage Jul 10, 2019
kmike
kmike approved these changes Jul 10, 2019
Copy link
Member

@kmike kmike left a comment

Looks good, thanks @heylouiz!

docs/topics/feed-exports.rst Outdated Show resolved Hide resolved
docs/topics/feed-exports.rst Outdated Show resolved Hide resolved
docs/topics/feed-exports.rst Outdated Show resolved Hide resolved
@Gallaecio Gallaecio merged commit df68c4b into scrapy:master Jul 10, 2019
0 of 2 checks passed
@kmike kmike added this to the v1.7 milestone Jul 11, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Linked issues

Successfully merging this pull request may close these issues.

None yet

4 participants