Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[MRG+1] provide better error message when disabling s3 exporter #3358

Merged

Conversation

@BurnzZ
Copy link
Member

@BurnzZ BurnzZ commented Jul 31, 2018

Fixes: #3357

A quick test to confirm would be to run: scrapy shell -s FEED_URI='s3://test.com' in a shell without either botocore or boto library installed/declared.

Output:

2018-07-31 19:03:55 [scrapy.utils.log] INFO: Scrapy 1.5.0 started (bot: scrapybot)
2018-07-31 19:03:55 [scrapy.utils.log] INFO: Versions: lxml 4.2.3.0, libxml2 2.9.8, cssselect 1.0.3, parsel 1.5.0, w3lib 1.19.0, Twisted 18.7.0, Python 2.7.15 (default, May  1 2018, 16:44:37) - [GCC 4.2.1 Compatible Apple LLVM 9.0.0 (clang-900.0.39.2)], pyOpenSSL 18.0.0 (OpenSSL 1.1.0h  27 Mar 2018), cryptography 2.3, Platform Darwin-16.7.0-x86_64-i386-64bit
2018-07-31 19:03:55 [scrapy.crawler] INFO: Overridden settings: {'FEED_URI': 's3://test.com', 'LOGSTATS_INTERVAL': 0, 'EDITOR': 'vim', 'DUPEFILTER_CLASS': 'scrapy.dupefilters.BaseDupeFilter'}
2018-07-31 19:03:56 [scrapy.extensions.feedexport] ERROR: Disabled feed storage scheme: s3. Reason: missing botocore or boto library

 <more log output truncated>
@codecov
Copy link

@codecov codecov bot commented Jul 31, 2018

Codecov Report

Merging #3358 into master will increase coverage by 0.01%.
The diff coverage is 0%.

@@            Coverage Diff             @@
##           master    #3358      +/-   ##
==========================================
+ Coverage   84.34%   84.35%   +0.01%     
==========================================
  Files         167      167              
  Lines        9359     9359              
  Branches     1388     1388              
==========================================
+ Hits         7894     7895       +1     
  Misses       1209     1209              
+ Partials      256      255       -1
Impacted Files Coverage Δ
scrapy/extensions/feedexport.py 78.46% <0%> (ø) ⬆️
scrapy/utils/trackref.py 86.48% <0%> (+2.7%) ⬆️

@kmike kmike changed the title provide better error message when disabling s3 exporter [MRG+1] provide better error message when disabling s3 exporter Jul 31, 2018
@kmike kmike added this to the v1.6 milestone Jul 31, 2018
@kmike
Copy link
Member

@kmike kmike commented Jul 31, 2018

A good idea!

@lopuhin lopuhin merged commit d90bdd5 into scrapy:master Aug 2, 2018
1 of 2 checks passed
@lopuhin
Copy link
Member

@lopuhin lopuhin commented Aug 2, 2018

Thanks @BurnzZ , the error message is much better now!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Linked issues

Successfully merging this pull request may close these issues.

None yet

3 participants