Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use pprint.pformat on overridden settings #4199

Merged
merged 3 commits into from
Nov 29, 2019
Merged

Conversation

grammy-jiang
Copy link
Contributor

@grammy-jiang grammy-jiang commented Nov 28, 2019

This commit use pprint.pformat on overridden settings to keep consistent
with scrapy.middleware, for example:

Original:

2019-11-28 22:23:57 [scrapy.crawler] INFO: Overridden settings: {'BOT_NAME': 'spider_demo', 'NEWSPIDER_MODULE': 'spider_demo.spiders', 'SPIDER_MODULES': ['spider_demo.spiders'], 'USER_AGENT': 'Mozilla/5.0 (X11; Fedora; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/74.0.3729.157 Safari/537.36'}
2019-11-28 22:23:57 [scrapy.middleware] INFO: Enabled extensions:
['scrapy.extensions.corestats.CoreStats',
 'scrapy.extensions.telnet.TelnetConsole',
 'scrapy.extensions.memusage.MemoryUsage',
 'scrapy.extensions.logstats.LogStats']
2019-11-28 22:23:57 [scrapy.crawler] INFO: Overridden settings:
{'BOT_NAME': 'spider_demo',
 'NEWSPIDER_MODULE': 'spider_demo.spiders',
 'SPIDER_MODULES': ['spider_demo.spiders'],
 'USER_AGENT': 'Mozilla/5.0 (X11; Fedora; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/74.0.3729.157 Safari/537.36'}
2019-11-28 22:23:57 [scrapy.middleware] INFO: Enabled extensions:
['scrapy.extensions.corestats.CoreStats',
 'scrapy.extensions.telnet.TelnetConsole',
 'scrapy.extensions.memusage.MemoryUsage',
 'scrapy.extensions.logstats.LogStats']

This commit use pprint.pformat on overridden settings to keep consistent
with scrapy.middleware
@codecov
Copy link

codecov bot commented Nov 28, 2019

Codecov Report

Merging #4199 into master will decrease coverage by <.01%.
The diff coverage is 100%.

@@            Coverage Diff             @@
##           master    #4199      +/-   ##
==========================================
- Coverage   83.74%   83.73%   -0.01%     
==========================================
  Files         165      165              
  Lines        9713     9714       +1     
  Branches     1446     1446              
==========================================
  Hits         8134     8134              
  Misses       1332     1332              
- Partials      247      248       +1
Impacted Files Coverage Δ
scrapy/crawler.py 90% <100%> (+0.05%) ⬆️
scrapy/utils/trackref.py 83.78% <0%> (-2.71%) ⬇️

This commit changes the log message string format from r to s
@elacuesta
Copy link
Member

If I'm not mistaken this is technically a slightly backward incompatible change - any log parsers which handle job's text output would need to adapt to this new format. I'm in favor of the patch, but we might want to mention this fact somewhere in the release notes.

scrapy/crawler.py Outdated Show resolved Hide resolved
Co-Authored-By: Adrián Chaves <adrian@chaves.io>
@Gallaecio Gallaecio merged commit d1cdfb4 into scrapy:master Nov 29, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants