Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use pprint.pformat on versions log message #4203

Open
wants to merge 3 commits into
base: master
Choose a base branch
from

Conversation

grammy-jiang
Copy link
Contributor

This commit use pprint.pformat on overridden settings to keep consistent
with scrapy.middleware, for example:
Original:

2019-12-01 11:35:40 [scrapy.utils.log] INFO: Versions: lxml 4.3.4.0, libxml2 2.9.9, cssselect 1.0.3, parsel 1.5.1, w3lib 1.20.0, Twisted 19.2.1, Python 3.7.5 (default, Oct 17 2019, 12:16:48) - [GCC 9.2.1 20190827 (Red Hat 9.2.1-1)], pyOpenSSL 19.0.0 (OpenSSL 1.1.1c  28 May 2019), cryptography 2.7, Platform Linux-5.3.7-301.fc31.x86_64-x86_64-with-fedora-31-Thirty_One

Now:

2019-12-01 11:04:22 [scrapy.utils.log] INFO: Versions:
{'Platform': 'Linux-5.3.7-301.fc31.x86_64-x86_64-with-fedora-31-Thirty_One',
 'Python': '3.7.5 (default, Oct 17 2019, 12:16:48) - [GCC 9.2.1 20190827 (Red '
           'Hat 9.2.1-1)]',
 'Twisted': '19.2.1',
 'cryptography': '2.7',
 'cssselect': '1.0.3',
 'libxml2': '2.9.9',
 'lxml': '4.3.4.0',
 'parsel': '1.5.1',
 'pyOpenSSL': '19.0.0 (OpenSSL 1.1.1c  28 May 2019)',
 'w3lib': '1.20.0'}

This commit use pprint.pformat on overridden settings to keep consistent
with scrapy.middleware
@codecov
Copy link

codecov bot commented Dec 1, 2019

Codecov Report

Merging #4203 into master will increase coverage by <.01%.
The diff coverage is 100%.

@@            Coverage Diff             @@
##           master    #4203      +/-   ##
==========================================
+ Coverage   83.73%   83.73%   +<.01%     
==========================================
  Files         165      165              
  Lines        9714     9715       +1     
  Branches     1446     1446              
==========================================
+ Hits         8134     8135       +1     
  Misses       1332     1332              
  Partials      248      248
Impacted Files Coverage Δ
scrapy/utils/log.py 88.63% <100%> (+0.13%) ⬆️

@wRAR
Copy link
Member

wRAR commented Dec 2, 2019

As seen in the provided example, this changes the order of components.

scrapy_components_versions() returns a list of tuples while the code creates a dict. Can you please fix this, maybe OrderedDict will help?

@grammy-jiang
Copy link
Contributor Author

grammy-jiang commented Dec 2, 2019

The order is changed by the pprint module, and disabling the sorted function inside pprint is only available in Python 3.8. Although a decorator can be added to fix this, this makes things too complicated. I want to keep the solution simple.

And with the help of OrderedDict, the output is like this:

2019-12-03 09:13:05 [scrapy.utils.log] INFO: Versions:
OrderedDict([('lxml', '4.4.2.0'),
             ('libxml2', '2.9.9'),
             ('cssselect', '1.1.0'),
             ('parsel', '1.5.2'),
             ('w3lib', '1.21.0'),
             ('Twisted', '19.10.0'),
             ('Python', '3.7.5 (default, Nov 27 2019, 16:53:22) - [GCC 7.4.0]'),
             ('pyOpenSSL', '18.0.0 (OpenSSL 1.1.0i  14 Aug 2018)'),
             ('cryptography', '2.3.1'),
             ('Platform',
              'Linux-5.0.0-36-generic-x86_64-with-debian-buster-sid')])

Emmm, which looks strange. I don't like this format break solution.

@Gallaecio
Copy link
Member

Gallaecio commented Dec 3, 2019

We could write a private function just to print the desired output format from the versions iterable.

It think it’s either that or wait until we drop support for CPython 3.5.

@grammy-jiang
Copy link
Contributor Author

grammy-jiang commented Dec 3, 2019

Consider nobody raises an issue about this output format and it is quite minor problem, I think we should not do too much work just for this. We can just leave it until the minimum support version becomes Python 3.8.

@Gallaecio Gallaecio added this to the Drop Python 3.5 support milestone Dec 3, 2019
@Gallaecio Gallaecio modified the milestones: Python 3.6+, 2.4, Python 3.8+ Aug 17, 2020
@wRAR
Copy link
Member

wRAR commented Jun 18, 2023

3.7 is almost EOL so we can now use sort_dicts=False.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants