Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix PEP8 errors and warnings in list creation script #71

Merged

Conversation

englehardt
Copy link
Contributor

This is the result of a run of autopep8 plus some manual editing. While the diff is impossible to read, we can confirm that it produces the same output:

First, we can compare the list contents using cat *digest256.log | sort | md5sum. I use the log files as a proxy since the actual lists aren't sorted and can't be easily sorted.

Current master with current prod.ini: 8802d79f6e4298d445d06f34484ad540
This PR with current prod.ini: 8802d79f6e4298d445d06f34484ad540

Second, we can compare the number of items in each list.

Current master run on current prod.ini:

Tracking protection(tracking-protection-base): publishing 1897 items; file size 60726
Tracking protection(tracking-protection-baseeff): publishing 2 items; file size 83
Tracking protection(tracking-protection-basew3c): publishing 1 items; file size 51
Tracking protection(tracking-protection-content): publishing 518 items; file size 16598
Tracking protection(tracking-protection-contenteff): publishing 1 items; file size 51
Tracking protection(tracking-protection-contentw3c): publishing 1 items; file size 51
Tracking protection(tracking-protection-ads): publishing 1555 items; file size 49782
Tracking protection(tracking-protection-analytics): publishing 281 items; file size 9013
Tracking protection(tracking-protection-social): publishing 63 items; file size 2037
Entity whitelist(entity-whitelist): publishing 9276 items; file size 296855
Tracking protection(tracking-protection-standard): publishing 1897 items; file size 60726
Tracking protection(tracking-protection-full): publishing 2414 items; file size 77270
Plugin blocklist(plugin-blocklist): publishing 94 items; file size 3029
Plugin blocklist(plugin-blocklist-experiment): publishing 132 items; file size 4245
Plugin blocklist(flash-blocklist): publishing 207 items; file size 6645
Plugin blocklist(flash-exceptions): publishing 2 items; file size 83
Plugin blocklist(flash-allow): publishing 1 items; file size 51
Plugin blocklist(flash-allow-exceptions): publishing 1 items; file size 51
Plugin blocklist(flash-subdoc): publishing 2293 items; file size 73398
Plugin blocklist(flash-subdoc-exceptions): publishing 3 items; file size 115
Plugin blocklist(flashinfobar-exceptions): publishing 10 items; file size 340
Entity whitelist(staging-entity-whitelist): publishing 9723 items; file size 311159
Tracking protection(staging-tracking-protection-standard): publishing 1897 items; file size 60726
Tracking protection(staging-tracking-protection-full): publishing 2414 items; file size 77270
Fastblock(fastblock1): publishing 1751 items; file size 56054
Fastblock whitelist(fastblock1-whitelist): publishing 607 items; file size 19446
Fastblock(fastblock2): publishing 1780 items; file size 56982
Fastblock whitelist(fastblock2-whitelist): publishing 4028 items; file size 128919
Fastblock(fastblock3): publishing 1773 items; file size 56758
Tracking protection(tracking-protection-base-fingerprinting): publishing 44 items; file size 1429
Tracking protection(tracking-protection-content-fingerprinting): publishing 1 items; file size 51
Tracking protection(tracking-protection-base-cryptomining): publishing 38 items; file size 1237
Tracking protection(tracking-protection-content-cryptomining): publishing 1 items; file size 51
Tracking protection(fanboy-annoyance): publishing 1000 items; file size 32022
Tracking protection(fanboy-social): publishing 333 items; file size 10678
Tracking protection(easylist): publishing 22742 items; file size 727767
Tracking protection(easyprivacy): publishing 9482 items; file size 303447
Tracking protection(adguard): publishing 900 items; file size 28822

This PR run on the same prod.ini:

Tracking protection(tracking-protection-base): publishing 1897 items; file size 60726
Tracking protection(tracking-protection-baseeff): publishing 2 items; file size 83
Tracking protection(tracking-protection-basew3c): publishing 1 items; file size 51
Tracking protection(tracking-protection-content): publishing 518 items; file size 16598
Tracking protection(tracking-protection-contenteff): publishing 1 items; file size 51
Tracking protection(tracking-protection-contentw3c): publishing 1 items; file size 51
Tracking protection(tracking-protection-ads): publishing 1555 items; file size 49782
Tracking protection(tracking-protection-analytics): publishing 281 items; file size 9013
Tracking protection(tracking-protection-social): publishing 63 items; file size 2037
Entity whitelist(entity-whitelist): publishing 9276 items; file size 296855
Tracking protection(tracking-protection-standard): publishing 1897 items; file size 60726
Tracking protection(tracking-protection-full): publishing 2414 items; file size 77270
Plugin blocklist(plugin-blocklist): publishing 94 items; file size 3029
Plugin blocklist(plugin-blocklist-experiment): publishing 132 items; file size 4245
Plugin blocklist(flash-blocklist): publishing 207 items; file size 6645
Plugin blocklist(flash-exceptions): publishing 2 items; file size 83
Plugin blocklist(flash-allow): publishing 1 items; file size 51
Plugin blocklist(flash-allow-exceptions): publishing 1 items; file size 51
Plugin blocklist(flash-subdoc): publishing 2293 items; file size 73398
Plugin blocklist(flash-subdoc-exceptions): publishing 3 items; file size 115
Plugin blocklist(flashinfobar-exceptions): publishing 10 items; file size 340
Entity whitelist(staging-entity-whitelist): publishing 9723 items; file size 311159
Tracking protection(staging-tracking-protection-standard): publishing 1897 items; file size 60726
Tracking protection(staging-tracking-protection-full): publishing 2414 items; file size 77270
Fastblock(fastblock1): publishing 1751 items; file size 56054
Fastblock whitelist(fastblock1-whitelist): publishing 607 items; file size 19446
Fastblock(fastblock2): publishing 1780 items; file size 56982
Fastblock whitelist(fastblock2-whitelist): publishing 4028 items; file size 128919
Fastblock(fastblock3): publishing 1773 items; file size 56758
Tracking protection(tracking-protection-base-fingerprinting): publishing 44 items; file size 1429
Tracking protection(tracking-protection-content-fingerprinting): publishing 1 items; file size 51
Tracking protection(tracking-protection-base-cryptomining): publishing 38 items; file size 1237
Tracking protection(tracking-protection-content-cryptomining): publishing 1 items; file size 51
Tracking protection(fanboy-annoyance): publishing 1000 items; file size 32022
Tracking protection(fanboy-social): publishing 333 items; file size 10678
Tracking protection(easylist): publishing 22742 items; file size 727767
Tracking protection(easyprivacy): publishing 9482 items; file size 303447
Tracking protection(adguard): publishing 900 items; file size 28822

Fixes #60.

@englehardt englehardt changed the base branch from master to list_parser_cleanup March 11, 2019 00:05
@englehardt
Copy link
Contributor Author

I've set this to merge into the list_parser_cleanup branch. I'll open a couple other PRs against that branch and we can merge all of them together. This should be merged first, and I can rebase the other patches on top of it.

Copy link
Contributor

@groovecoder groovecoder left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ran lists2safebrowsing.py with the same config and verified the list item counts and md5 hashes are the same between mozilla-services/master branch and englehardt/autopep8 branch.

Copy link
Contributor

@groovecoder groovecoder left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ran lists2safebrowsing.py between both branches of the code and got the same list item counts and md5 hashes.

@groovecoder groovecoder merged commit 0a77586 into mozilla-services:list_parser_cleanup Mar 12, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants