Skip to content

Commit

Permalink
Fixes for flake8 and pep8/pylint
Browse files Browse the repository at this point in the history
Changed max line length to 119 characters
  • Loading branch information
gleitz committed Apr 15, 2019
1 parent c6f0c67 commit db67bda
Show file tree
Hide file tree
Showing 2 changed files with 30 additions and 17 deletions.
File renamed without changes.
47 changes: 30 additions & 17 deletions howdoi/howdoi.py
Expand Up @@ -7,10 +7,9 @@
# inspired by Rich Jones (rich@anomos.info)
#
######################################################

from __future__ import print_function
import gc
gc.disable() # disable right at the start, we don't need it

import argparse
import os
import appdirs
Expand All @@ -29,6 +28,8 @@
from requests.exceptions import ConnectionError
from requests.exceptions import SSLError

gc.disable() # disable right at the start, we don't need it

This comment has been minimized.

Copy link
@gleitz

gleitz Apr 15, 2019

Author Owner

@paulie-g flake8 E402 wants all the imports first, then code. I would think that moving this below the imports would still give the speed advantage, but lmk.

This comment has been minimized.

Copy link
@paulie-g

paulie-g Apr 15, 2019

Contributor

It gives most of the advantage, yes, but imports produce a ton of garbage. It's not worth giving up just to make a linter happy. This isn't wanton, capricious bad coding form - it serves a purpose. There should be a way to disable just that warning manually. Even if not, I'm +75 of 100 on putting it back where it was.

This comment has been minimized.

Copy link
@paulie-g

paulie-g Apr 15, 2019

Contributor

As an aside, now that you've OKed the networking work, I'll have to stick a thread launch almost at the top before most of the imports anyway and that will absolutely be necessary and we'll still have the linting warning.

This comment has been minimized.

Copy link
@gleitz

gleitz Apr 15, 2019

Author Owner

Sure thing – just give it the 'ol # noqa: E402 and it should be fine. As an FYI I generally run flake8 (mostly vanilla save for the line length here) and pylint (configuration here, also with the line length edit)

This comment has been minimized.

Copy link
@paulie-g

paulie-g Apr 15, 2019

Contributor

Maybe stick that into contributing.md? I never run linters personally - my megalomania says the only person who knows better is Brian Kernighan and until he writes one (unlikely at this point), I'm not running one ;) I try very hard to stick to the coding style of the project I'm contributing to though because that is actually important imo (and stops you getting abused in public on lkml ;).

This comment has been minimized.

Copy link
@gleitz

gleitz Apr 15, 2019

Author Owner

Yes, will make a section in the README


# Handle imports for Python 2 and 3
if sys.version < '3':
import codecs
Expand All @@ -45,10 +46,12 @@ def u(x):
def u(x):
return x


# rudimentary standardized 3-level log output
_print_err = lambda x: print("[ERROR] " + x)
_print_ok = print
_print_dbg = lambda x: print("[DEBUG] " + x)
def _print_err(x): print("[ERROR] " + x)
_print_ok = print # noqa: E305
def _print_dbg(x): print("[DEBUG] " + x) # noqa: E302


if os.getenv('HOWDOI_DISABLE_SSL'): # Set http instead of https
SCHEME = 'http://'
Expand All @@ -72,9 +75,10 @@ def u(x):
}

BLOCK_INDICATORS = (
'form id="captcha-form"',
'This page appears when Google automatically detects requests coming from your computer network which appear to be in violation of the <a href="//www.google.com/policies/terms/">Terms of Service'
)
'form id="captcha-form"',
'This page appears when Google automatically detects requests coming from your computer '
'network which appear to be in violation of the <a href="//www.google.com/policies/terms/">Terms of Service'
)

STAR_HEADER = u('\u2605')
ANSWER_HEADER = u('{2} Answer from {0} {2}\n{1}')
Expand All @@ -85,15 +89,17 @@ def u(x):
CACHE_ENTRY_MAX = 128

if os.getenv('HOWDOI_DISABLE_CACHE'):
cache = NullCache() # works like an always empty cache, cleaner than 'if cache:' everywhere
cache = NullCache() # works like an always empty cache
else:
cache = FileSystemCache(CACHE_DIR, CACHE_ENTRY_MAX, default_timeout=0)

howdoi_session = requests.session()


class BlockError(RuntimeError):
pass


def _random_int(width):
bres = os.urandom(width)
if sys.version < '3':
Expand All @@ -103,9 +109,11 @@ def _random_int(width):

return ires


def _random_choice(seq):
return seq[_random_int(1) % len(seq)]


def get_proxies():
proxies = getproxies()
filtered_proxies = {}
Expand All @@ -125,7 +133,7 @@ def _get_result(url):
verify=VERIFY_SSL_CERTIFICATE).text
except requests.exceptions.SSLError as e:
_print_err('Encountered an SSL Error. Try using HTTP instead of '
'HTTPS by setting the environment variable "HOWDOI_DISABLE_SSL".\n')
'HTTPS by setting the environment variable "HOWDOI_DISABLE_SSL".\n')
raise e


Expand Down Expand Up @@ -171,23 +179,25 @@ def _extract_links(html, search_engine):
def _get_search_url(search_engine):
return SEARCH_URLS.get(search_engine, SEARCH_URLS['google'])


def _is_blocked(page):
for indicator in BLOCK_INDICATORS:
if page.find(indicator) != -1:
return True

return False


def _get_links(query):
search_engine = os.getenv('HOWDOI_SEARCH_ENGINE', 'google')
search_url = _get_search_url(search_engine)

result = _get_result(search_url.format(URL, url_quote(query)))
if _is_blocked(result):
_print_err('Unable to find an answer because the search engine temporarily blocked the request. '
'Please wait a few minutes or select a different search engine.')
'Please wait a few minutes or select a different search engine.')
raise BlockError("Temporary block by search engine")

html = pq(result)
return _extract_links(html, search_engine)

Expand Down Expand Up @@ -244,7 +254,7 @@ def _get_answer(args, links):
if args.get('link'):
return link

cache_key = link
cache_key = link
page = cache.get(link)
if not page:
page = _get_result(link + '?answertab=votes')
Expand Down Expand Up @@ -276,6 +286,7 @@ def _get_answer(args, links):
text = text.strip()
return text


def _get_links_with_cache(query):
cache_key = query + "-links"
res = cache.get(cache_key)
Expand All @@ -293,6 +304,7 @@ def _get_links_with_cache(query):

return question_links


def _get_instructions(args):
question_links = _get_links_with_cache(args['query'])
if not question_links:
Expand Down Expand Up @@ -330,9 +342,10 @@ def _clear_cache():
global cache
if not cache:
cache = FileSystemCache(CACHE_DIR, CACHE_ENTRY_MAX, 0)

return cache.clear()


def howdoi(args):
args['query'] = ' '.join(args['query']).replace('?', '')
cache_key = str(args)
Expand All @@ -342,11 +355,11 @@ def howdoi(args):
return res

try:
res = _get_instructions(args)
res = _get_instructions(args)
if not res:
res = 'Sorry, couldn\'t find any help with that topic\n'
cache.set(cache_key, res)

return res
except (ConnectionError, SSLError):
return 'Failed to establish network connection\n'
Expand Down Expand Up @@ -396,7 +409,7 @@ def command_line_runner():

utf8_result = howdoi(args).encode('utf-8', 'ignore')
if sys.version < '3':
print(utf8_result)
_print_ok(utf8_result)
else:
# Write UTF-8 to stdout: https://stackoverflow.com/a/3603160
sys.stdout.buffer.write(utf8_result)
Expand Down

3 comments on commit db67bda

@gleitz
Copy link
Owner Author

@gleitz gleitz commented on db67bda Apr 15, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@paulie-g I pushes some fixes that correct flake8/pep8/pylint errors. It's mostly whitespace related, except the gc bit.

@paulie-g
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Two things:

  • I put the gc.disable() where I put it advisedly. It's better to put it back and disable the lint warning on it with a pragma.
  • It's up to you obvs, but I left the final result as a direct call to print on purpose. As amended, it's asymmetrical with what we do in the py3 case. I also can't see where we'd want to do something with the final output by using the _print_xxx() indirection. I envisioned using it in future to mute everything other than the final output with a command line flag (ie for machine readability).

@gleitz
Copy link
Owner Author

@gleitz gleitz commented on db67bda Apr 15, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 5a3d8e6

Please sign in to comment.