New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enable Pylint checks for PyWren #144

Merged
merged 10 commits into from Aug 4, 2017

Conversation

Projects
None yet
4 participants
@shivaram
Collaborator

shivaram commented Jul 11, 2017

This PR addresses a bunch of remaining Pylint issues and for the ones which are not addressed, we add a local comment to disable Pylint. Finally this PR also adds a Pylint check into Travis.

@shivaram shivaram changed the title from Enable PyLint checks for PyWren to Enable Pylint checks for PyWren Jul 11, 2017

@shivaram

This comment has been minimized.

Show comment
Hide comment
@shivaram

shivaram Jul 11, 2017

Collaborator

@ooq This probably needs more closer review than the last one as it actually has some code changes. The travis check on the branch passes https://travis-ci.org/pywren/pywren/builds/252271798

Collaborator

shivaram commented Jul 11, 2017

@ooq This probably needs more closer review than the last one as it actually has some code changes. The travis check on the branch passes https://travis-ci.org/pywren/pywren/builds/252271798

@@ -34,7 +34,7 @@ install:
- conda info -a
- conda create -q -n test-environment python=$TRAVIS_PYTHON_VERSION numpy pytest cython nose boto3 PyYAML Click pytest numba
- source activate test-environment
- pip install glob2
- pip install glob2 pylint tornado

This comment has been minimized.

@ooq

ooq Jul 19, 2017

Collaborator

how was tornado installed before?

@ooq

ooq Jul 19, 2017

Collaborator

how was tornado installed before?

This comment has been minimized.

@shivaram

shivaram Jul 19, 2017

Collaborator

It was not installed before but we seemed to use it in cloudpickle.py[1] if it was available. The problem is lint doesn't like it when we import packages in _rebuild_tornado_coroutine.

[1]

def is_tornado_coroutine(func):

@shivaram

shivaram Jul 19, 2017

Collaborator

It was not installed before but we seemed to use it in cloudpickle.py[1] if it was available. The problem is lint doesn't like it when we import packages in _rebuild_tornado_coroutine.

[1]

def is_tornado_coroutine(func):

@@ -87,3 +88,4 @@ env:
- RUN_LAMBDA=true
- RUN_MACROREDUCE=true
- RUN_COMMANDLINE=true
- RUN_PYLINT=true

This comment has been minimized.

@ooq

ooq Jul 26, 2017

Collaborator

Maybe it's more resource-economic to run PyLint for all configurations, and halt the actual tests if linting fails? Essentially, passing linting should be a pre-requisite.

@ooq

ooq Jul 26, 2017

Collaborator

Maybe it's more resource-economic to run PyLint for all configurations, and halt the actual tests if linting fails? Essentially, passing linting should be a pre-requisite.

This comment has been minimized.

@ooq

ooq Jul 26, 2017

Collaborator

Ideally, it would be nice if we can run RUN_PYLINT=true first, and make its passing be a dependence for RUN_LAMBDA=true, etc. But I don't think it's feasible with Travis?

@ooq

ooq Jul 26, 2017

Collaborator

Ideally, it would be nice if we can run RUN_PYLINT=true first, and make its passing be a dependence for RUN_LAMBDA=true, etc. But I don't think it's feasible with Travis?

This comment has been minimized.

@shivaram

shivaram Jul 26, 2017

Collaborator

We can form a master script that does this - like they describe in https://docs.travis-ci.com/user/customizing-the-build/#Implementing-Complex-Build-Steps ?

@shivaram

shivaram Jul 26, 2017

Collaborator

We can form a master script that does this - like they describe in https://docs.travis-ci.com/user/customizing-the-build/#Implementing-Complex-Build-Steps ?

This comment has been minimized.

@ooq

ooq Jul 26, 2017

Collaborator

Cool. I think we can make a separate issue/PR for this.

@ooq

ooq Jul 26, 2017

Collaborator

Cool. I think we can make a separate issue/PR for this.

@@ -173,10 +173,10 @@ def terminate_instances(instance_list):
# FIXME delete individuals
"""
for instance_name, instance_obj in instance_list:
logger.debug('Terminating instance %s', instance_name)

This comment has been minimized.

@ooq

ooq Jul 26, 2017

Collaborator

This is from merging?

@ooq

ooq Jul 26, 2017

Collaborator

This is from merging?

This comment has been minimized.

@shivaram

shivaram Jul 26, 2017

Collaborator

No its actually trying to use instance_name to avoid an unused variable warning. i can also make it an _

@shivaram

shivaram Jul 26, 2017

Collaborator

No its actually trying to use instance_name to avoid an unused variable warning. i can also make it an _

This comment has been minimized.

@ooq

ooq Jul 26, 2017

Collaborator

I see. It's cool.

@ooq

ooq Jul 26, 2017

Collaborator

I see. It's cool.

for mod_path in list(mod_paths):
if module in mod_path and mod_path in mod_paths:
mod_paths.remove(mod_path)
if exclude_modules:

This comment has been minimized.

@apengwin

apengwin Jul 26, 2017

Contributor

Should we be more explicit, with if exclude_modules is None?

@apengwin

apengwin Jul 26, 2017

Contributor

Should we be more explicit, with if exclude_modules is None?

This comment has been minimized.

@shivaram

shivaram Jul 26, 2017

Collaborator

We also dont want to run this loop if exclude_modules is an empty list ?

@shivaram

shivaram Jul 26, 2017

Collaborator

We also dont want to run this loop if exclude_modules is an empty list ?

This comment has been minimized.

@apengwin

apengwin Jul 26, 2017

Contributor

You're right. I'd assumed that a user would never pass in an empty list to exclude_modules, but that's not the case.

@apengwin

apengwin Jul 26, 2017

Contributor

You're right. I'd assumed that a user would never pass in an empty list to exclude_modules, but that's not the case.

@ooq

This looks good to me. I have only one question w.r.t. how we should maintain modified external sources such as cloudpickle.py.

@shivaram

This comment has been minimized.

Show comment
Hide comment
@shivaram

shivaram Aug 4, 2017

Collaborator

@ooq I reverted the cloudpickle.py changes (it will look like a diff here because the earlier PR had some whitespace fixes and I also reverted those).

Waiting for Travis now and I guess somebody will need to approve this PR as well !

Collaborator

shivaram commented Aug 4, 2017

@ooq I reverted the cloudpickle.py changes (it will look like a diff here because the earlier PR had some whitespace fixes and I also reverted those).

Waiting for Travis now and I guess somebody will need to approve this PR as well !

@ooq

ooq approved these changes Aug 4, 2017

LGTM. Thanks @shivaram ! Let's merge it as soon as tests pass.

@shivaram shivaram merged commit ba14fa2 into master Aug 4, 2017

2 checks passed

continuous-integration/travis-ci/pr The Travis CI build passed
Details
continuous-integration/travis-ci/push The Travis CI build passed
Details
@shivaram

This comment has been minimized.

Show comment
Hide comment
@shivaram

shivaram Aug 4, 2017

Collaborator

Thanks all ! This is merged now

Collaborator

shivaram commented Aug 4, 2017

Thanks all ! This is merged now

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment