Use Travis CI to run tests on every pull request #1336
Conversation
The owner of the this repo would need to go to https://travis-ci.org/profile and flip the repository switch __on__ to enable free automated flake8 testing of each pull request. Use flake8 to find Python syntax errors and undefined names. There are Python 3 syntax errors and many undefined names which may raise NameError at runtime. Flake8 runs in two passes: The first looks at critical issues in stop-the-build mode and the second looks at style violations in everything-is-a-warning mode.
|
@nodejs/gyp @bnoordhuis @maclover7 @richardlau @gibfahn @Fishrock123 @vsemozhetbyt @refack Can we please get Travis CI tuned on for this project like other Node projects? There are existing tests and linting for JavaScript (#1497) and Python (this PR) that should be run on all pull requests before they are reviewed. |
|
This is a policy decision for @nodejs/node-gyp to take (I'm not a comitter on this project). There is an existing https://ci.nodejs.org/job/nodegyp-test-pull-request on Node.js' own CI, but I think it currently only runs tests and not linting. |
That could be easily fixed by having the flake8 command be part of an I'm not a fan of this as it stands, as people will think that the test suite has been run successfully, when in fact only the linter has run. It's also a step that has to be run manually by contributors.
@cclauss what exactly is your goal here? If it's to have flake8 run on test suites, then my suggestion above is the quickest way to do that. If it's to enable travis for node-gyp, then that is a different discussion.
Why should they be run on all pull requests before they are reviewed? As long as they are run before the PR is landed, a linter change is unlikely to be the difference between landing and closing a PR. |
|
The entire trend of CI/CD is about using automation to test early and test often. It questions the value of having humans review code before it has gone through some level of automated testing. As example, let’s consider #1333 and #1334 where our code could be raising NameError exceptions at runtime. These PRs have been open for eight months with no comment. If flake8 was turned on and automated reports were seen by everybody who were trying to improve the code then many of them would put a 👍 on these PRs and they would have been merged months ago. Sometime over the next 500 days this repo either needs to be ported to Python 3 or archived. Publicly visible automated testing is recommended for efficiently doing the former. This is PR is not attempting to be the end solution. It is merely trying to get Travis CI turned on for this repo (as it is for nodejs/node, etc.) so that we can collaborate on creating better and better tests. |
__DO NOT MERGE__: This will run the [flake8](http://flake8.pycqa.org) linter on several Nodejs repos that contain Python code to detect syntax errors and undefined names which can raise NameError at runtime. There is a discussion in nodejs/node-gyp#1336 about the importance of using tools like Travis CI to automate the discovery of code quality issues. __E901,E999,F821,F822,F823__ are the "showstopper" flake8 issues that can halt the runtime with a SyntaxError, NameError, etc. Most other flake8 issues are merely "style violations" -- useful for readability but they do not effect runtime safety. This PR therefore demonstrates such a flake8 run these Nodejs codebases. * F821: undefined name `name` * F822: undefined name `name` in `__all__` * F823: local variable `name` referenced before assignment * E901: SyntaxError or IndentationError * E999: SyntaxError -- failed to compile a file into an Abstract Syntax Tree
Thanks for the reply, but that doesn't really answer my question. If you want to add Travis testing to node-gyp, then Travis should be running If you want to add flake8 to the test suite, it should be run as part of |
|
Is there someone who could re-implement this PR in a way that satisfies @gibfahn's request above? I do not understand the npm approach well enough but this PR has been open for almost a year and I believe that we would all benefit from having some basic Python linting in place. |
|
@gibfahn Can you or someone else please mentor me through this issue? There are 42 open PRs and none of them go through any CI testing. No new commit in the past week. I opened this PR and #1337 over a year ago and one year from now Python 2 is end of life. Is it really so difficult to turn on Travis CI or any similar CI tool so that we can try to achieve Python 3 compatibility in time? |
|
I'm thinking the current version looks reasonable to me. I don't think we necessarily want to integrate flake8 into the npm test step as that would force everybody who wants to run npm test to install that through pip. If we agree that makes sense we can always do it in a follow on PR. In terms of running this on Travis before the review, we do that for Node core now, running the linter and a linux test when the PR is submitted so I think this aligns with that approach. Given that we want to make progress on nodejs/TSC#642, it seems important to me to start getting info on whether changes will break when node-gyp is used with Python3. @refack @thefourtheye what do you think? |
This makes sense to me and I am okay with the current approach as it is. @gibfahn What do you think? |
|
@gustavosizilio See #1336 (comment) for a URL to the Jenkins tests. |
|
Are there any strong arguments against activating TravisCI? Is there anything I did miss? //cc @refack @maclover7 |
This is a second attempt at nodejs#1336 which got into a bad git-state... Use flake8 to find Python syntax errors and undefined names. There are Python 3 syntax errors and many undefined names which may raise NameError at runtime. This PR runs flake8 runs in two passes: The first looks at critical issues in stop-the-build mode and the second looks at style violations in everything-is-a-warning mode.
This is a second attempt at nodejs#1336 which got into a bad git-state... Use flake8 to find Python syntax errors and undefined names. There are Python 3 syntax errors and many undefined names which may raise NameError at runtime. This PR runs flake8 runs in two passes: The first looks at critical issues in stop-the-build mode and the second looks at style violations in everything-is-a-warning mode.
This is a second attempt at nodejs#1336 which got into a bad git-state... Use flake8 to find Python syntax errors and undefined names. There are Python 3 syntax errors and many undefined names which may raise NameError at runtime. This PR runs flake8 runs in two passes: The first looks at critical issues in stop-the-build mode and the second looks at style violations in everything-is-a-warning mode.
This is a second attempt at nodejs#1336 which got into a bad git-state... Use flake8 to find Python syntax errors and undefined names. There are Python 3 syntax errors and many undefined names which may raise NameError at runtime. This PR runs flake8 runs in two passes: The first looks at critical issues in stop-the-build mode and the second looks at style violations in everything-is-a-warning mode.
This is a second attempt at #1336 which got into a bad git-state... Use flake8 to find Python syntax errors and undefined names. There are Python 3 syntax errors and many undefined names which may raise NameError at runtime. This PR runs flake8 runs in two passes: The first looks at critical issues in stop-the-build mode and the second looks at style violations in everything-is-a-warning mode. PR-URL: #1752 Reviewed-By: Rod Vagg <rod@vagg.org>
This is a second attempt at #1336 which got into a bad git-state... Use flake8 to find Python syntax errors and undefined names. There are Python 3 syntax errors and many undefined names which may raise NameError at runtime. This PR runs flake8 runs in two passes: The first looks at critical issues in stop-the-build mode and the second looks at style violations in everything-is-a-warning mode. PR-URL: #1752 Reviewed-By: Rod Vagg <rod@vagg.org>
The owner of the this repo would need to go to https://travis-ci.org/profile and flip the repository switch on to enable free automated flake8 testing of each pull request.
Use flake8 to find Python syntax errors and undefined names. There are Python 3 syntax errors and many undefined names which may raise NameError at runtime. This PR runs flake8 runs in two passes: The first looks at critical issues in stop-the-build mode and the second looks at style violations in everything-is-a-warning mode.
Output
Python 2: https://travis-ci.com/nodejs/node/jobs/135827428#L505
Python 3: https://travis-ci.com/nodejs/node/jobs/135827419#L501
Checklist
npm install && npm testpassesDescription of change