Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Motivation
Developers need to run several tools and setting these up could be a hassle.
These tools include but are not limited to setting up the environment, running style checks, running tests etc.
For different projects, these tools may have different default values.
Documentation is needed for these default values to allow users to run these tools.
But we never read documentation, so an interactive script with default values and options to run these tools in a limited scope might be useful.
Limited scope might mean only the files changed, different python or spark versions etc.
Tools like pylint, prospector etc. do a great job of giving users flexibility, but they are mostly run with the default values.
They also do not give options to run on only currently touched files -- this defintion is also broad.
This is why we might want a script to help the user do everything they need and also guide them
through the process through "--help" and autocomplete options.
It should also be easy for us to maintain such a script and users to read it to see what goes on under the hood.
Luckily for us python
argh
package usesparseargs
andargscomplete
packages to expose python functionson bash with great help and autocompletion funcitonality.
Demo
Tools
pylint
is configured for variable names as bothsnake_case
andcamelCase
camelCase
while keeping everything elsesnake_case
prospector
haspylint
and some other tests like code complexity, we can start using these laternose
tests inrun-tests.sh
after setting some environment variables and specifying defaultsRFormula
orVectorAssembler
orml.feautres
but to run bothRFormula
andVectorAssembler
we need more than one commandsrun-tests.sh
and some requirements for that are also requirements ofpylint
, all such can be included inenvsetup
Misc:
argh
generates help files from argumentsargcomplete
generates autocomplete -- some config required.