Adding more checks

[hroncok]

I'd like to add more checks:

• Is the package following the new naming scheme?
• Does it use requires using the new scheme?
• Is Python 3 supported upstream but not in the package?
• Does only Python 2 version contain the executables?
• Shebangs checks

@kparal @mkrizek What is the right approach here, extending this repo or creating new task for each problem? It's all related to "Python versions". Thanks

[mkrizek]

Let's go with extending this repo and using different check name for each check when reporting results (e.g. dist.python-versions.new-naming-scheme). Look at task-rpmgrill [1] [2] for inspiration. Let us know if you need any help.

[1] http://taskotron-dev.fedoraproject.org/resultsdb/results?groups=12fc647c-3024-11e7-8192-525400cb0b45
[2] https://bitbucket.org/fedoraqa/task-rpmgrill

[kparal]

I think it's really up to you and what you think will be the most helpful for your consumers (i.e. package maintainers reading the results in Bodhi). You can have just a single overall pass/fail and people need to click on it to see the log and read the issues. Or you can report subresults like @mkrizek mentioned. The first is more succinct and occupying less space (showing less clutter) in bodhi, the second is faster to see what the issue was from the overall view in bodhi. As long as there's just a couple of tests (questions to answer), I'd go with subresults. Once you have ten or twenty of them, you might rather group them together (think rpmlint, you wouldn't want to see a separate result for each of the hundred tests it performs, it's easier to see the overall one and then read the log). The overall result is always useful, even when having subresults, so something like:

dist.python-versions FAILED
dist.python-versions.naming PASSED
dist.python-versions.py3upstream FAILED
dist.python-versions.exec PASSED

[hroncok]

Grounds for the separation to subtasks are in #8