WArchive Software Quality Award
See http://www.yegor256.com/award.html
Summary:
- Contest in which a project can win a price of $4096
- Submissions are accepted until September 1, 2015
Each project must be:
- Open source.
- At least 5,000 lines of code.
- At least one year old. (the beginnings of coala age much earlier than one year including the rewrites)
- Object-oriented (that's the only thing I understand).
The best projects will feature:
- Strict and visible principles of design.
- Continuous delivery.
- Traceability of changes.
- Self-documented source code.
- Strict rules of code formatting.
Why not?
Hi,
I want to submit my project coala [0] for your Software Quality Award.
The reason why I'm eager to do this is that coala is my personal effort to make the world of coding a better one - qualitatively.
My GitHub username is sils1297, the project is coala-analyzer/coala. As I am trying not to be a bus factor of one for this project I moved it into an own organization.
So let's come to the content :) What are we doing to keep our own quality high: most things we do are tied to continuous integration so developers don't have to care and human errors (like forgetting things...) are avoided.
1. Test automation
We have continuous integration for all supported platforms and python versions.
There are seven builds running all our tests for each commit on a PR/branch:
* Python 3.2-3.4 on Linux via Travis
* Python 3.3, 3.4 32/64 bit on Windows via AppVeyor
* Mac is planned as soon as Travis gains support for it.
If anything fails the commit may not go on master. This toolchain works fully automatic for all pull requests on GitHub. Having several very different build servers in terms of platform or speed helped us finding even many nondeterministic bugs in the past.
We have another linux build that is much faster and can be ssh'd into so developers can debug fast if problems occur only on CI servers.
2. A sane workflow
We don't do merges. A merge commit is a commit that can hardly be reviewed and is not in every case checked by continuous integration. Commits first get validated and then the master gets fast-forwarded to a commit that is known to be valid, not a new merge commit. Validating a commit once it is on master is just too late.
3. Reviews
Every commit gets reviewed by a human which is not the author of the commit. After a rebase every commit needs at least to get a short reack from someone after investigating if the changes done on master affect the commits.
Reviews are done to the standards written down at [2] to ensure that changes are traceable and help in bug investigation. With such standards bugs can easily be found after writing a short test to detect them with the `git bisect run` command. The test is needed anyway as mentioned in the next point.
Review documentation can be found at [3].
3.1. Codestyle
To ensure consistent and good readable code we've written down codestyle guidelines at [4]. Although this document is not completely finished yet it is already quite long and covers the most important things.
4. Coverage measurements
Continuous integration can only catch bugs covered by the tests. Our tests must execute each line of code which is not explicitly marked as not to cover (e.g. platform specific branches in code) in order to pass the build status on GitHub. If a new partial covered line is added this will automatically show up on the pull request and will thus be exposed immediately.
Coverage is measured on linux with python 3.4. We have set up coveralls and coverage.io to get a "broken" build status if anything goes down.
5. Code Quality Services
We do use Codacy for a few basic linting checks. If Codacy yields any issues the build is automatically marked as failed. As a more advanced supplement we use Scrutinizer which yields some false positives at times and thus is not required to not yield issues.
The last quality assurance is coala itself. After the software we write is meant to "review" source code, coala will be run on the coala for each build and if any issues occur the build will fail.
6. Documentation
We have manually written documentation for users as well as developers. This documentation is automatically built from GitHub master.
Methods and classes are documented as needed. Many small and self-explanatory methods are undocumented, many big and not self-explanatory methods and classes are documented. Probably there are some methods which need documentation, if you find one, please open a bug. Those things usually get fixed fairly fast, review is what takes longest. This is an area where we currently are working on improvements.
7. Translation
Keep automation high. Keep maintenance low. Keep avoiding human errors. Our continuous integration servers automatically push the newest strings from master to zanata so translators only work with the most recent and stable strings. See [5] for more info. Pulling down translations is only needed for releasing and done by a script which generates commits.
Translations are reviewable on zanata as it supports states (untranslated, needs-work, translated, reviewed).
8. Code Architecture
At coala we care a lot about having a sane code architecture. Actually we rewrote the whole project two times completely from ground up because we were not satisfied with the design.
Above all we try to keep code more generic than we need it. There will come a time when this will be needed. For most objects this already paid off more than once. We are not afraid of rewriting large parts of the code base, currently we're overthinking our Interactor design (a class that tries to abstract user interaction) and will refactor that in the next few weeks although many classes rely on it currently. As we have a full test coverage refactoring can be done without the common danger of unknown trivial regressions.
Another big refactor lying ahead is finally catching up with the evil "-er" objects. See [6] for more info. This did grow historically because at first we wanted to put everything in an object, however this philosophy has changed but the refactors have not been finished yet as they are large.
Singletons are evil global variables and not allowed.
coala is generic modular by design in its code and purpose. Every analysis coala does is a module which can do anything on any unicode file. See [7] for more information on how analysis is done and how anyone can do it easily.
We usually don't look for the most performant solution but the cleanest ones. Code bases are getting bigger and more complex for sure. PCs are getting faster for sure. So obviously what matters in the end is definitely not one statement executed too much but simplicity. After all programming is the art of simplification.
9. The Will to Improve
At coala we know that we cannot reach the end of the road. We try to raise the awareness about our own code quality problems (a few listed in this email). This knowledge is the most important aspect as it enables us to improve our code quality. As we add features our codebase will probably grow and we need not only to maintain our current level of code quality but also we need to *improve* the quality to keep bugs away from the user.
10. Usage
Currently coala has no funds at its disposal. It does receive some indirect funding through the Google Summer of Code via GNOME as an umbrella organization. In the past I have spent some limited amount of own money to be able to keep pushing coala forward. It would be very helpful to have some limited funds to be able to push this project forward, cover minor costs like having an own domain, set up bug and review bounties and for the project to not rely on my private money as I am a student with very limited resources maintaining this project fully in my free time.
I hope all this gave some impression of what we do and how we do it. I'd love to get some response - we also like if someone doesn't like some aspect of our code so we know where we can improve :). Feel free to drop in to our IRC replacement on [8] to meet me and the community. I would be personally very curious to meet you.
Greetings,
Lasse Schuirmann
GitHub/Gitter: sils1297
GNOME Foundation Member
GNOME GSoC Admin and Mentor
[0] https://github.com/coala-analyzer/coala
[1] http://coala.readthedocs.org/en/latest/README/
[2] http://coala.readthedocs.org/en/latest/Getting_Involved/Writing_Good_Commits/
[3] http://coala.readthedocs.org/en/latest/Getting_Involved/Review/
[4] http://coala.readthedocs.org/en/latest/Getting_Involved/Codestyle/
[5] http://coala.readthedocs.org/en/latest/Getting_Involved/Translating/
[6] https://github.com/coala-analyzer/coala/issues/508
[7] http://coala.readthedocs.org/en/latest/Getting_Involved/Writing_Bears/
[8] https://gitter.im/coala-analyzer/coala
coala