Add coverage with coveralls and make use of travis stages

[opacam]

To achieve the desired result we also modify the travis file, Implementing travis stages which are:

• Lint (our tox tests runs in here and also we collect the coverage data)
• Test (our build tests, which are exactly the same than the ones we have been using for a while, but now those are labelled)

Make use of the stages allow us to sequentially run some tests (without loosing the parallel builds, so the Test stage will be runned in parallel mode...the same behaviour than we have right now). Be aware that if the Lint stage fails, then the Test stage will never be run. This feature has been implemented in here in order to only send the coverage reports in case that we succeed with Lint stage.

This is friendly with the idea to implement some kind of automation to auto release/publish p4a versions (see issue/discussion #1833). We could implement it in a new stage (eg: deploy), to build docs, release a new version when master branch is tagged, upload it to pypi....

As a side note: notice that we currently perform our tox tests 4 times (one for each test we make)...with this pr we only make the tox tests one time (Lint stage), so this should improve the timings with our ci tests

Notes:

• I took myself the liberty to enable coveralls for python-for-android (not only for my fork, also enabled for the main project kivy/python-for-android, if this pr is not merged, then we should disable it)
• The added badge in README.md file should be automatically updated to the right value at the time of merging this in master branch
• The coverage is configured to only track the pythonforandroid module
• The coverage report is not totally accurate because we skip some tests in travis

Solves: #1788

Coverage report at: https://coveralls.io/jobs/49254277

[inclement]

Sounds good to me, but I'll let @AndreMiras check the travis details :)

[AndreMiras]

Sounds awesome! I'll try to review tonight, otherwise tomorrow.
Thanks!

[AndreMiras]

I'm not aware of this syntax and also the <<: *testing below. Could you share me a doc to read or a keyword to look for?

[opacam]

Oh yes, no problem, I learned this syntax for this pr, they are called Yaml Anchors & Aliases and are great to create templates, I discovered this in this article:

• Optimizing Mocha's Builds with Travis

Also mentioned in Travis docs examples
And here some more information: atlassian - yaml anchors

[opacam]

The yaml Anchors & Aliases that I wrote works the following way:

First we define a template named testing using the keywords &testing

    - &testing
      stage: test
      before_script:
        # build docker image
        - docker build --tag=p4a --file Dockerfile.py3 .
        # Run a background process to make sure that travis will not kill our tests in
        # case that the travis log doesn't produce any output for more than 10 minutes
        - while sleep 540; do echo "==== Still running (travis, don't kill me) ===="; done &
      script:
        - docker run -e CI -e TRAVIS_JOB_ID="$TRAVIS_JOB_ID" -e TRAVIS_BRANCH="$TRAVIS_BRANCH" p4a /bin/sh -c "$COMMAND"
      after_script:
        # kill the background process started before run docker
        - kill %1

The following code will take the above defined template, actually will also be part of the template but we override later for the other jobs:

      name: Python 3 basic
      # overrides requirements to skip `peewee` pure python module, see:
      # https://github.com/kivy/python-for-android/issues/1263#issuecomment-390421054
      env: COMMAND='. venv/bin/activate && cd testapps/ && python setup_testapp_python3_sqlite_openssl.py apk --sdk-dir $ANDROID_SDK_HOME --ndk-dir $ANDROID_NDK_HOME --requirements libffi,sdl2,pyjnius,kivy,python3,openssl,requests,sqlite3,setuptools'

For the following jobs we want to use the same scripts defined in &testing so we use the keywords <<:*testing but we want to override the name and env defined in the template so we do the following:

    - <<: *testing
      name: Python 2 basic
      env: COMMAND='. venv/bin/activate && cd testapps/ && python setup_testapp_python2_sqlite_openssl.py apk --sdk-dir $ANDROID_SDK_HOME --ndk-dir $ANDROID_NDK_HOME --requirements sdl2,pyjnius,kivy,python2,openssl,requests,sqlite3,setuptools'

The same for the other jobs...

[AndreMiras]

Thanks for the explanation makes sense 👍
Our Travis file start to get more complex, but that's the way it is I guess.
Dropping python2 at some point may help slightly

[AndreMiras]

Awesome work, very useful.
Now we need to increase the coverage 😛

[opacam]

Yeah, I'm working on that...I'm writing unittests for a couple of uncovered modules (archs and bootstrap) I almost have it...I will create proper pr when ready 😉