Skip to content
Permalink
Browse files

Copy updated test runner into the docker

Handle -e and do not exit on error
  • Loading branch information
elpaso committed Nov 19, 2018
1 parent e8e66d0 commit ed9709b4faa7b565ee75b7298e4b02069bcaf994
Showing with 24 additions and 17 deletions.
  1. +17 −12 .ci/travis/linux/script.sh
  2. +5 −0 python/testing/__init__.py
  3. +2 −5 tests/src/python/test_testrunner.py
@@ -35,17 +35,22 @@ if [[ ${DOCKER_BUILD_QGIS_IMAGE} =~ true ]]; then
popd
else
# running QGIS tests
#docker-compose -f ${TRAVIS_BUILD_DIR}/.docker/docker-compose.travis.yml run --rm qgis-deps
docker-compose -f ${TRAVIS_BUILD_DIR}/.docker/docker-compose.travis.yml run --rm qgis-deps

# running tests for the python test runner
docker run -d --name qgis-testing-environment -v ${TRAVIS_BUILD_DIR}:/tests/src/python -e DISPLAY=:99 "qgis/qgis:${DOCKER_TAG}"
docker exec -it qgis-testing-environment sh -c "qgis_testrunner.sh test_testrunner"
[ $? -ne 0 ] # expected failure
docker exec -it qgis-testing-environment sh -c "qgis_testrunner.sh test_testrunner.run_all"
[ $? -ne 0 ] # expected failure
docker exec -it qgis-testing-environment sh -c "qgis_testrunner.sh test_testrunner.run_failing"
[ $? -ne 0 ] # expected failure
docker exec -it qgis-testing-environment sh -c "qgis_testrunner.sh test_testrunner.run_passing"
[ $? -eq 0 ] # expected pass
docker exec -it qgis-testing-environment sh -c "qgis_testrunner.sh test_testrunner.run_skipped_and_passing"
[ $? -eq 0 ] # expected pass
docker run -d --name qgis-testing-environment -v ${TRAVIS_BUILD_DIR}/tests/src/python:/tests_directory -e DISPLAY=:99 "qgis/qgis:${DOCKER_TAG}"

This comment has been minimized.

Copy link
@3nids

3nids Mar 5, 2019

Member

@elpaso this command is actually pulling the last image build for QGIS. Which is older than the code in the current job. You should be using the image that has been built in the docker compose (I am not sure how to do it).
This is also what makes failing jobs on new branches on Travis.
While the current situation could be acceptable, I think it should be fixed:

  • to detect issue in the proper job and not a few commits later (and nobody will understand)
  • to avoid losing time at downloading an image
sleep 10 # Wait for xvfb to finish starting
# Temporary workaround until docker images are built
docker cp ${TRAVIS_BUILD_DIR}/.docker/qgis_resources/test_runner/qgis_testrunner.sh qgis-testing-environment:/usr/bin/qgis_testrunner.sh
docker exec -it qgis-testing-environment sh -c "cd /tests_directory && qgis_testrunner.sh test_testrunner.run_passing"
docker exec -it qgis-testing-environment sh -c "cd /tests_directory && qgis_testrunner.sh test_testrunner.run_skipped_and_passing"
# Failing cases:
set +e
ret=0 && docker exec -it qgis-testing-environment sh -c "cd /tests_directory && qgis_testrunner.sh test_testrunner" || ret=127
[ $ret -eq 127 ] || exit 1 # expected failure
ret=0 && docker exec -it qgis-testing-environment sh -c "cd /tests_directory && qgis_testrunner.sh test_testrunner.run_all" || ret=127
[ $ret -eq 127 ] || exit 1 # expected failure
ret=0 && docker exec -it qgis-testing-environment sh -c "cd /tests_directory && qgis_testrunner.sh test_testrunner.run_failing" || ret=127
[ $ret -eq 127 ] || exit 1 # expected failure
set -e
fi
@@ -394,6 +394,11 @@ def start_app(cleanup=True):
except NameError:
myGuiFlag = True # All test will run qgis in gui mode

try:
sys.argv
except:
sys.argv = ['']

# In python3 we need to convert to a bytes object (or should
# QgsApplication accept a QString instead of const char* ?)
try:
@@ -16,12 +16,9 @@
import qgis # NOQA
import sys

from qgis.testing import unittest, start_app
from qgis.testing import unittest
from console import console
from qgis.core import Qgis
from qgis.PyQt.QtCore import QCoreApplication

start_app()


class TestTestRunner(unittest.TestCase):
@@ -57,7 +54,7 @@ def run_failing():
return _make_runner(['test_fails'])


def run_passes():
def run_passing():
"""Run passing test only"""
return _make_runner(['test_passes'])

0 comments on commit ed9709b

Please sign in to comment.
You can’t perform that action at this time.