|
| 1 | +QGIS Docker images |
| 2 | +================== |
| 3 | + |
| 4 | +The QGIS project provides a few official docker images that can be |
| 5 | +used for testing purposes. |
| 6 | + |
| 7 | +These dockers are currently used to run continuous integration |
| 8 | +tests for the QGIS project itself and to run continuous integration |
| 9 | +tests for several third party Python plugins. |
| 10 | + |
| 11 | +The images are automatically built every day and pushed on docker hub |
| 12 | +to the QGIS account: https://hub.docker.com/r/qgis/ |
| 13 | + |
| 14 | +# Available images |
| 15 | + |
| 16 | +## Dependencies image |
| 17 | + |
| 18 | +`qgis/qgis3-build-deps` |
| 19 | + |
| 20 | +This is a simple base image that contains all the dependencies required to build |
| 21 | +QGIS, it is used by the other images. |
| 22 | + |
| 23 | +Multiple versions of this image may be available: the suffix in the image name indicates the Ubuntu version they are based on. |
| 24 | + |
| 25 | +## Main QGIS image |
| 26 | + |
| 27 | +`qgis/qgis` |
| 28 | + |
| 29 | +This is the main image containing a build of QGIS. |
| 30 | + |
| 31 | +The docker tags for this image are assigned for each point release (prefixed with `final-`), for the active development branches (prefixed with `release-`) while the `latest` tag refers to a build of the current master branch. |
| 32 | + |
| 33 | + |
| 34 | +### Features |
| 35 | + |
| 36 | +The docker file builds QGIS from the current directory and |
| 37 | +sets up a testing environment suitable for running tests |
| 38 | +inside QGIS. |
| 39 | + |
| 40 | +You can use this docker image to test QGIS and/or to run unit tests inside |
| 41 | +QGIS, `xvfb` (A fake X server) is available and running as a service inside |
| 42 | +the container to allow for fully automated headless testing in CI pipelines |
| 43 | +such as Travis or Circle-CI. |
| 44 | + |
| 45 | +### Building |
| 46 | + |
| 47 | +You can build the image from the main directory of the QGIS source tree with: |
| 48 | + |
| 49 | +``` |
| 50 | +$ docker build -t qgis/qgis:latest \ |
| 51 | + --build-arg DOCKER_TAG=latest \ |
| 52 | + -f .docker/qgis.dockerfile \ |
| 53 | + . |
| 54 | +``` |
| 55 | + |
| 56 | +The `DOCKER_TAG` argument, can be used to specify the tag of the dependencies image. |
| 57 | + |
| 58 | + |
| 59 | +### Running QGIS |
| 60 | + |
| 61 | +You can also use this image to run QGIS on your desktop. |
| 62 | + |
| 63 | +To run a QGIS container, assuming that you want to use your current |
| 64 | +display to use QGIS and the image is tagged `qgis/qgis:latest` you can use a script like the one here below: |
| 65 | + |
| 66 | +```bash |
| 67 | +# Allow connections from any host |
| 68 | +$ xhost + |
| 69 | +$ docker run --rm -it --name qgis \ |
| 70 | + -v /tmp/.X11-unix:/tmp/.X11-unix \ |
| 71 | + -e DISPLAY=unix$DISPLAY \ |
| 72 | + qgis/qgis:latest qgis |
| 73 | +``` |
| 74 | + |
| 75 | +This code snippet will launch QGIS inside a container and display the |
| 76 | +application on your screen. |
| 77 | + |
| 78 | +### Running unit tests inside QGIS |
| 79 | + |
| 80 | +Suppose that you have local directory containing the tests you want to execute into QGIS: |
| 81 | + |
| 82 | +``` |
| 83 | +/my_tests/travis_tests/ |
| 84 | +├── __init__.py |
| 85 | +└── test_TravisTest.py |
| 86 | +``` |
| 87 | + |
| 88 | +To run the tests inside the container, you must mount the directory that |
| 89 | +contains the tests (e.g. your local directory `/my_tests`) into a volume |
| 90 | +that is accessible by the container, see `-v /my_tests/:/tests_directory` |
| 91 | +in the example below: |
| 92 | + |
| 93 | +```bash |
| 94 | +$ docker run -d --name qgis -v /tmp/.X11-unix:/tmp/.X11-unix \ |
| 95 | + -v /my_tests/:/tests_directory \ |
| 96 | + -e DISPLAY=:99 \ |
| 97 | + qgis/qgis:latest |
| 98 | +``` |
| 99 | + |
| 100 | +Here is an extract of `test_TravisTest.py`: |
| 101 | + |
| 102 | +```python |
| 103 | +# -*- coding: utf-8 -*- |
| 104 | +import sys |
| 105 | +from qgis.testing import unittest |
| 106 | + |
| 107 | +class TestTest(unittest.TestCase): |
| 108 | + |
| 109 | + def test_passes(self): |
| 110 | + self.assertTrue(True) |
| 111 | + |
| 112 | +def run_all(): |
| 113 | + """Default function that is called by the runner if nothing else is specified""" |
| 114 | + suite = unittest.TestSuite() |
| 115 | + suite.addTests(unittest.makeSuite(TestTest, 'test')) |
| 116 | + unittest.TextTestRunner(verbosity=3, stream=sys.stdout).run(suite) |
| 117 | + |
| 118 | +``` |
| 119 | + |
| 120 | +When done, you can invoke the test runnner by specifying the test |
| 121 | +that you want to run, for instance: |
| 122 | + |
| 123 | +``` |
| 124 | +$ docker exec -it qgis sh -c "cd /tests_directory && qgis_testrunner.sh travis_tests.test_TravisTest.run_fail" |
| 125 | +
|
| 126 | +``` |
| 127 | + |
| 128 | +The test can be specified by using a dotted notation, similar to Python |
| 129 | +import notation, by default the function named `run_all` will be executed |
| 130 | +but you can pass another function name as the last item in the dotted syntax: |
| 131 | + |
| 132 | +```bash |
| 133 | +# Call the default function "run_all" inside test_TravisTest module |
| 134 | +qgis_testrunner.sh travis_tests.test_TravisTest |
| 135 | +# Call the function "run_fail" inside test_TravisTest module |
| 136 | +qgis_testrunner.sh travis_tests.test_TravisTest.run_fail |
| 137 | +``` |
| 138 | + |
| 139 | +Please note that in order to make the test script accessible to Python |
| 140 | +the calling command must ensure that the tests are in Python path. |
| 141 | +Common patterns are: |
| 142 | +- change directory to the one containing the tests (like in the examples above) |
| 143 | +- add to `PYTHONPATH` the directory containing the tests |
| 144 | + |
| 145 | +#### Running tests for a Python plugin |
| 146 | + |
| 147 | +All the above considerations applies to this case too, however in order |
| 148 | +to simulate the installation of the plugin inside QGIS, you'll need to |
| 149 | +make an additional step: call `qgis_setup.sh <YourPluginName>` in the |
| 150 | +docker container before actually running the tests (see the paragraph |
| 151 | +about Running on Travis for a complete example). |
| 152 | + |
| 153 | +The `qgis_setup.sh` script prepares QGIS to run in headless mode and |
| 154 | +simulate the plugin installation process: |
| 155 | + |
| 156 | +- creates the QGIS profile folders |
| 157 | +- "installs" the plugin by making a symbolic link from the profiles folder to the plugin folder |
| 158 | +- installs `startup.py` monkey patches to prevent blocking dialogs |
| 159 | +- enables the plugin |
| 160 | + |
| 161 | +Please note that depending on your plugin repository internal directory structure |
| 162 | +you may need to adjust (remove and create) the symbolic link created by `qgis_setup.sh`, |
| 163 | +this is required in particular if the real plugin code in your repository is contained |
| 164 | +in the main directory and not in a subdirectory with the same name of the plugin |
| 165 | +internal name (the name in `metadata.txt`). |
| 166 | + |
| 167 | +#### Options for the test runner |
| 168 | + |
| 169 | +The env var `QGIS_EXTRA_OPTIONS` defaults to an empty string and can |
| 170 | +contains extra parameters that are passed to QGIS by the test runner. |
| 171 | + |
| 172 | + |
| 173 | +#### Running on Travis |
| 174 | + |
| 175 | +Here is a simple example for running unit tests of a small QGIS plugin (named *QuickWKT*), assuming that the tests are in `tests/test_Plugin.py` under |
| 176 | +the main directory of the QuickWKT plugin: |
| 177 | + |
| 178 | +```yml |
| 179 | +services: |
| 180 | + - docker |
| 181 | +install: |
| 182 | + - docker run -d --name qgis-testing-environment -v ${TRAVIS_BUILD_DIR}:/tests_directory -e DISPLAY=:99 qgis/qgis:latest |
| 183 | + - sleep 10 # This is required to allow xvfb to start |
| 184 | + # Setup qgis and enables the plugin |
| 185 | + - docker exec -it qgis-testing-environment sh -c "qgis_setup.sh QuickWKT" |
| 186 | + # Additional steps (for example make or paver setup) here |
| 187 | + # Fix the symlink created by qgis_setup.sh |
| 188 | + - docker exec -it qgis-testing-environment sh -c "rm -f /root/.local/share/QGIS/QGIS3/profiles/default/python/plugins/QuickWKT" |
| 189 | + - docker exec -it qgis-testing-environment sh -c "ln -s /tests_directory/ /root/.local/share/QGIS/QGIS3/profiles/default/python/plugins/QuickWKT" |
| 190 | + |
| 191 | +script: |
| 192 | + - docker exec -it qgis-testing-environment sh -c "cd /tests_directory && qgis_testrunner.sh tests.test_Plugin" |
| 193 | +``` |
| 194 | +
|
| 195 | +Please note that `cd /tests_directory && ` before the call to `qgis_testrunner.sh` could be avoided here, because QGIS automatically |
| 196 | +adds the plugin main directory to Python path. |
| 197 | + |
| 198 | + |
| 199 | +#### Running on Circle-CI |
| 200 | + |
| 201 | +Here is an example for running unit tests of a small QGIS plugin (named *QuickWKT*), assuming |
| 202 | +that the tests are in `tests/test_Plugin.py` under the main directory of the QuickWKT plugin: |
| 203 | + |
| 204 | +```yml |
| 205 | +version: 2 |
| 206 | +jobs: |
| 207 | + build: |
| 208 | + docker: |
| 209 | + - image: qgis/qgis:latest |
| 210 | + environment: |
| 211 | + DISPLAY: ":99" |
| 212 | + working_directory: /tests_directory |
| 213 | + steps: |
| 214 | + - checkout |
| 215 | + - run: |
| 216 | + name: Setup plugin |
| 217 | + command: | |
| 218 | + qgis_setup.sh QuickWKT |
| 219 | + - run: |
| 220 | + name: Fix installation path created by qgis_setup.s |
| 221 | + command: | |
| 222 | + rm -f /root/.local/share/QGIS/QGIS3/profiles/default/python/plugins/QuickWKT |
| 223 | + ln -s /tests_directory/ /root/.local/share/QGIS/QGIS3/profiles/default/python/plugins/qgisce |
| 224 | + - run: |
| 225 | + name: run tests |
| 226 | + command: | |
| 227 | + sh -c "/usr/bin/Xvfb :99 -screen 0 1024x768x24 -ac +extension GLX +render -noreset -nolisten tcp &" |
| 228 | + qgis_testrunner.sh tests.test_Plugin |
| 229 | +``` |
| 230 | + |
| 231 | + |
| 232 | +#### Implementation notes |
| 233 | + |
| 234 | +The main goal of the test runner in this image is to execute unit tests |
| 235 | +inside a real instance of QGIS (not a mocked one). |
| 236 | + |
| 237 | +The QGIS tests should be runnable from a Travis/Circle-CI CI job. |
| 238 | + |
| 239 | +The implementation is: |
| 240 | + |
| 241 | +- run the docker, mounting as volumes the unit tests folder in `/tests_directory` |
| 242 | + (or the QGIS plugin folder if the unit tests belong to a plugin and the |
| 243 | + plugin is needed to run the tests) |
| 244 | +- execute `qgis_setup.sh MyPluginName` script in docker that sets up QGIS to |
| 245 | + avoid blocking modal dialogs and installs the plugin into QGIS if needed |
| 246 | + - create config and python plugin folders for QGIS |
| 247 | + - enable the plugin in the QGIS configuration file |
| 248 | + - install the `startup.py` script to disable python exception modal dialogs |
| 249 | +- execute the tests by running `qgis_testrunner.sh MyPluginName.tests.tests_MyTestModule.run_my_tests_function` |
| 250 | +- the output of the tests is captured by the `test_runner.sh` script and |
| 251 | + searched for `FAILED` (that is in the standard unit tests output) and other |
| 252 | + string that indicate a failure or success condition, if a failure condition |
| 253 | + is identified, the script exits with `1` otherwise it exits with `0`. |
| 254 | + |
| 255 | +`qgis_testrunner.sh` accepts a dotted notation path to the test module that |
| 256 | +can end with the function that has to be called inside the module to run the |
| 257 | +tests. The last part (`.run_my_tests_function`) can be omitted and defaults to |
| 258 | +`run_all`. |
0 commit comments