Update test base image and move CI to Github workflow

[3nids]

Dependency image has been moved to ubuntu focal (this means Qt 5.12)

CI is moved away from Travis to Github workflow
Caching is working (ccache)

issues:

• tests are failing
• QML needed to be turned off (probably faulty QML files) ping @PeterPetrik
• oracle doesn't seem to work, updating the instant client had no effect

List of failing tests:

• ProcessingQgisAlgorithmsTestPt1 (Failed)
• ProcessingQgisAlgorithmsTestPt2 (Failed)
• ProcessingQgisAlgorithmsTestPt3 (Failed)
• ProcessingQgisAlgorithmsTestPt4 (Failed)
• ProcessingGdalAlgorithmsRasterTest (Failed)
• ProcessingGdalAlgorithmsVectorTest (Failed)
• ProcessingGrass7AlgorithmsImageryTest (Failed)
• ProcessingGrass7AlgorithmsRasterTest (Failed)
• ProcessingGrass7AlgorithmsVectorTest (Failed)
• ProcessingSagaAlgorithmsTest (Failed)
• qgis_authmanagertest (Failed)
• qgis_coordinatereferencesystemtest (Timeout)
• qgis_geometrytest (Failed)
• qgis_geonodeconnectiontest (Failed)
• qgis_maprendererjobtest (Failed)
• qgis_networkaccessmanagertest (Timeout)
• qgis_painteffecttest (Failed)
• qgis_pallabelingtest (Failed)
• qgis_ogrproviderguitest (Timeout)
• qgis_3drenderingtest (Failed)
• qgis_processingtest (Failed)
• qgis_geometrycheckstest (Failed)
• qgis_arcgisrestutilstest (Failed)
• PyCoreAdittions (Failed)
• PyQgsActionManager (Failed)
• PyQgsPythonProvider (Failed)
• PyQgsAnnotation (Failed)
• PyQgsCoordinateOperationWidget (Failed)
• PyQgsGeometryTest (Failed)
• PyQgsMapLayerComboBox (Failed)
• PyQgsMapLayerProxyModel (Failed)
• PyQgsOGRProviderGpkg (Failed)
• PyQgsProcessExecutable (Failed)
• PyQgsProviderConnectionGpkg (Failed)
• PyQgsRulebasedRenderer (Failed)
• PyQgsShapefileProvider (Failed)
• PyQgsSvgCache (Failed)
• PyQgsSpatialiteProvider (Failed)
• PyQgsVectorFileWriter (Failed)
• PyQgsVectorLayer (Failed)
• PyQgsVirtualLayerProvider (Failed)
• PyQgsFileDownloader (Failed)
• PyQgsSettings (Failed)
• PyQgsRelationEditWidget (Failed)
• PyQgsAuthManagerPasswordPostgresTest (Failed)
• PyQgsProviderConnectionPostgres (Failed)
• PyQgsOracleProvider (Failed)
• PyQgsServerWMS (Failed)
• PyQgsServerApi (Failed)
• PyQgsServerLandingPage (Failed)
• PyQgsServerWMSGetMap (Failed)
• PyQgsServerWMSGetFeatureInfo (Failed)
• PyQgsServerWMSGetPrint (Failed)
• PyQgsServerWMSGetPrintExtra (Failed)
• PyQgsServerWMSDimension (Failed)
• PyQgsAuthManagerPKIOWSTest (Failed)
• PyQgsAuthManagerOAuth2OWSTest (Failed)

Link to current raw log

Open question: how to deal with failing tests? is there any volunteer?

[troopa81]

oracle doesn't seem to work, updating the instant client had no effect

When I took a look to the build, I see

failed to register layer: Error processing tar file(exit status 1): write /opt/oracle/product/18c/dbhomeXE/jdk/jre/lib/amd64/libjfxwebkit.so: no space left on device

Is there any sort of disk limitation on GitHub workflow. It doesn't look to an Oracle related issue

[nyalldawson]

@3nids looks like the results pushed to cdash are missing the PR identifier too and show up as just

[nyalldawson]

Well... at a preliminary glance it looks like there's ~15 of these which just need new reference images. The rest look... complicated 😱

[PeterPetrik]

ok with QML

[3nids]

@troopa81 I increased the Docker base image to 50gb (9365c55) and it went through in the push build (https://github.com/qgis/QGIS/pull/39815/checks?check_run_id=1359335494#step:11:33) but not the pull request one

[troopa81]

in the push build (https://github.com/qgis/QGIS/pull/39815/checks?check_run_id=1359335494#step:11:33) but not the pull request one

The timeout is too low (5 minutes here). It takes approximately 10 minutes on my machine for the Oracle database to be ready to accept connections... And this is without the image pulling. I assume there is a Docker image cache but since we switch to GH workflow maybe the image is not in the cache.

[3nids]

@troopa81, thanks I increased the timeout to 17 minutes

[3nids]

@troopa81 just wondering, should we start the DBs before building QGIS? that would let them time to start while we compile?

[troopa81]

@troopa81 just wondering, should we start the DBs before building QGIS? that would let them time to start while we compile?

It could be good idea indeed. I'm currently trying to make an already started oracle database image. The resulting image would be huge (around 8 GB I think) but we wouldn't have to wait for 10 minutes! Not completely sure I manage to do it though

[3nids]

I would propose to run everything in the compose (i.e. also the compilation):

1. start compose and init DBs
2. compile QGIS
3. run the tests

The backward is that we fire the DBs (a few minutes to pull the images) even if the compilation fails.
But I would say that most of the jobs are run with a succeeding compilation.

[troopa81]

@troopa81, thanks I increased the timeout to 17 minutes

Still a timeout. Is it possible to have some logs about what happen inside the Docker Oracle instance? The equivalent of docker logs

[troopa81]

I'm currently trying to make an already started oracle database image.

I finally managed to reduce the oracle start-up time from 16 minutes to 3 minutes. It's here. You could rebase or just pick the modification.

[3nids]

@troopa81, rebased, thanks a lot!

[troopa81]

failed to register layer: Error processing tar file(exit status 1): write /opt/oracle/product/18c/dbhomeXE/assistants/dbca/templates/pdbseed.dfb: no space left on device

@3nids you may have to increase disk space. The new oracle image is around 2 Go bigger than the previous one

[nyalldawson]

PyCoreAdittions
PyQgsProviderConnectionPostgres
PyQgsSettings

These are fixed by #40039

[nyalldawson]

@3nids would it be possible to commit the parts of this which add the new workflow only for now? Then we could at least avoid people introducing new test failures

[nyalldawson]

Manually marking as stale because stalebot has left for christmas break

[github-actions]

The QGIS project highly values your contribution and would love to see this work merged! Unfortunately this PR has not had any activity in the last 14 days and is being automatically marked as "stale". If you think this pull request should be merged, please check

• that all unit tests are passing

• that all comments by reviewers have been addressed

• that there is enough information for reviewers, in particular

  • link to any issues which this pull request fixes

  • add a description of workflows which this pull request fixes

  • add screenshots if applicable

• that you have written unit tests where possible
  In case you should have any uncertainty, please leave a comment and we will be happy to help you proceed with this pull request.
  If there is no further activity on this pull request, it will be closed in a week.

[github-actions]

While we hate to see this happen, this PR has been automatically closed because it has not had any activity in the last 21 days. If this pull request should be reconsidered, please follow the guidelines in the previous comment and reopen this pull request. Or, if you have any further questions, just ask! We love to help, and if there's anything the QGIS project can do to help push this PR forward please let us know how we can assist.