Segmentation fault in qgis_process when running PCRaster algorithms

[florisvdh]

When calling the QGIS CLI program qgis_process (which we call in R package qgisprocess) to run a PCRaster algorithm, the algorithm is run and the output is written, but in the end a segfault happens (which at the R side interrupts further handling in R and throws an error). It is specific to PCRaster, not seen for other QGIS processing providers.

I compiled and installed PCRaster from source in Linux Mint, using the Ubuntu guidelines at https://jvdkwast.github.io/qgis-processing-pcraster/. A Windows R user appeared to have similar results (r-spatial/qgisprocess#159).

I'm not sure whether this is to be solved on the PCRaster side alone, or also in qgis_process. Feel free to move this as appropriate.

Example below.

$ wget -q https://github.com/pcraster/pcraster/raw/master/data/demo/dem.map
$ 
$ myJSON=$(</dev/stdin)
{
  "inputs": {
    "INPUT": "dem.map",
    "OUTPUT": "dem_slope.map"
  }
}
$ 
$ echo "$myJSON" | qgis_process run pcraster:Slope -
/usr/lib/python3/dist-packages/qgis/utils.py:888: DeprecationWarning: The distutils package is deprecated and slated for removal in Python 3.12. Use setuptools or check PEP 632 for potential alternatives
  mod = _builtin_import(name, globals, locals, fromlist, level)
{
  "algorithm_details": {
    "can_cancel": true,
    "deprecated": false,
    "group": "Derivatives of digital elevation models",
    "has_known_issues": false,
    "help_url": null,
    "id": "pcraster:Slope",
    "name": "slope",
    "requires_matching_crs": false,
    "short_description": null,
    "tags": []
  },
  "gdal_version": "3.6.2",
  "geos_version": "3.11.1-CAPI-1.17.1",
  "inputs": {
    "INPUT": "dem.map",
    "OUTPUT": "dem_slope.map"
  },
  "log": {},
  "proj_version": "Rel. 9.1.1, December 1st, 2022",
  "provider_details": {
    "can_be_activated": true,
    "default_raster_file_extension": "map",
    "default_vector_file_extension": "gpkg",
    "is_active": true,
    "long_name": "PCRaster",
    "name": "PCRaster",
    "supported_output_raster_extensions": [
      "map"
    ],
    "supported_output_table_extensions": [
      "gpkg",
      "shp",
      "000",
      "csv",
      "dgn",
      "dxf",
      "fgb",
      "gdb",
      "geojson",
      "geojsonl",
      "geojsons",
      "gml",
      "gpx",
      "gxt",
      "ili",
      "itf",
      "json",
      "kml",
      "ods",
      "sql",
      "sqlite",
      "tab",
      "txt",
      "xlsx",
      "xml",
      "xtf"
    ],
    "supported_output_vector_extensions": [
      "gpkg",
      "shp",
      "000",
      "csv",
      "dgn",
      "dxf",
      "fgb",
      "gdb",
      "geojson",
      "geojsonl",
      "geojsons",
      "gml",
      "gpx",
      "gxt",
      "ili",
      "itf",
      "json",
      "kml",
      "ods",
      "sql",
      "sqlite",
      "tab",
      "txt",
      "xlsx",
      "xml",
      "xtf"
    ],
    "supports_non_file_based_output": false,
    "version": "QGIS PCRaster Provider version 0.2.0"
  },
  "python_version": "3.10.6",
  "qgis_code_revision": "0992b53397",
  "qgis_version": "3.30.2-'s-Hertogenbosch",
  "qt_version": "5.15.3",
  "results": {
    "OUTPUT": "dem_slope.map"
  }
Segmentation fault (core dumped)
$ 
$ ls dem_slope.map 
dem_slope.map
$ 
$ qgis_process --version
QGIS 3.30.2-'s-Hertogenbosch ''s-Hertogenbosch' (0992b53397)
QGIS code revision 0992b53397
Qt version 5.15.3
Python version 3.10.6
GDAL/OGR version 3.6.2
PROJ version 9.1.1
EPSG Registry database version v10.076 (2022-08-31)
GEOS version 3.11.1-CAPI-1.17.1
SQLite version 3.37.2
OS Linux Mint 21.1
$ 
$ python
Python 3.10.6 (main, Mar 10 2023, 10:55:28) [GCC 11.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import pcraster as pcr
>>> pcr.__version__
'4.5.0-dev'
>>> exit()
$ 
$ echo $PYTHONPATH
:/usr/local/python
$ 
$ which qgis_process
/usr/bin/qgis_process
$ 
$ apt policy qgis
qgis:
  Installed: 1:3.30.2+36jammy-ubuntugis
  Candidate: 1:3.30.2+36jammy-ubuntugis
  Version table:
 *** 1:3.30.2+36jammy-ubuntugis 500
        500 https://qgis.org/ubuntugis jammy/main amd64 Packages
        100 /var/lib/dpkg/status
     3.22.16+dfsg-1~jammy3 500
        500 http://ppa.launchpad.net/ubuntugis/ubuntugis-unstable/ubuntu jammy/main amd64 Packages
     3.22.4+dfsg-3build1 500
        500 http://ftp.belnet.be/ubuntu jammy/universe amd64 Packages
$ 

[jvdkwast]

@OliverSchmitz Is this this something on the PCRaster side?

[OliverSchmitz]

I'll have a look and try to reproduce

[OliverSchmitz]

I never used qgis_process, is this supposed to just work? on Debian testing I only get this

$ qgis_process
QSocketNotifier: Can only be used with threads started with QThread
Command --noversioncheck not known!
$

does something else need to be installed or set up?

[florisvdh]

Yes, should just work. Entering qgis_process without parameters should print its help to stdout. You can also do qgis_process --help.

The Qt message may not be directly related to qgis_process. Perhaps try running as:

env QT_QPA_PLATFORM='offscreen' qgis_process

If no success, can you print the output of apt policy qgis?

[florisvdh]

@OliverSchmitz any progress on getting qgis_process to work with PCRaster? The qgis_process shell command comes bundled with QGIS.

[OliverSchmitz]

Thanks for the reminder. We are mainly busy with preparing a new PCRaster release that includes more checks and fixes, and hopefully resolves segfaults...
regarding qgis_process

$ which qgis_process
/usr/bin/qgis_process

$ apt policy qgis
qgis:
  Installed: 3.28.9+dfsg-1+b1
  Candidate: 3.28.9+dfsg-1+b1
  Version table:
 *** 3.28.9+dfsg-1+b1 500
        500 http://debian.snt.utwente.nl/debian testing/main amd64 Packages
        100 /var/lib/dpkg/status
        
$ qgis_process -- help
Command --noversioncheck not known!

I also tried the QT_QPA_PLATFORM, doesn't help either. PYTHONPATH is not set

[OliverSchmitz]

I can try a conda qgis if that helps...

[florisvdh]

Thanks – if indeed it can be solved on the PCRaster side that would be great!

AFAIK it should not need PYTHONPATH. At least not for the basic commands.

$ qgis_process -- help
Command --noversioncheck not known!

This seems like a bug in itself, somehow tied to your Debian system. Never seen it before, but I only test Ubuntu for the Linux platform. QGIS runs automated tests on Ubuntu and Fedora it seems. IMO best to post this finding as such to QGIS.

Indeed you can use another QGIS install to try and get qgis_process working, potentially some image/container.

[OliverSchmitz]

With a conda QGIS qgis_process works, and I get a segfault...

[florisvdh]

I'm glad you can reproduce the error 🙂.

[OliverSchmitz]

I just pushed a fix that should resolve this. Perhaps you can try to rebuild PCRaster and check? Thanks!

[florisvdh]

You solved it @OliverSchmitz 🚀!! Thank you very much. R package qgisprocess can now use PCRaster through QGIS!

$ qgis_process run pcraster:Slope --INPUT=dem.map --OUTPUT=dem_slope.map

----------------
Inputs
----------------

INPUT:	dem.map
OUTPUT:	dem_slope.map


/usr/lib/python3/dist-packages/qgis/utils.py:888: DeprecationWarning: The distutils package is deprecated and slated for removal in Python 3.12. Use setuptools or check PEP 632 for potential alternatives
  mod = _builtin_import(name, globals, locals, fromlist, level)

----------------
Results
----------------

OUTPUT:	dem_slope.map
$ 
$ ls dem_slope.map 
dem_slope.map
$ 
$ rm dem_slope.map 
$ 
$ myJSON=$(</dev/stdin)
{
  "inputs": {
    "INPUT": "dem.map",
    "OUTPUT": "dem_slope.map"
  }
}
$ 
$ echo "$myJSON" | qgis_process run pcraster:Slope - | tail -8
/usr/lib/python3/dist-packages/qgis/utils.py:888: DeprecationWarning: The distutils package is deprecated and slated for removal in Python 3.12. Use setuptools or check PEP 632 for potential alternatives
  mod = _builtin_import(name, globals, locals, fromlist, level)
  "python_version": "3.10.12",
  "qgis_code_revision": "c0b88339649",
  "qgis_version": "3.32.2-Lima",
  "qt_version": "5.15.3",
  "results": {
    "OUTPUT": "dem_slope.map"
  }
}$ 
$ 
$ ls dem_slope.map 
dem_slope.map
$ 
$ qgis_process --version
QGIS 3.32.2-Lima 'Lima' (c0b88339649)
QGIS code revision c0b88339649
Qt version 5.15.3
Python version 3.10.12
GDAL/OGR version 3.6.4
PROJ version 9.1.1
EPSG Registry database version v10.076 (2022-08-31)
GEOS version 3.11.1-CAPI-1.17.1
SQLite version 3.37.2
OS Linux Mint 21.2