minimumValue() in 2.6.x for Windows different than Linux for NULL

[qgib]

Author Name: Jeff Cavner (Jeff Cavner)
Original Redmine Issue: 11887
Affected QGIS version: 2.6.0
Redmine category:vectors

---

minimumValue() against the data provider or a vector layer object in Windows if a NULL value is present in the attributes for a field will return a 0 treating the NULL as the minimum value, instead of the minimum numeric value in that field (which is my case is greater than 0). In Linux for 2.6.x it works correctly and returns the minimum numeric value and does not treat NULLs as numeric.

[qgib]

Author Name: Matthias Kuhn (@m-kuhn)

---

Which data provider are you using?
Is the layer in edit mode and does it contain changes when the mistake happens?

---

• status_id was changed from Open to Feedback

[qgib]

Author Name: Jeff Cavner (Jeff Cavner)

---

The dataprovider from a vector layer returned from vectorLyr.dataProvider(). Also the same thing happens against just the vectorLyr, so vectorLyr.minimumValue(index) returns the same 0.0 for a null value. The layer is not in edit mode and does not contain changes.

[qgib]

Author Name: Matthias Kuhn (@m-kuhn)

---

And which data provider as in postgis/spatialite/ogr ...?

[qgib]

Author Name: Jeff Cavner (Jeff Cavner)

---

ogr -- shapefile

[qgib]

Author Name: Matthias Kuhn (@m-kuhn)

---

Are the OGR versions on your Windows and Linux machine the same?

[qgib]

Author Name: Jeff Cavner (Jeff Cavner)

---

windows ogr 1.11.1

linux ogr 1.9.2

[qgib]

Author Name: Matthias Kuhn (@m-kuhn)

---

Can you open an issue on their tracker?
There's a similar one open that is being fixed for GDAL/OGR 2.0
https://trac.osgeo.org/gdal/ticket/5333

---

• status_id was changed from Feedback to Open
• resolution was changed from to up/downstream

[qgib]

Author Name: Jeff Cavner (Jeff Cavner)

---

Yes, just created a ticket over there. Thank you for all of the help.

[qgib]

Author Name: Jukka Rahkonen (Jukka Rahkonen)

---

By looking at this ESRI document http://webhelp.esri.com/arcgisdesktop/9.3/index.cfm?TopicName=Geoprocessing_considerations_for_shapefile_output NULL values in a numeric field are really stored as zeroes into the .dbf part of shapefile. And if NULL is stored as a zero into the .dbf file there is no way for GDAL to separate it from zero that represents a real zero. I may be wrong but this ticket and the corresponding GDAL ticket feel invalid for me. But I do not understand why Linux makes a difference for you. Could you add a shapefile with a few features and some NULLs for testing?

GDAL ticket 5333 is not related, it was about creating new functions for finding min/max/avg dates from a DATE field of a shapefile which actually a string "CCYYMMDD" in the .dbf file.

[qgib]

Author Name: Matthias Kuhn (@m-kuhn)

---

I can reproduce the problem on Linux with GDAL 1.11

Taking the shapefile lakes from here:

https://github.com/opengisch/QGIS-Sampledata/tree/master/shapefiles

Minimum value for the xlabel column (values 28399 and a some NULLs):

iface.activeLayer().dataProvider().minimumValue( 3 )
0

Editing the attriubte table and changing a NULL value to 0 works. The change can be saved and reloaded properly, so GDAL seems to be able to distinguish 0 and NULL for shapefiles.

Upstream issue: https://trac.osgeo.org/gdal/ticket/5799

[qgib]

Author Name: Jukka Rahkonen (Jukka Rahkonen)

---

I downloaded the lakes shapefile and had a try with GDAL 2.0-dev on Windows 7 but this version does not show the issue with ogrinfo:

ogrinfo lakes.shp -sql "select min(xlabel) from lakes"
INFO: Open of lakes.shp' using driver ESRI Shapefile' successful.

Layer name: lakes
Geometry: None
Feature Count: 1
Layer SRS WKT:
(unknown)
MIN_xlabel: Integer (8.0)
OGRFeature(lakes):0
MIN_xlabel (Integer) = 28399

[qgib]

Author Name: Matthias Kuhn (@m-kuhn)

---

Hmmm, same result here with 1.11.1.

Debugging shows that ogr fails to interpret the sql qgis sends it (see below) and it's actually our fallback code that fails to properly handle NULL values.

QGIS debug output:

src/providers/ogr/qgsogrprovider.cpp: 2360: (minimumValue) [30361ms] Failed to execute SQL: SELECT MIN("xlabel") FROM "lakes"

ogrinfo output:

ogrinfo lakes.shp -sql 'SELECT MIN("xlabel") FROM "lakes"'
INFO: Open of `lakes.shp'
      using driver `ESRI Shapefile' successful.
ERROR 1: Argument of column Summary Function 'MIN' should be a column.

Therefore the proper question is, should we send an unquoted column name to ogr or should that be changed on gdal side?

[qgib]

Author Name: Matthias Kuhn (@m-kuhn)

---

Fixed in changeset "1f8dee0caf7b995b1ccef41b2c92636082557da4".

---

• status_id was changed from Open to Closed