Union & Intersection results are Incorrect #23878

qgib · 2016-12-08T23:01:40Z

Author Name: Devdatta Tengshe (@devdattaT)
Original Redmine Issue: 15962
Affected QGIS version: 2.18.11
Redmine category:processing/qgis

I found some strange issues when running Intersect & Union from the Vector>>GeoProcessing Menu. I've reproduced this on QGIS 2.14.4 as well as 2.18.0 on Windows 8

I observed, that for some datasets, the results of Overlay analysis are strange, and not at all what you expect. I've attached two sample datasets which show this.
1)Sample 1 consists of 2 shapefiles and the results of Union & Intersection. As you can see from the Image, this is not the results one should be getting.!Intersect1!

2)Sample 2 consists of the data where I saw the issue for the first time. When you Union 'dissolved.shp' & 'wards.shp' you expect to get an layer with unioned shapes of all features , but you get some kind of data which is closer to a 'Difference'; But if you closely look, there are more artifacts present in the data.

This might be related to Bugs #22799 & #17224 ,bu I'm not sure.

It should also be noted that the input shapefiles were created in QGIS itself, so if there is any issue with incorrect Geometries, there might be a bug in the module which created these shapefiles as well.

Sample1.zip (Devdatta Tengshe)
Sample2.zip (Devdatta Tengshe)
Intersect.png (Devdatta Tengshe)
Screenshot_20161212_131856.png (Giovanni Manghi)
Screenshot_20161212_131355.png (Giovanni Manghi)
Input_files.zip (Amy Taylor)
QGIS_union.zip (Amy Taylor)
input_layers.PNG (Amy Taylor)
circ_Y_union_feature.PNG (Amy Taylor)
circ_Y_duplicate_feature_in_union.PNG (Amy Taylor)

Related issue(s): #17224 (relates), #17905 (relates)
Redmine related issue(s): 8456, 9297

qgib · 2016-12-12T04:17:24Z

Author Name: Giovanni Manghi (@gioman)

There are different issues here, both in QGIS (and that is very unfortunate) and in your data.

The Union between your 'dissolved.shp' & 'wards.shp'fails because the "wards" layer has a lot of issue, both from a geometry point of view but also from a topology one, in fact the polygons that you probably expected to be adjacent are not, many of them have gaps, that at the end of the operation you see as "artifacts". If you give a pass trough Processing/v.clean using also the snapping parameter then the union works well.

On the other hand the union/intersection of "areaA" and "areaB" does not works because in one of the two layers you have geometry errors (auto intersections and duplicate nodes). This errors are reported in the QGIS log after the operation runs, but of course this is NOT enough for the average user, a proper warning must be thrown. This is the error

2016-12-12T07:36:28	2	GEOS geoprocessing error: One or more input features have invalid geometry.
2016-12-12T07:36:28	0	Feature geometry error: One or more output features ignored due to invalid geometry.
2016-12-12T07:36:28	2	GEOS geoprocessing error: One or more input features have invalid geometry.
2016-12-12T07:36:28	0	Feature geometry error: One or more output features ignored due to invalid geometry.

Moreover while the intersection operation seems to work ok (after fixing those issues) the union one creates an output that smells rotten fish, in fact it seems to create duplicate geometries (see attached images).

See also: #17905

I'm tagging as SEVERE all this geoprocessing issues because they MUST be fixed once for all.

version was changed from 2.14.4 to 2.18.0
10601 was configured as Screenshot_20161212_131856.png
10602 was configured as Screenshot_20161212_131355.png
category_id was configured as Processing/QGIS
priority_id was changed from High to Severe/Regression
operating_system was changed from Windows to
os_version was changed from 8 to

Screenshot_20161212_131856.png (Giovanni Manghi)
Screenshot_20161212_131355.png (Giovanni Manghi)

qgib · 2017-02-20T01:13:09Z

Author Name: Amy Taylor (Amy Taylor)

Re the union, I've been having issues with unexpected duplicated polygons in the results.

Attached are two input shapefiles and the result of a QGIS union between these.
The resulting union has one duplicated polygon which I wouldn't expect to be there: FID 3 (circ Y) from Test_input_B is duplicated in it's entirety as FID 9 in the union output shapefile.
The input, expected and duplicated features are shown in the attached images.

10746 was configured as circ_Y_union_feature.PNG
10743 was configured as Input_files.zip
10744 was configured as QGIS_union.zip
10745 was configured as input_layers.PNG

circ_Y_union_feature.PNG (Amy Taylor) - screenshot expected circ Y union shape
Input_files.zip (Amy Taylor) - Test input layers A and B
QGIS_union.zip (Amy Taylor) - Resulting union
input_layers.PNG (Amy Taylor) - screenshot input layers

qgib · 2017-02-20T01:15:41Z

Author Name: Amy Taylor (Amy Taylor)

Sorry, forgot to attach last image, screenshot of duplicated feature attached.

10747 was configured as circ_Y_duplicate_feature_in_union.PNG

circ_Y_duplicate_feature_in_union.PNG (Amy Taylor) - screenshot duplicated feature in union

qgib · 2017-02-27T23:22:43Z

Author Name: Giovanni Manghi (@gioman)

fixed_version_id was configured as Version 2.18
version was changed from 2.18.0 to 2.18.4

qgib · 2017-04-30T15:08:20Z

Author Name: Giovanni Manghi (@gioman)

regression was configured as 1

qgib · 2017-04-30T15:08:56Z

Author Name: Giovanni Manghi (@gioman)

priority_id was changed from Severe/Regression to High

qgib · 2017-04-30T23:10:55Z

Author Name: Giovanni Manghi (@gioman)

easy_fix was configured as 0

qgib · 2017-05-12T18:08:01Z

Author Name: Giovanni Manghi (@gioman)

see also #20192 for more intersection errors!

description was changed from I found some strange issues when running Intersect & Union from the Vector>>GeoProcessing Menu. I've reproduced this on QGIS 2.14.4 as well as 2.18.0 on Windows 8

I observed, that for some datasets, the results of Overlay analysis are strange, and not at all what you expect. I've attached two sample datasets which show this.
1)Sample 1 consists of 2 shapefiles and the results of Union & Intersection. As you can see from the Image, this is not the results one should be getting.!Intersect1!

2)Sample 2 consists of the data where I saw the issue for the first time. When you Union 'dissolved.shp' & 'wards.shp' you expect to get an layer with unioned shapes of all features , but you get some kind of data which is closer to a 'Difference'; But if you closely look, there are more artifacts present in the data.

This might be related to Bugs #22799 & #17224 ,bu I'm not sure.

It should also be noted that the input shapefiles were created in QGIS itself, so if there is any issue with incorrect Geometries, there might be a bug in the module which created these shapefiles as well. to I found some strange issues when running Intersect & Union from the Vector>>GeoProcessing Menu. I've reproduced this on QGIS 2.14.4 as well as 2.18.0 on Windows 8

I observed, that for some datasets, the results of Overlay analysis are strange, and not at all what you expect. I've attached two sample datasets which show this.
1)Sample 1 consists of 2 shapefiles and the results of Union & Intersection. As you can see from the Image, this is not the results one should be getting.!Intersect1!

2)Sample 2 consists of the data where I saw the issue for the first time. When you Union 'dissolved.shp' & 'wards.shp' you expect to get an layer with unioned shapes of all features , but you get some kind of data which is closer to a 'Difference'; But if you closely look, there are more artifacts present in the data.

This might be related to Bugs #22799 & #17224 ,bu I'm not sure.

It should also be noted that the input shapefiles were created in QGIS itself, so if there is any issue with incorrect Geometries, there might be a bug in the module which created these shapefiles as well.

qgib · 2017-05-12T18:09:22Z

Author Name: Giovanni Manghi (@gioman)

In 2.18.7 the union operation with the provided dataset, that usually returned a wrong result, now is BROKEN

2017-05-12T19:08:56 2 Uncaught error while executing algorithm
Traceback (most recent call last):
File "C:/OSGEO41/apps/qgis/./python/plugins\processing\core\GeoAlgorithm.py", line 203, in execute
self.processAlgorithm(progress)
File "C:/OSGEO41/apps/qgis/./python/plugins\processing\algs\qgis\Union.py", line 164, in processAlgorithm
if diff_geom.isGeosEmpty() or not diff_geom.isGeosValid():
AttributeError: 'NoneType' object has no attribute 'isGeosEmpty'

subject was changed from Union & Intersection results are Incorrect to Union & Intersection results are Incorrect or borken

qgib · 2017-05-12T18:10:39Z

Author Name: Giovanni Manghi (@gioman)

subject was changed from Union & Intersection results are Incorrect or borken to Union & Intersection results are Incorrect or broken

qgib · 2017-05-12T18:11:26Z

Author Name: Giovanni Manghi (@gioman)

see also #17224

qgib · 2017-05-12T18:11:36Z

Author Name: Giovanni Manghi (@gioman)

version was changed from 2.18.4 to 2.18.7

qgib · 2017-05-13T07:44:07Z

Author Name: Alexander Bruy (@alexbruy)

Python error should be fixed in bc4e70a

qgib · 2017-05-15T15:42:51Z

Author Name: Giovanni Manghi (@gioman)

subject was changed from Union & Intersection results are Incorrect or broken to Union & Intersection results are Incorrect

qgib · 2017-05-16T07:25:11Z

Author Name: Sandro Santilli (@strk)

Was Redmine plugin automatically showing commits referencing tickets lost ?
I guess Alexander meant bc4e70a (with prefix it turns to a link)

qgib · 2017-05-16T08:20:51Z

Author Name: Giovanni Manghi (@gioman)

Sandro Santilli wrote:

Was Redmine plugin automatically showing commits referencing tickets lost ?
I guess Alexander meant bc4e70a (with prefix it turns to a link)

Jurgen and Richard updated the platform, maybe something is still not working 100%.

qgib · 2017-05-16T09:20:47Z

Author Name: Sandro Santilli (@strk)

I'm looking at an example sent by Giovanni Manghi, will report here what I see.

assigned_to_id was configured as Sandro Santilli

qgib · 2017-05-16T09:33:12Z

Author Name: Sandro Santilli (@strk)

The example sent to me by Giovanni, with the instructions provided, results in an error sent to the user about the input having invalid geometries. It seems to be the correct behavior with regard to this ticket (which is about incorrect results), so this is still in a "cannot reproduce" state for me. This is as of fd7b52d in release-2_18 branch (which is what "Affected QGIS version" references).

I'm closing again. Please consider filing a new ticket if it helps reducing the confusion between what was fixed and what not (as this one seems to be about many different cases)

status_id was changed from Open to Closed
resolution was changed from to worksforme

qgib · 2017-05-16T09:42:01Z

Author Name: Giovanni Manghi (@gioman)

status_id was changed from Closed to Open
resolution was changed from worksforme to

qgib · 2017-05-16T09:42:25Z

Author Name: Giovanni Manghi (@gioman)

Sandro Santilli wrote:

The example sent to me by Giovanni, with the instructions provided, results in an error sent to the user about the input having invalid geometries. It seems to be the correct behavior with regard to this ticket (which is about incorrect results), so this is still in a "cannot reproduce" state for me. This is as of fd7b52d in release-2_18 branch (which is what "Affected QGIS version" references).

I'm closing again. Please consider filing a new ticket if it helps reducing the confusion between what was fixed and what not (as this one seems to be about many different cases)

do not close important tickets that are all BUT fixed/solved, thanks.

qgib · 2017-05-16T09:48:45Z

Author Name: Sandro Santilli (@strk)

assigned_to_id removed Sandro Santilli

qgib · 2017-05-16T09:54:55Z

Author Name: Sandro Santilli (@strk)

Giovanni for tickets to be tractable there's a need of a recipe to reproduce the problem. This ticket is very vague in what it is saying. It's about wrong results, in its original report. Then it changed to "Wrong or Broken" later. Then was closed by a commit.

At this moment it isn't clear at all what this issue is about.
I think a proper issue must have:

Instructions for steps to take
Description of what happens by taking those steps
Description of what would instead be the expected behavior

qgib · 2017-05-16T09:58:30Z

Author Name: Giovanni Manghi (@gioman)

Sandro Santilli wrote:

Giovanni for tickets to be tractable there's a need of a recipe to reproduce the problem. This ticket is very vague in what it is saying. It's about wrong results, in its original report. Then it changed to "Wrong or Broken" later. Then was closed by a commit.

At this moment it isn't clear at all what this issue is about.
I think a proper issue must have:

Instructions for steps to take

Description of what happens by taking those steps

Description of what would instead be the expected behavior

I already wrote to you the EXACT steps to replicate: take "sample1.zip", open areaA and areaB shapes. Do the intersection > wrong result. How difficult is that?

qgib · 2017-05-16T10:00:09Z

Author Name: Giovanni Manghi (@gioman)

Description of what would instead be the expected behavior

do you mean -seriously- that you do not know how it should look like the result of the intersection between 3/4 simple test polygons? come on...

qgib · 2017-05-16T10:42:11Z

Author Name: Sandro Santilli (@strk)

No, I meant that I don't get a WRONG result, but rather I get a FAILURE telling me (via GUI) that the operation cannot be completed. Here's a screencast: http://strk.kbt.io/tmp/vokoscreen-2017-05-16_12-37-05.mkv
That's what happens on Linux with QGIS 2.8.17+57 commits in the release-2_18 branch, and is the reason why I closed this ticket.

qgib · 2017-07-25T18:41:52Z

Author Name: Giovanni Manghi (@gioman)

regression was changed from 1 to 0
version was changed from 2.18.7 to 2.18.11

qgib · 2017-07-25T21:21:49Z

Author Name: Giovanni Manghi (@gioman)

crashes_corrupts_data was changed from 0 to 1

qgib · 2017-09-13T09:09:46Z

Author Name: Giovanni Manghi (@gioman)

replaced by #25030

status_id was changed from Open to Closed
resolution was changed from to duplicate

qgib closed this as completed Sep 13, 2017

qgib added the Bug Either a bug report, or a bug fix. Let's hope for the latter! label May 25, 2019

qgib added High Priority Processing Relating to QGIS Processing framework or individual Processing algorithms Crash/Data Corruption labels May 25, 2019

qgib added this to the Version 2.18 milestone May 25, 2019

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Union & Intersection results are Incorrect #23878

Union & Intersection results are Incorrect #23878

qgib commented Dec 8, 2016 •

edited

qgib commented Dec 12, 2016 •

edited

qgib commented Feb 20, 2017

qgib commented Feb 20, 2017

qgib commented Feb 27, 2017

qgib commented Apr 30, 2017

qgib commented Apr 30, 2017

qgib commented Apr 30, 2017

qgib commented May 12, 2017 •

edited

qgib commented May 12, 2017

qgib commented May 12, 2017

qgib commented May 12, 2017 •

edited

qgib commented May 12, 2017

qgib commented May 13, 2017

qgib commented May 15, 2017

qgib commented May 16, 2017

qgib commented May 16, 2017

qgib commented May 16, 2017

qgib commented May 16, 2017

qgib commented May 16, 2017

qgib commented May 16, 2017

qgib commented May 16, 2017

qgib commented May 16, 2017

qgib commented May 16, 2017

qgib commented May 16, 2017

qgib commented May 16, 2017

qgib commented Jul 25, 2017

qgib commented Jul 25, 2017

qgib commented Sep 13, 2017 •

edited

Union & Intersection results are Incorrect #23878

Union & Intersection results are Incorrect #23878

Comments

qgib commented Dec 8, 2016 • edited

qgib commented Dec 12, 2016 • edited

qgib commented Feb 20, 2017

qgib commented Feb 20, 2017

qgib commented Feb 27, 2017

qgib commented Apr 30, 2017

qgib commented Apr 30, 2017

qgib commented Apr 30, 2017

qgib commented May 12, 2017 • edited

qgib commented May 12, 2017

qgib commented May 12, 2017

qgib commented May 12, 2017 • edited

qgib commented May 12, 2017

qgib commented May 13, 2017

qgib commented May 15, 2017

qgib commented May 16, 2017

qgib commented May 16, 2017

qgib commented May 16, 2017

qgib commented May 16, 2017

qgib commented May 16, 2017

qgib commented May 16, 2017

qgib commented May 16, 2017

qgib commented May 16, 2017

qgib commented May 16, 2017

qgib commented May 16, 2017

qgib commented May 16, 2017

qgib commented Jul 25, 2017

qgib commented Jul 25, 2017

qgib commented Sep 13, 2017 • edited

qgib commented Dec 8, 2016 •

edited

qgib commented Dec 12, 2016 •

edited

qgib commented May 12, 2017 •

edited

qgib commented May 12, 2017 •

edited

qgib commented Sep 13, 2017 •

edited