crash loading big shapefile #11512

qgib · 2008-12-10T10:30:29Z

Author Name: Maciej Sieczka - (Maciej Sieczka -)
Original Redmine Issue: 1452

Assignee: nobody -

QGIS crashes loading a 382 MB shapefile, 180514 polygons:

$ ogrinfo -al -so neg_patch_poly_bnd_area_splt

INFO: Open of @robota/fucha/2008_javier/neg_patch_poly_bnd_area_splt'
      using driver @ESRI Shapefile' successful.

Layer name: neg_patch_poly_bnd_area_splt
Geometry: Polygon
Feature Count: 180514
Extent: (-180.000000, -90.000000) - (180.000000, 90.000000)
Layer SRS WKT:
GEOGCS["GCS_WGS_1984",
    DATUM["WGS_1984",
        SPHEROID[[WGS_1984]],
    PRIMEM[[Greenwich]],
    UNIT[[Degree]]
cat: Real (11.0)

Backtrace:

Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x7f6abcda2710 (LWP 21711)]
0x00007f6abb8f80fc in ?? () from /usr/lib/libQtGui.so.4
(gdb) bt
#_0  0x00007f6abb8f80fc in ?? () from /usr/lib/libQtGui.so.4
#_3905  0x00007f6abb8c1295 in ?? () from /usr/lib/libQtGui.so.4
#_3906  0x00007f6abb8bde4f in QStroker::processCurrentSubpath ()
   from /usr/lib/libQtGui.so.4
#_3907  0x00007f6abb8beee0 in QStrokerOps::strokePath ()
   from /usr/lib/libQtGui.so.4
#_3908  0x00007f6abb8f758a in ?? () from /usr/lib/libQtGui.so.4
#_3909  0x00007f6abc6d38ef in [[QgsVectorLayer]]::drawPolygon (this=0x2b2b980, 
    feature=0x7f6a69433010 "\\001\\003", p=0x7fffc4edde70, mtp=0x1fa2c78, 
    ct=0x0, drawingToEditingCanvas=true)
    at /home/shoofi/src/straight/qgis-trunk/src/core/qgsvectorlayer.cpp:649
#_3910  0x00007f6abc6d4908 in [[QgsVectorLayer]]::drawFeature (this=0x2b2b980, 
    p=0x7fffc4edde70, fet=@0x7fffc4edd8d0, theMapToPixelTransform=0x1fa2c78, 
    ct=0x0, marker=0x7fffc4edd920, widthScale=3.8582677165354333, 
    rasterScaleFactor=1, drawingToEditingCanvas=true)
    at /home/shoofi/src/straight/qgis-trunk/src/core/qgsvectorlayer.cpp:3389
#_3911  0x00007f6abc6d5043 in [[QgsVectorLayer]]::draw (this=0x2b2b980, 
    rendererContext=@0x1fa2c38)
    at /home/shoofi/src/straight/qgis-trunk/src/core/qgsvectorlayer.cpp:767
#_3912  0x00007f6abc685776 in [[QgsMapRenderer]]::render (this=0x1fa2ba0, 
    painter=0x7fffc4edde70)
    at /home/shoofi/src/straight/qgis-trunk/src/core/qgsmaprenderer.cpp:340
#_3913  0x00007f6abca85049 in [[QgsMapCanvasMap]]::render (this=0x1fc0630)
    at /home/shoofi/src/straight/qgis-trunk/src/gui/qgsmapcanvasmap.cpp:84
#_10 0x00007f6abca7fc72 in [[QgsMapCanvas]]::refresh (this=0x1f99700)
    at /home/shoofi/src/straight/qgis-trunk/src/gui/qgsmapcanvas.cpp:365
#_3914 0x00000000004c517e in [[QgisApp]]::addVectorLayers (this=0x1eee610, 
    theLayerQStringList=@0x7fffc4ede1a0, enc=@0x7fffc4ede190)
    at /home/shoofi/src/straight/qgis-trunk/src/app/qgisapp.cpp:2171
#_3915 0x00000000004c6c64 in [[QgisApp]]::addVectorLayer (this=0x1eee610)
    at /home/shoofi/src/straight/qgis-trunk/src/app/qgisapp.cpp:2109
#_3916 0x000000000069b636 in [[QgisApp]]::qt_metacall (this=0x1eee610, 
    _c=QMetaObject::InvokeMetaMethod, _id=86, _a=0x7fffc4ede3b0)
    at /home/shoofi/src/straight/qgis-trunk/build/src/app/moc_qgisapp.cxx:328
#_3917 0x00007f6abc27c6d4 in QMetaObject::activate () from /usr/lib/libQtCore.so.4
#_3918 0x00007f6abb786d77 in QAction::triggered () from /usr/lib/libQtGui.so.4
#_3919 0x00007f6abb787540 in QAction::activate () from /usr/lib/libQtGui.so.4
#_3920 0x00007f6abba97cca in ?? () from /usr/lib/libQtGui.so.4
#_3921 0x00007f6abba97f65 in QAbstractButton::mouseReleaseEvent ()
   from /usr/lib/libQtGui.so.4
#_3922 0x00007f6abbb5f74a in QToolButton::mouseReleaseEvent ()
   from /usr/lib/libQtGui.so.4
#_3923 0x00007f6abb7df139 in QWidget::event () from /usr/lib/libQtGui.so.4
#_3924 0x00007f6abb78ca5d in QApplicationPrivate::notify_helper ()
   from /usr/lib/libQtGui.so.4
#_3925 0x00007f6abb79504a in QApplication::notify () from /usr/lib/libQtGui.so.4
#_3926 0x00007f6abc638ebe in [[QgsApplication]]::notify (this=0x7fffc4edfc20, 
    receiver=0x1f6f710, event=0x7fffc4eded40)
    at /home/shoofi/src/straight/qgis-trunk/src/core/qgsapplication.cpp:78
#_3927 0x00007f6abc268381 in QCoreApplication::notifyInternal ()
   from /usr/lib/libQtCore.so.4
#_3928 0x00007f6abb7943e8 in QApplicationPrivate::sendMouseEvent ()
   from /usr/lib/libQtGui.so.4
#_3929 0x00007f6abb7f8909 in ?? () from /usr/lib/libQtGui.so.4
#_3930 0x00007f6abb7f77ff in QApplication::x11ProcessEvent ()
   from /usr/lib/libQtGui.so.4
#_3931 0x00007f6abb81ec84 in ?? () from /usr/lib/libQtGui.so.4
#_3932 0x00007f6ab80c778b in g_main_context_dispatch ()
   from /usr/lib/libglib-2.0.so.0
#_3933 0x00007f6ab80caf5d in ?? () from /usr/lib/libglib-2.0.so.0
#_3934 0x00007f6ab80cb11b in g_main_context_iteration ()
   from /usr/lib/libglib-2.0.so.0
#_3935 0x00007f6abc29070f in QEventDispatcherGlib::processEvents ()
   from /usr/lib/libQtCore.so.4
#_3936 0x00007f6abb81e44f in ?? () from /usr/lib/libQtGui.so.4
#_3937 0x00007f6abc266ca2 in QEventLoop::processEvents ()
   from /usr/lib/libQtCore.so.4
#_3938 0x00007f6abc266e2d in QEventLoop::exec () from /usr/lib/libQtCore.so.4
#_36 0x00007f6abc2692dd in QCoreApplication::exec ()
   from /usr/lib/libQtCore.so.4
#_37 0x00000000004b5639 in main (argc=1, argv=0x7fffc4ee0318)
    at /home/shoofi/src/straight/qgis-trunk/src/app/main.cpp:683

Debian testing amd64, QGIS SVN trunk r9770, QT 4.4.3, GDAL 1.6 SVN trunk r15902 (05.12.2008).

Schermata.jpg (luca76 -)

The text was updated successfully, but these errors were encountered:

qgib · 2008-12-16T07:21:16Z

Author Name: Horst Düster (@hdus)

Works for me. I just loaded a shape with size of 1.2GB and more than 1000000 polygons without any problems regardless whether I use on the fly projection to WGS84 or not. Did you check your data?

qgib · 2008-12-16T11:34:18Z

Author Name: Maciej Sieczka - (Maciej Sieczka -)

Replying to [comment:1 hdus]:

Works for me. I just loaded a shape with size of 1.2GB and more than 1000000 polygons without any problems regardless whether I use on the fly projection to WGS84 or not. Did you check your data?

The data is OK AFAICT. I can easily process it with OGR tools, export/import in GRASS. Will soon try rendering with [[MapServer]] and I'll post the result if I remember to.

Interesting - the Shapefile crashes whole Xorg (!!!) if "Fix problems..." is set ON in Options>Rendering, and only crashes QGIS alone if the option is set OFF.

I can't share the data in public. It's for sale. I could share it with an interested QGIS developer in private if he promises not to redistribute it.

qgib · 2008-12-17T06:48:43Z

Author Name: Horst Düster (@hdus)

Replying to [comment:2 msieczka]:

Replying to [comment:1 hdus]:

Works for me. I just loaded a shape with size of 1.2GB and more than 1000000 polygons without any problems regardless whether I use on the fly projection to WGS84 or not. Did you check your data?

The data is OK AFAICT. I can easily process it with OGR tools, export/import in GRASS. Will soon try rendering with [[MapServer]] and I'll post the result if I remember to.

Interesting - the Shapefile crashes whole Xorg (!!!) if "Fix problems..." is set ON in Options>Rendering, and only crashes QGIS alone if the option is set OFF.

That's the difference to my attempt. I had swiched off this option. Now I switched "Fix problems ..." on, but every works fine for me.

I can't share the data in public. It's for sale. I could share it with an interested QGIS developer in private if he promises not to redistribute it.

qgib · 2009-01-10T04:04:15Z

Author Name: Maciej Sieczka - (Maciej Sieczka -)

I have isolated the smalest single polygon that crashes Xorg when loaded to QGIS. It's 2.5 MB. You can download it at http://www.sieczka.org/tmp/feat_8.7z (0.5 MB). It's a subset of the big Shapefile I originaly reported about. I have 4 more such polygons (larger though) if needed.

ogr2ogr can process the polygon just fine. It also renders OK with [[MapServer]]'s shp2img. As well as QGIS with "Fix problems..." set ON in Options>Rendering set OFF.

Yet it constantly crashes Xorg when loaded to QGIS with "Fix problems..." set ON. Differently than with the original big shapefile, if "Fix problems..." is set OFF there's no crash with this single polygon. After looking more into this, it showed that the original big shapefile crashes QGIS because the memory runs out.

So probably two issues here - QGIS crashes when it runs out of memory with big shapefiles AND it makes Xorg crash rendering certain shapes.

qgib · 2009-04-08T08:40:45Z

Author Name: Paolo Cavallini (@pcav)

The shp causing the crash is no longer available. Is the bug still valid? Please check, and if it is, make the shp available (if not, just close the ticket).
Thanks

qgib · 2009-04-19T01:59:39Z

Author Name: Paolo Cavallini (@pcav)

According to our new bug policy, I'm closing this pending feedback from the user. In case it is still valid, please reopen it. Sorry for the additional trouble.

resolution was changed from to worksforme
status_id was changed from Open to Closed

qgib · 2009-08-21T23:02:39Z

Author Name: Anónimo (Anónimo)

Milestone Version 1.0.2 deleted

qgib · 2009-08-27T08:40:42Z

Author Name: sieczka - (sieczka -)

Replying to [comment:5 pcav]:

The shp causing the crash is no longer available. Is the bug still valid? Please check, and
if it is, make the shp available (if not, just close the ticket).

I can again reproduce the crash with GSSHS 2.0 Shapefiles: ftp://ftp.soest.hawaii.edu/pwessel/gshhs/GSHHS_shp_2.0.zip.

QGIS trunk , GDAL 1.6.2, QT 4.4.3, GEOS 3.1.0, Debian stable amd64.

status_id was changed from Closed to Feedback
resolution was changed from worksforme to

qgib · 2009-08-28T01:01:54Z

Author Name: Giovanni Manghi (@gioman)

Hi,

I have loaded all the shapes included in the archive you linked (including a a 180000 polygon one) and I have no crashes. Qgis trunk compiled under ubuntu 9.04 32 bit with 512mb of ram.

qgib · 2009-08-28T01:21:15Z

Author Name: Giovanni Manghi (@gioman)

Tested also under windows xp 32 bit, 512mb of ram with qgis 1.2 (osgeo4w). No problems at all.

Can anyone on a 64bit platform make a test and report back? Thanks.

qgib · 2009-08-31T21:30:13Z

Author Name: luca76 - (luca76 -)

Works to me on x64 system, with QGIS trunk. See my attachment.

[https://trac.osgeo.org/qgis/attachment/ticket/1452/Schermata.jpg]

closed fixed.

resolution was changed from to fixed
status_id was changed from Feedback to Closed

qgib added the Bug Either a bug report, or a bug fix. Let's hope for the latter! label May 24, 2019

qgib added this to the Version 1.0.3 milestone May 24, 2019

qgib closed this as completed May 24, 2019

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

crash loading big shapefile #11512

crash loading big shapefile #11512

qgib commented Dec 10, 2008

qgib commented Dec 16, 2008

qgib commented Dec 16, 2008

qgib commented Dec 17, 2008

qgib commented Jan 10, 2009

qgib commented Apr 8, 2009

qgib commented Apr 19, 2009

qgib commented Aug 21, 2009

qgib commented Aug 27, 2009

qgib commented Aug 28, 2009

qgib commented Aug 28, 2009

qgib commented Aug 31, 2009

crash loading big shapefile #11512

crash loading big shapefile #11512

Comments

qgib commented Dec 10, 2008

qgib commented Dec 16, 2008

qgib commented Dec 16, 2008

qgib commented Dec 17, 2008

qgib commented Jan 10, 2009

qgib commented Apr 8, 2009

qgib commented Apr 19, 2009

qgib commented Aug 21, 2009

qgib commented Aug 27, 2009

qgib commented Aug 28, 2009

qgib commented Aug 28, 2009

qgib commented Aug 31, 2009