QGis hang when loading large project

[qgib]

Author Name: Patrick Dunford (@enzedrailmaps)
Original Redmine Issue: 6474
Affected QGIS version: master
Redmine category:project_loading/saving

---

I have a number of projects that use reasonably large datasets in a number of files. The biggest of these sits in about 3 GB of data on the disk (using file based shape layer rather than any kind of database). The very large layers are rasters of topographical relief data which Qgis always loads into layers first. Qgis seems to run into some kind of arbitrary limit on memory when it loads these and is therefore hanging. Either that or there is a problem with another layer that it is trying to load.

When Qgis is loading this project the progress bar will stall at some arbitrary percentage depending on how many attempts you have had (and therefore how many times you have killed QGis) etc. The first times it would stall at 7% or 9%. One time it stalled at 97%.

Each time it stopped loading I would go into Task Manager to see what it was doing and would watch as the memory usage climbed up and this would usually reach about 190,000 K of memory in use by the application and then some ceiling would be hit and no more memory would be allocated. Qgis would just sit there with the memory usage flatlined and the CPU using 49-50%.

This particular project when loaded uses nearly 1 GB of memory. How do I know this? I decided I would run Qgis under Dependency Walker which is the nearest thing I have to a debugger. Of course it can't start a batch file so it would load the executable and all its dependencies until it ran into problems with paths defined in the batch file. While it was doing this the existing hung copy of Qgis somehow got over its memory limitation and finished loading the project and pushed the memory up to about 960 MB.

If there is some arbitrary limitation of size there needs to be some way of aborting the load if the project is too big. The killer is probably some OSM layers I loaded which have a lot of data in them (because they cover the whole country). I will try taking these layers out to see if the project becomes smaller so it can be hopefully sure of loading. But now is there actually a size limit that a project shouldn't go over if we want to be sure it will be reloadable.

[qgib]

Author Name: Giovanni Manghi (@gioman)

---

Are you using qgis 1.8? any wms/wfs connection? did you try to optimize your data creating for example spatial indexes for vectors? maybe a plugin problem, can you test launching qgis with the --noplugins option? Can you test on another machine?

any chance you can share one of the problematic projects to allow doing some tests on other environments?

I have also very big projects (up to several GB of data on disk, including OSM) and I don't have any particular problem on both Windows and Linux, it just takes its time to load the project.

---

• fixed_version_id was configured as Version 2.0.0
• status_id was changed from Open to Feedback
• category_id was configured as Project Loading/Saving

[qgib]

Author Name: Giovanni Manghi (@gioman)

---

closing for lack of feedback.

---

• resolution was changed from to invalid
• status_id was changed from Feedback to Closed