Spatial Indexing of Shapefiles on first load

[qgib]

Author Name: Jonathan Moules (Jonathan Moules)
Original Redmine Issue: 4330

Redmine category:vectors

---

Hi,
Earlier today I loaded three shapefiles into my 1.7.0 QGIS with a combined 5million+ features. QGIS defaults to "visible" for new layers, so these hung the program for several minutes as they don't have spatial indexes.
My suggestion therefore is that whenever a user adds a shapefile with more than x features (x = 10,000? I dunno) that doesn't have a QGIS spatial index, QGIS pops up a dialog asking the user if they want to create an index. For bonus points it could have checkboxes for "remember my answer for this session" and "remember my answer always". ArcGIS does this with Rasters and pyramids - it's very useful.

[qgib]

Author Name: Giovanni Manghi (@gioman)

---

I'm not sure this issue (of slow vector loading) goes away by creating spatial indexes, from my tests it doesn't.

The real issue is this: #12038

Have you made a test and noticed a noticable difference (in a big shapefile) with indexes and without?

[qgib]

Author Name: Giovanni Manghi (@gioman)

---

• category_id was configured as Vectors

[qgib]

Author Name: Jonathan Moules (Jonathan Moules)

---

Ah, it depends what you mean. I actually encountered this by loading a shapefile with ~3million line features so I can answer your question.
A spatial index will MASSIVELY increase loading speed if you're only looking at part of the entire dataset; that's what they're designed for. I.e., my 3million features cover the entirety of the English county of Warwickshire (about 2 thousand sq km), if I'm zoomed to a small area, say a town or village, it loads very fast when I have a spatial index (a few seconds). If I don't have the index, looking at that same small area takes forever (several minutes).

On the other hand, I doubt (but haven't tested it) that a spatial index will speed viewing of the entire dataset at once, which is what you may be testing. It shouldn't, that's not how they work. But you don't typically use a dataset with that many features by viewing it all at once (you won't be able to see the features after all).

[qgib]

Author Name: Giovanni Manghi (@gioman)

---

Jonathan Moules wrote:

Ah, it depends what you mean. I actually encountered this by loading a shapefile with ~3million line features so I can answer your question.
A spatial index will MASSIVELY increase loading speed if you're only looking at part of the entire dataset; that's what they're designed for. I.e., my 3million features cover the entirety of the English county of Warwickshire (about 2 thousand sq km), if I'm zoomed to a small area, say a town or village, it loads very fast when I have a spatial index (a few seconds). If I don't have the index, looking at that same small area takes forever (several minutes).

On the other hand, I doubt (but haven't tested it) that a spatial index will speed viewing of the entire dataset at once, which is what you may be testing. It shouldn't, that's not how they work. But you don't typically use a dataset with that many features by viewing it all at once (you won't be able to see the features after all).

Of course, you are right. The feature request (to add spatial indexes on load) make perfectly sense.

[qgib]

Author Name: Giovanni Manghi (@gioman)

---

• fixed_version_id was configured as Version 1.7.4

[qgib]

Author Name: Alexander Bruy (@alexbruy)

---

• subject was changed from Spatial Indexing of Shapefiles to Spatial Indexing of Shapefiles on first load

[qgib]

Author Name: Giovanni Manghi (@gioman)

---

• fixed_version_id was changed from Version 1.7.4 to Version 2.0.0

[qgib]

Author Name: Pirmin Kalberer (Pirmin Kalberer)

---

• fixed_version_id was changed from Version 2.0.0 to Future Release - Nice to have

[qgib]

Author Name: Giovanni Manghi (@gioman)

---

• easy_fix was configured as 0

[theroggy]

I'd like te add a vote on this one. Is a nice to have, but it would eliminate quite some performance annoyance with new users...

[alexbruy]

See also #38859 and #30528