Saving edits on larger vector layers is slow

[theroggy]

What is the bug or the crash?

Saving edits on larger vector layers is slow.

Steps to reproduce the issue

1. Download (and unzip) a larger vector file, e.g. Agricultural parcels flanders, 2021 is a 720 MB geopackage with ~600.000 polygons.
2. Go in the menu to "Layer"/"Add layer"/"Add vector layer..." and choose the downloaded file to open it in QGIS
3. Click on the layer in the "Layer" pane to select it + in the menu, click "Layer"/"Toggle editing" to make it editable
4. In the "Attributes" toolbar, click the "Open Field Calculator" tool
5. Check "update existing field", then choose e.g. field "GWSNAM_V"
6. For expression type e.g. 'test_value' (including the single quotes)
7. Click "OK"
8. In the menu, click "Layer"/"Toggle editing" to stop editing, and in the popup show click "Save"
9. QGIS is "not responding" for several minutes for save to be ready if the file is on a network drive. On local SSD +- 1.5 minutes.

The larger the file, the slower the save...

I also wrote a pyqgis script (see below) to do some tests around this behaviour, and the speed of the storage being used indeed seems to have quite an impact on the time "commitchanges" takes:

• If I first copy the file to a memory layer, commitchanges takes 1.1 seconds
• If the file is located on a local ssd, commitchanges takes 77 seconds
• If the file is located on a network drive on a corporate netapp storage, commitchanges takes 105 seconds

from datetime import datetime
from pathlib import Path

import qgis.core  # type: ignore


layer_path = Path("C:/Temp/perc_2021_2022-02-02_copy.gpkg")
qgs = qgis.core.QgsApplication([], False)
qgs.initQgis()
layer = qgis.core.QgsVectorLayer(str(layer_path), layer_path.stem, "ogr")

"""
print("Start materialize to memory layer")
start = datetime.now()
layer_mem = layer.materialize(qgis.core.QgsFeatureRequest())
print(f"Materialize took {datetime.now()-start}")
del layer
layer = layer_mem
"""

print("Start layer.getFeatures() without update")
start = datetime.now()
print(f"layer.getFeatures() found {len(list(layer.getFeatures()))} features and took {datetime.now()-start}")

layer.startEditing()
print("Start loop to update features")
fieldid = layer.fields().indexFromName("GWSCOD_V")
start = datetime.now()
for feature in layer.getFeatures():
    layer.changeAttributeValue(feature.id(), fieldid, "test")
print(f"Loop took {datetime.now()-start}")

start = datetime.now()
layer.commitChanges()
print(f"Commitchanges took {datetime.now()-start}")

del layer

Versions

The python script was tests on a conda installation of qgis on a windows 10 computer:

# Name                    Version                   Build  Channel
qgis                      3.30.1          py310h4d4cdf1_1    conda-forge

The desktop test was on a different windows 10 computer:

QGIS version
3.30.2-'s-Hertogenbosch
QGIS code revision
0992b533
Qt version
5.15.3
Python version
3.9.5
GDAL/OGR version
3.6.4
PROJ version
9.2.0
EPSG Registry database version
v10.082 (2023-02-06)
GEOS version
3.11.2-CAPI-1.17.2
SQLite version
3.41.1
PDAL version
2.5.2
PostgreSQL client version
unknown
SpatiaLite version
5.0.1
QWT version
6.1.6
QScintilla2 version
2.13.1
OS version
Windows 10 Version 2009


Active Python plugins
qfieldsync
v3.3.1
qgiscloud
3.3.11
QuickWKT
3.1
db_manager
0.1.20
grassprovider
2.12.99
MetaSearch
0.3.6
processing
2.12.99

Supported QGIS version

• I'm running a supported QGIS version according to the roadmap.

New profile

• I tried with a new QGIS profile

Additional context

No response