Loading and displaying CADRG/NITF data is slower than it used to be

[bbarre9]

What is the bug or the crash?

Dragging and dropping a large number of CADRG frame files (basically NITF images) into QGIS 3.22.8 takes longer to load than in QGIS 2.18.16. For a very large number of images (~1000+), it takes far longer, to the point where it's uncertain whether QGIS will ever become responsive. I understand CADRG may not get much attention because it is a very outdated format, but it is still used heavily by certain, large organizations. And while there are better CADRG viewers such as FalconView, sometimes throwing frames into QGIS is much more convenient.

Side note in case this is related: QGIS 3.22.8 automatically expands many of the image entries in the layers panel when they are first loaded, showing an entry for each of the 216 colors of each image. I find this to be unnecessary; QGIS 2.18.16 does not automatically expand these layer entries.

Steps to reproduce the issue

1. Unzip the attached cwdz02.zip which contains 186 CADRG frame files comprising US coverage.
2. Select all .wd2 files (frame files).
3. Drag and drop the files into QGIS.
4. Observe the difference in load time between QGIS 2.18.6 (~7 seconds for me) and QGIS 3.22.8 (~57 seconds for me).
5. Doing the above for a much larger dataset (which I cannot provide) scales up the time, causing a very long, unknown wait time.

Versions

QGIS version
3.22.8-Białowieża
QGIS code revision
8d5e976
Qt version
6.15.3
Python version
3.9.5
GDAL/OGR version
3.5.0
PROJ version
9.0.1
EPSG Registry database version
v10.064 (2022-05-19)
GEOS version
3.10.3-CAPI-1.16.1
SQLite version
3.38.1
PDAL version
2.3.0
PostgreSQL client version
14.3
SpatiaLite version
5.0.1
QWT version
7.1.6
QScintilla2 version
2.13.1
OS version
Windows 10 Version 1909

Active Python plugins
db_manager
0.1.20
MetaSearch
0.3.5
processing
2.12.99
sagaprovider
2.12.99

Supported QGIS version

• I'm running a supported QGIS version according to the roadmap.

New profile

• I tried with a new QGIS profile

Additional context

_No response
cwdz02.zip
_

[gioman]

Select a group of 1000+ CADRG frame files (e.g. representing a single CADRG zone, such as zone 01 from a 1M scale product).

@bbarre9 were we can find such datasources?

[bbarre9]

Select a group of 1000+ CADRG frame files (e.g. representing a single CADRG zone, such as zone 01 from a 1M scale product).

@bbarre9 were we can find such datasources?

@gioman I just edited my original post: I attached an example dataset and edited the steps to reproduce accordingly.

[gioman]

@gioman I just edited my original post: I attached an example dataset and edited the steps to reproduce accordingly.

It is considerably faster also on 3.4. The main difference between then and now is the GDAL version used, which is used to handle such type of data. The regression likely lies there rather than in QGIS code.

[rouault]

The regression likely lies there rather than in QGIS code.

actually no :-) Pure QGIS issue. Fix in #50302