identify feature on ecw extremely long

[qgib]

Author Name: Regis Haubourg (@haubourg)
Original Redmine Issue: 4594

Redmine category:data_provider

---

Hi dev's,
I'am using big ecw file (from 500Mo to 92 Go) and an Identify action on it take ages to answer band values (5 minutes sometimes).

• First question: is there any workaround to optimize it? is that a question dedicated to gdal developpers or Qgis dev's?

• Second Question: users do often get trap interrogating an ecw when they did not want to. That could be because of a bad habit coming from mapinfo OR identify mode different from "current layer" in settings). If there's no way to optimize the identify to respond within ~ 3 seconds, is there any solution to set ecw "not identifiable" by default. It's possible in project properties by hand, but users won't do that. that's why a default "identifiable value" to 'off' in ecw addlayers method would be perfect.

All the best Régis

[qgib]

Author Name: Giovanni Manghi (@gioman)

---

With not that big ecws (~100mb) the identify operation is normal fast. Can you test with smaller images and/or with different formats?

---

• status_id was changed from Open to Feedback

[qgib]

Author Name: Regis Haubourg (@haubourg)

---

Hi Ghiovanni,
here are result tests:

QGIS 1.7.2, on the fly "off", Xeon Windows xp 32 bits. ecw on NAS servers via LAN 100Mbit/s:

• scan 1 / Size: 139Mo / pixels : 12150 * 11000 (sdk ecw 4.1) / identify delay: 1s / RAM consumption : opening ecw + 50Mo, each identify action + ~ 15 Mo. Memory not released after identify

• scan 2 / Size: 525Mo/ pixels : 30000*30000 (sdk ecw 4.1) / identify delay: 4s / RAM consumption : opening ecw + 10Mo, each identify action + ~ 15 Mo. Memory not released after identify

• scan 3 / Size: 870Mo/ pixels : 60000*60000 (sdk ecw 4.1) / identify delay: 6s / RAM consumption : opening ecw + 30Mo, each identify action + ~ 15 Mo. Memory not released after identify

• scan 4 / Size: 1,05Go/ pixels : 220 000* 220 000 (sdk ecw 4.1) / identify delay: 20s / RAM consumption : opening ecw + 30Mo, identify action rises once RAM to 663 Mo! / Memory not released after identify

• aerial views / Size: 74 Go/ pixels : 1 080 000* 1 090 000 (sdk ecw 4.1) / identify delay: 50s / RAM consumption : opening ecw + 30Mo, identify action rises once RAM to 663 Mo! / Memory not released after identify. Many crashes if on-the_fly reprojection is "On" (true also on 1.7.1).

I tested 1.7.1 on a windows server 2003, with gigabit access to datas, (vmware environnement). only identify delay is shortened to 30 s instead of 50s. Crashes sometimes, 660 Mo of RAM consumption.

I guess gdal or Qgis uncompress datas and also have a memory leak. I have not noticed any improvement identifying on a hih or low scale (more pixels inside cursor tolerance?)

Régis

[qgib]

Author Name: Nathan Woodrow (@NathanW2)

---

Just a note: The large memory usage with ecws is due to the caching the gdal driver does. We could turn this off as I don't find it improves performance a lot.

[qgib]

Author Name: Regis Haubourg (@haubourg)

---

Thanks Nathan, I didn't find in source code where to toggle gdal cache off. Any idea? I also wanted to test the identify query directly in gdal command line. What is the query send by qgis to gdal?
Régis

[qgib]

Author Name: Giovanni Manghi (@gioman)

---

• fixed_version_id was changed from Version 1.8.0 to Version 2.0.0

[qgib]

Author Name: Radim Blazek (@blazek)

---

The problem was that identify requested single pixel by GDALRasterIO() and GDAL ECW driver reads and caches whole line if requested block height is 1.

It should be fixed in master a795a5d.

Please test and close if it is resolved.

[qgib]

Author Name: Regis Haubourg (@haubourg)

---

Problem is solved for me on a795a5d. Thanks a lot guys. Can you close the bug? I don't have permissions apparently.

[qgib]

Author Name: Giovanni Manghi (@gioman)

---

• resolution was changed from to fixed
• status_id was changed from Feedback to Closed