Raster Clip and Merge issues (QGIS 3.2.1)

[qgib]

Author Name: Loren Amelang (@LorenAmelang)
Original Redmine Issue: 19584
Affected QGIS version: 3.2.1
Redmine category:processing/qgis

---

Maybe this should be several separate issues, but it will be more understandable if I describe them in sequence...

I had three 400MB 1-meter DEM .tif files, and RiverGIS only lets you query one file name per project, so I wanted to consolidate them.

First I moved them to the project CRS with Raster -> Projections -> Warp (reproject). It altered the contrast of the visible DEM image, two got way paler, one slightly higher contrast. Is that proper?

Then I tried Raster -> Miscellaneous -> Merge, but got a Memory Error every time.

I thought I'd cut the file size, so I made a vector layer with the rectangular boundary of my project. I used Raster -> Extraction -> Clip Raster by Mask Layer, and it clipped the parts outside the boundary. BUT it made each output file the size of the full mask boundary! Is that expected? So they didn't get much smaller.

Merge still got a memory error each time. And, for each new try, I found I had to close the Merge window and start over from the QGIS menu bar. If I just edited a setting and tried again, I got "Incorrect parameter value for INPUT". It would still show the files selected, and re-selecting the input files didn't help.

Win 10 Resource Monitor showed zero memory faults, and just a tiny blip from 41% to 51% memory used, for these "Memory Error" tries, so I don't think it was a hardware limitation.

I tried Raster -> Extraction -> Clip Raster by Extent (Select extent on canvas)
First file worked, second showed correct "zoom to" extent but no image; under the Layers entry it showed:
1.79769e+308
-1.79769e+308
instead of
238.166
822.751
Again, turns out you can't edit and re-use a dialog! Must start over and re-enter everything each time.

After the manual clip, there were a couple of empty cracks near the boundaries where the segments should meet... They met perfectly on the screen before the clip. I found I has to draw my manual boundary at least 300' beyond the apparent edge of the DEM to get it all! Seems strange... Yes, all the files and the map image were in the same 2226 project CRS.

Also, if I tried to save to the same filename I had used in the last test, the .tif.aux.xml file would be saved, but not the actual new .tif! When I added a character to make a new filename, the .tif was saved with the new name, but no new .tif.aux.xml was saved under either name! (Looks like all of my .tif.aux.xml files have different timestamps from the matching .tif...)

I checked the Merge command before clicking run this time:

gdal_merge.bat -ot Float32 -of GTiff -o "C:/Users/loren/Giant Files/HEC-RAS Dam Break/QGIS Projects/FEMA808/DEM_2226_clip_man_merge.tif" --optfile C:/Users/loren/AppData/Local/Temp/processing_b1a1b0b73d594cc4a2747620a75d7b50\mergeInputFiles.txt

I was a bit suspicious of all the '/' dividers being valid in Windows... The file did exist:
C:\Users\loren\AppData\Local\Temp\processing_b1a1b0b73d594cc4a2747620a75d7b50\mergeInputFiles.txt
But it was blank before I clicked "go".

But it worked! With 225MB of input data instead of 955MB (or 1201MB input which had made an all-black 1565MB output file).

After the operation completed, the mergeInputFiles.txt file listed my input files.

One more curiosity... Each time I tried one of these operations with "Open Output File" checked it made another OGR error:

2018-08-09T17:56:41 WARNING Cannot open C:/Users/loren/Giant Files/HEC-RAS Dam Break/QGIS Projects/FEMA808/DEM_47_435_to2226_clip_manual.tif.()
2018-08-09T18:01:12 WARNING Cannot open C:/Users/loren/Giant Files/HEC-RAS Dam Break/QGIS Projects/FEMA808/DEM_47_436_to2226_clip_manual.tif.()
2018-08-09T18:11:12 WARNING Cannot open C:/Users/loren/Giant Files/HEC-RAS Dam Break/QGIS Projects/FEMA808/DEM_2226_clip_man_merge.tif.()

--> All of those files (and lots more before them) appeared to open normally in the Layers menu. What can this mean?

---

Related issue(s): #27421 (relates), #27422 (relates), #27423 (relates), #27424 (relates)
Redmine related issue(s): 19594, 19595, 19596, 19597

---

[qgib]

Author Name: Nyall Dawson (@nyalldawson)

---

Hi Loren,

Can you file these as separate distinct issues?

[qgib]

Author Name: Loren Amelang (@LorenAmelang)

---

Merge - #27421
Resize - #27422
Clip - #27423
OGRwarn - #27424

[qgib]

Author Name: Jürgen Fischer (@jef-n)

---

• relates was configured as "Memory Error" in Raster Merge #27421

[qgib]

Author Name: Jürgen Fischer (@jef-n)

---

• relates was configured as "Clip Raster by Mask Layer" is actually "Resize to mask layer" #27422

[qgib]

Author Name: Jürgen Fischer (@jef-n)

---

• relates was configured as Can't re-use "Clip Raster by Extent" dialog; boundary offset?  #27423

[qgib]

Author Name: Jürgen Fischer (@jef-n)

---

• relates was configured as OGR: "WARNING Cannot open <file>" when they seem to open fine? #27424

[qgib]

Author Name: Giovanni Manghi (@gioman)

---

Separate tickets have been filed, closing this.

---

• resolution was changed from to duplicate
• status_id was changed from Open to Closed