DEM download takes a long time

[jlmaurer]

The DEM download can take the longest to run of the whole code.

Starting line in RAiDER/tools/RAiDER/demdownload.py:

RAiDER/tools/RAiDER/demdownload.py

Line 59 in 4c66748

memRaster = '/vsimem/warpedDEM'

Description

DEM download takes a long time. Not sure why this is happening; but could be that passing boundaries as a warp option is not as fast as cropping a VRT?

Relevant code

_world_dem = ('https://cloud.sdsc.edu/v1/AUTH_opentopography/Raster/'
              'SRTM_GL1_Ellip/SRTM_GL1_Ellip_srtm.vrt')
inRaster ='/vsicurl/{}'.format(_world_dem)
memRaster = '/vsimem/warpedDEM'
wrpOpt = gdal.WarpOptions(outputBounds = (minlon, minlat,maxlon, maxlat))
gdal.Warp(memRaster, inRaster, options = wrpOpt)
out = RAiDER.utilFcns.gdal_open(memRaster)

[dbekaert]

@sssangha can you provide some insight from ARIA-tools experience?
Might first generate a cropped vrt and then do a warp in mem on the cropped vrt?
Recall @bbuzz31 had some similar issue for the nlc map.

[bbuzz31]

I would recommend replacing the gdal.Warp command (and wrpOpt) with:
gdal.BuildVRT(memRaster, inRaster, outputBounds=[minlon, minlat, maxlon, maxlat])
This seems to increase the access speed, but when the array is read in utilFcns.gdal_open() it will still take awhile for a large region.

[dbekaert]

Agreed; the first vrt makes it such the warp only need to take into memory the part that is needed. This makes warping much more efficient.

…

On Fri, 26 Jun 2020 at 17:14, BB ***@***.***> wrote: I would recommend replacing the gdal.Warp command (and wrpOpt) with: gdal.BuildVRT(memRaster, inRaster, outputBounds=[minlon, minlat, maxlon, maxlat]) This seems to increase the access speed, but when the array is read in utilFcns.gdal_open() it will still take awhile for a large region. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#55 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AESZPSPIBVCQM7X673NTHQLRYU2W3ANCNFSM4OHK3P6Q> .

[jlmaurer]

Thanks @bbuzz31 and @dbekaert I'll implement that change and push it to the repo!

[piyushrpt]

If you are not transforming map projections, using translate is going to be lot faster than warp

[jlmaurer]

Thanks @piyushrpt, we implemented the virtual raster and seems to be working well!

I just noticed though that the DEM we download is ellipsoidal heights; should we be doing an ellipsoidal correction as is done in ISCE?

[dbekaert]

@jlmaurer see here for details open topogrpahy
SRTM GL1 (30m Ellipsoidal) is a version of the popular Shuttle Radar Topography Mission dataset where elevation values are WGS84 ellipsoidal height as opposed to the standard orthometric, or geoid-referenced, elevation.

This is correct as input for tropospheric delays estiamtion, and does not need any more correction. @piyushrpt can you confirm my statement?

[dbekaert]

Will close it out, @piyushrpt let us know in care you disagree.