[gdal] Don't try to download whole COG locally in order to determine open options

[nyalldawson]

Don't try to download whole COG locally in order to determine open options in Add Raster Layer dialog, if a "http" URL is entered without the /vsicurl/ prefix

If a user directly enters a HTTP URL into the "Raster datasets" option in the Data Source Manager, QGIS will otherwise hang while the whole remote file is retrieved.

Instead automatically update the entered URI to a proper /vsicurl/ one so that GDAL's optimisations for remote data sources can take effect.

[nyalldawson]

Can I grab your thoughts @rouault ?

[rouault]

That's always a bit tricky. That will work with COGs or other raster that are served by a HTTP range friendly server, but that will fail for unfriendly servers where only the ingest-the-whole-file strategy would work. But as noted the later is only practical for smallish files.
An "ideal" strategy would probably start with the /vsicurl/ one, and if that fails, fallback to the GDAL HTTP driver (as currently), but only for files "not too big" (which might not be easy to figure out quickly for non-cooperative servers). That said, that would seriously complicate the logic.
I guess your PR is a reasonable option that will be an improvement w.r.t current situation.

[nyalldawson]

@rouault

Hmm.... so the other option I thought of was that we just entirely skip the creation option population for http urls. Do you think that's a better solution?

[rouault]

skip the creation option population for http urls

open options you mean ? That would be reasonable too, at least for the GeoTIFF driver they are quite esoteric for QGIS use cases. But would the /vsicurl/ prefix be added when adding the file ?

[nyalldawson]

But would the /vsicurl/ prefix be added when adding the file ?

Ahh, I misunderstood your original concern -- it was with always adding the prefix, right?

So I propose:

• revert the auto-add prefix
• if a http(s) url without vsicurl prefix is entered, then always just clear the open options

[rouault]

it was with always adding the prefix, right?

yes, but actually trying (my previous comments were done without trying) with a relatively small file of 2 MB, adding without /vsicurl/ is barely usable

So I propose:
revert the auto-add prefix
if a http(s) url without vsicurl prefix is entered, then always just clear the open options

sounds good.

[nyalldawson]

@rouault
how about the latest iteration -- now we show a warning to users if a non-vsicurl http(s) uri is entered:

[rouault]

how about the latest iteration -- now we show a warning to users if a non-vsicurl http(s) uri is entered:

Great !