Overwriting COG causes Titiler to return 500 until restart

[mradamcox]

Hi! I've been integrating Titiler into an existing project and it's been great so far, but I'm encountering something that may be a bug or perhaps I'm just missing something. If you think it's the former, I'm happy to write up a more detailed ticket.

Part of my workflow involves (re)generating a COG and copying that file over an existing COG. This means that the url I provide to Titiler doesn't change, but the file it is reading does. This causes a 500 error, until I restart Titiler, and then it reads the new file fine (I have it running locally with uvicorn). So I'm wondering, is there a "force reload" operation or something? I don't have any caching set up, but it seems like Titiler is expecting something from the file after it has read it once, and errors if gets something different.

In the /docs preview, when I try the preview the COG after changing it, I get this error message:

"detail": "Read or write failed. IReadBlock failed at X offset 0, Y offset 0: IReadBlock failed at X offset 0, Y offset 0: TIFFReadEncodedTile() failed."

When I restart Titiler and run the same preview request, it works fine.

Thank you! I really appreciate this project and have been enjoying working with it.

[vincentsarago]

@mradamcox this is interesting! I believe this is due to the GDAL Cache

In addition, a global least-recently-used cache of 16 MB shared among all downloaded content is enabled by default, and content in it may be reused after a file handle has been closed and reopen, during the life-time of the process or until VSICurlClearCache() is called. Starting with GDAL 2.3, the size of this global LRU cache can be modified by setting the configuration option CPL_VSIL_CURL_CACHE_SIZE (in bytes).

This is unfortunate. You could try setting CPL_VSIL_CURL_CACHE_SIZE=0

[vincentsarago]

Note: VSICurlClearCache() is not available in rasterio (we could open an issue asking if we can add it)

[mradamcox]

Hi Vincent, I appreciate the quick reply! Interesting about the cache, it makes sense though. I have tried starting Titiler like this:

CPL_VSIL_CURL_CACHE_SIZE=0 uvicorn titiler.application.main:app --port 8008

But the problem remains. I also tried adding GDAL_CACHEMAX=0 but no difference. Is there somewhere else I should be changing these configuration options?

[vincentsarago]

GDAL_CACHEMAX=0 will only have influence on the block cache (which get cleaned when closing the file.)

can you try VSI_CACHE=FALSE

[mradamcox]

VSI_CACHE=FALSE makes no difference either. Perhaps there is a url param I could append that would make Titiler think it's a new file? I'll poke around.

[vincentsarago]

try CPL_VSIL_CURL_CACHE_SIZE=1

[mradamcox]

Still no change. I think I'll change my workflow to append a random string to the new COG name and delete the old one instead of overwriting. Not ideal, but overall it's a pretty low-effort fix to get around the cache for now. Is there somewhere I should put a ticket related to this?

[mradamcox]

(In this particular case, the files are handled with Django so technically their names don't matter, I just want to have them as uniform as possible.)