Memory leak using rasterio.Env

[eregnier]

Hi folks.

Thank you for your work on this project.

I meet an issue on our project where

while True:
    with rasterio.Env(**{}):
        pass

Generates a memory leak. When one run this code and look for process memory usage, it looks like memory is never released overtime.

By digging a bit more in the code, it looks like the following code is related to the issue:

https://github.com/rasterio/rasterio/blob/main/rasterio/env.py#L334

this is where __enter__ code generate a new GDAL environment instance

then

https://github.com/rasterio/rasterio/blob/main/rasterio/env.py#L338

is where (I guess) the memory allocation happens because there no more memory leak when this line is commented.

In my case this happens because I am instanciating this raster io environment into a web (fastapi) route callback where the with statement code is called on every route call.

I cannot do much in my code context in an obvious way with current rasterio.Env api which is usable only using a with statement.

For now I fixed my code with this code which is not clean but does the trick. I never release ressources from the __exit__ method but this is ok as we are in an ever living web route callback and I cache the rasterio.Env instance.

if getattr(self, "gdal_env", None) is None:
    rasterio_env = rasterio.Env(**env)
    self.gdal_env = rasterio_env.__enter__()

On the first call, I instanciate the rasterio context and it lives and is cached for all subsequent calls from self.gdal_env variable.

Thank you for any help solving this.

[sgillies]

@eregnier Thanks for the report! This memory leak is fixed in the main branch and I've backported for 1.3.9.

[eregnier]

Thank you very much !