Open remote cog file in docker image 'aliyunfc/runtime-python3.6' raise an error

[yuhangch]

Expected behavior and actual behavior.

I expected to read the profile from a cog file store in alibaba OSS and raise an error below:

< HTTP/1.1 206 Partial Content
< Server: AliyunOSS
< Date: Wed, 01 Apr 2020 15:12:14 GMT
< Content-Type: image/tiff
< Content-Length: 16384
< Connection: keep-alive
< x-oss-request-id: 5E84AF4E0D92D90586C75D52
< Content-Range: bytes 0-16383/99400948
< Accept-Ranges: bytes
< ETag: "533480C231D9644E5024D00E2D5D4732"
< Last-Modified: Wed, 01 Apr 2020 05:22:10 GMT
< x-oss-object-type: Normal
< x-oss-hash-crc64ecma: 15205173349628656431
< x-oss-storage-class: Standard
< x-oss-server-time: 1
<
* Connection #0 to host bucketname.oss-cn-hongkong.aliyuncs.com left intact
Traceback (most recent call last):
  File "rasterio/_base.pyx", line 216, in rasterio._base.DatasetBase.__init__
  File "rasterio/_shim.pyx", line 67, in rasterio._shim.open_dataset
  File "rasterio/_err.pyx", line 205, in rasterio._err.exc_wrap_pointer
**rasterio._err.CPLE_AppDefinedError: Cannot open TIFF file due to missing codec.**

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "test/oss.py", line 10, in <module>
    with rasterio.open("https://bucketname.oss-cn-hongkong.aliyuncs.com/origin.tif") as src:
  File "/code/.fun/python/lib/python3.6/site-packages/rasterio/env.py", line 434, in wrapper
    return f(*args, **kwds)
  File "/code/.fun/python/lib/python3.6/site-packages/rasterio/__init__.py", line 219, in open
    s = DatasetReader(path, driver=driver, sharing=sharing, **kwargs)
  File "rasterio/_base.pyx", line 218, in rasterio._base.DatasetBase.__init__
rasterio.errors.RasterioIOError: Cannot open TIFF file due to missing codec.

Alibaba function compute local debug logs show below :

* Connection #0 to host bucketname.oss-cn-hongkong.aliyuncs.com left intact
**2020-04-01T15:00:14.38Z 1519e3af-5436-4b7e-ab28-d442f30e828f [WARNING] CPLE_AppDefined in /vsicurl/https://bucketname.oss-cn-hongkong.aliyuncs.com/origin.tif:ZSTD compression support is not configured**
2020-04-01T15:00:14.55Z 1519e3af-5436-4b7e-ab28-d442f30e828f [ERROR] {
    "errorMessage": "Cannot open TIFF file due to missing codec.",
    "errorType": "RasterioIOError",
    "stackTrace": [
        [
            "File \"/code/index.py\"",
            "line 31",
            "in handler",
            "with rasterio.open(\"https://bucketname.oss-cn-hongkong.aliyuncs.com/origin.tif\") as src:"
        ],
        [
            "File \"/code/.fun/python/lib/python3.6/site-packages/rasterio/env.py\"",
            "line 434",
            "in wrapper",
            "return f(*args, **kwds)"
        ],
        [
            "File \"/code/.fun/python/lib/python3.6/site-packages/rasterio/__init__.py\"",
            "line 219",
            "in open",
            "s = DatasetReader(path, driver=driver, sharing=sharing, **kwargs)"
        ],
        [
            "File \"rasterio/_base.pyx\"",
            "line 218",
            "in rasterio._base.DatasetBase.__init__",
            ""
        ]
    ]
}

Steps to reproduce the problem.

Use docker image : aliyunfc/runtime-python3.6

$ pip install rasterio

Run scripts :

import rasterio
from rasterio.session import OSSSession

with rasterio.Env(CPL_DEBUG=True, CPL_CURL_VERBOSE=True,
                  session=OSSSession(oss_access_key_id='id',
                                     oss_secret_access_key='secret)):
    with rasterio.open("https://bucketname.oss-cn-hongkong.aliyuncs.com/origin.tif") as src:
        print(src.profile)

On my mac (10.15) ,create a new env ,install rasterio then run scripts above , works well, so it may caused by environment , or missed some dependencies,,sadly, I have no idea to make it work.

Operating system

Docker image : aliyunfc/runtime-python3.6 .
Dockerfile link

Rasterio version and provenance

v 1.1.3 installed from PyPI using pip 20.0.2.

How to adjust the environment or install dependencies to make it work? Thanks.

[sgillies]

@yuhangch I found the problem in this warning: CPLE_AppDefined in /vsicurl/https://bucketname.oss-cn-hongkong.aliyuncs.com/origin.tif:ZSTD compression support is not configured.

The rasterio wheels on PyPI do not include support for ZSTD compression and so the origin.tif file cannot be read. I would guess that the rasterio installed on your mac comes from conda or you built it from a source distribution?

To get support for ZSTD in your containers you will need to install everything -- GDAL, libtiff, zstd, rasterio -- from source. Or use conda. Or you could switch from ZSTD to DEFLATE, LZW, WebP, or JPEG compression.

[yuhangch]

Thanks again, after switching the compression type, the script worked, I will try to install everything to make zstd available.