sentinelsat crashing when using command line and -d option

[Opadera]

When i use
sentinel search --sentinel2 -s 20170115 user pwd aoi.json -d
i get the output

Will download 8 products Downloading [...]
and a corresponding ZIP file is created
then python crashes without an error (python.exe stopped responding)

I reinstalled python and checked if homura, PycURL and certifi are updated.

Windows 10
Python 2.7.12

[valgur]

Can you provide the aoi.json file that you used? I'll try to reproduce the issue.

[kr-stn]

Does it crash when you reorder the arguments to sentinel search <options> <user> <passwd> <aoi>?

sentinel search --sentinel2 -s 20170115 -d user pwd aoi.json

[Opadera]

I just tried on another machine where i had a working installation of sentinelsat from a while ago and there the same command works. Both list sentinelsat 0.7.4. Anything I have to consider when installing python and sentinelsat? I did a complete reinstallation of python and pip install sentinelsat

@valgur
it happens with any json. I create them with http://geojson.io

@Fernerkundung
yes, it still crashes.

[valgur]

I'm fairly certain that this bug happens in pycurl, since any interpreter crashes almost always happen due to issues in external C code. I have no idea what might be going on in there, but if you are willing to add some debugging code inside homura, it would help a ton.
Can you add

    def curl(self):
        """Sending a single cURL request to download"""
        c = self._pycurl
# --- snippet start ---
        def debug(debug_type, debug_msg):
            print("debug({}): {}".format(debug_type, debug_msg))
        c.setopt(c.VERBOSE, 1)
        c.setopt(c.DEBUGFUNCTION, debug)
# --- snippet end ---
        # Resume download
        if os.path.exists(self.path) and self.resume:

to around line 177 in <python root>\Lib\site-packages\homura.py, rerun the command and report any received output here? If there happens to be a Set-Cookie header string in that output, then you should probably remove that one, since it contains your session data which can be used to hijack it.

[Opadera]

done. I removed some information for security/anonymity. If you see something else please let me know.

debug(0): TCP_NODELAY set

debug(0): Connected to scihub.copernicus.eu (131.176.236.28) port 443 (#0)

debug(0): schannel: SSL/TLS connection with scihub.copernicus.eu port 443 (step 1/3)

debug(0): schannel: checking server certificate revocation

debug(0): schannel: sending initial handshake data: sending 201 bytes...

debug(0): schannel: sent initial handshake data: sent 201 bytes

debug(0): schannel: SSL/TLS connection with scihub.copernicus.eu port 443 (step 2/3)

debug(0): schannel: failed to receive handshake, need more data

debug(0): schannel: SSL/TLS connection with scihub.copernicus.eu port 443 (step 2/3)

debug(0): schannel: encrypted data buffer: offset 4096 length 4096

debug(0): schannel: encrypted data length: 4034

debug(0): schannel: encrypted data buffer: offset 4034 length 4096

debug(0): schannel: received incomplete message, need more data

debug(0): schannel: SSL/TLS connection with scihub.copernicus.eu port 443 (step 2/3)

debug(0): schannel: encrypted data buffer: offset 4861 length 5058

debug(0): schannel: sending next handshake data: sending 314 bytes...

debug(0): schannel: SSL/TLS connection with scihub.copernicus.eu port 443 (step 2/3)

debug(0): schannel: encrypted data buffer: offset 254 length 5058

debug(0): schannel: SSL/TLS handshake complete

debug(0): schannel: SSL/TLS connection with scihub.copernicus.eu port 443 (step 3/3)

debug(0): schannel: stored credential handle in session cache

debug(0): Server auth using Basic with user [user]

debug(2): GET /apihub/odata/v1/Products('798e5aa6-4a13-4c95-8658-f1db69d484c9')/$value HTTP/1.1
Host: scihub.copernicus.eu
Authorization: Basic dW5uaWM6dGluZ2EkNTU=
User-Agent: homura/0.1.5
Accept: */*
Cookie: dhusIntegrity=[id]; dhusAuth=[id]; JSESSIONID=[id]


debug(0): schannel: client wants to read 16384 bytes

debug(0): schannel: encdata_buffer resized 17408

debug(0): schannel: encrypted data buffer: offset 0 length 17408

debug(0): schannel: encrypted data got 5840

debug(0): schannel: encrypted data buffer: offset 5840 length 17408

debug(0): schannel: failed to decrypt data, need more data

debug(0): schannel: schannel_recv cleanup

debug(0): schannel: client wants to read 16384 bytes

debug(0): schannel: encrypted data buffer: offset 5840 length 17408

debug(0): schannel: encrypted data got 1460

debug(0): schannel: encrypted data buffer: offset 7300 length 17408

debug(0): schannel: failed to decrypt data, need more data

debug(0): schannel: schannel_recv cleanup

debug(0): schannel: client wants to read 16384 bytes

debug(0): schannel: encrypted data buffer: offset 7300 length 17408

debug(0): schannel: encrypted data got 10108

debug(0): schannel: encrypted data buffer: offset 17408 length 17408

debug(0): schannel: decrypted data length: 15341

debug(0): schannel: decrypted data added: 15341

debug(0): schannel: decrypted data cached: offset 15341 length 16384

debug(0): schannel: encrypted data length: 2042

debug(0): schannel: encrypted data cached: offset 2042 length 17408

debug(0): schannel: decrypted data length: 1533

debug(0): schannel: decrypted data added: 1533

debug(0): schannel: decrypted data cached: offset 16874 length 16874

debug(0): schannel: encrypted data length: 484

debug(0): schannel: encrypted data cached: offset 484 length 17408

debug(0): schannel: encrypted data buffer: offset 484 length 17408

debug(0): schannel: decrypted data buffer: offset 16874 length 16874

debug(0): schannel: schannel_recv cleanup

debug(0): schannel: decrypted data returned 16384

debug(0): schannel: decrypted data buffer: offset 490 length 16874

debug(1): HTTP/1.1 200 OK

debug(1): Server: Apache-Coyote/1.1

debug(1): Pragma: no-cache

debug(1): Accept-Ranges: bytes

debug(1): Content-Disposition: inline;filename="S2A_MSIL1C_20170115T094331_N0204_R036_T34TES_20170115T094646.zip"

debug(1): Content-Range: bytes 0-422242230/422242231

debug(1): DataServiceVersion: 2.0

debug(1): Date: Wed, 25 Jan 2017 09:57:06 GMT

debug(1): ETag: 432D3721E1CF2FD1BE6D60009F2F0287

debug(1): Expires: Wed, 01 Feb 2017 09:57:06 UTC

debug(1): Last-Modified: Tue, 17 Jan 2017 14:14:30 UTC

debug(1): Content-Type: application/octet-stream

debug(1): Content-Length: 422242231

debug(1):

debug(3): [a bunch of gibberish]

[valgur]

Thanks a lot, @UNNIC! The crash appears to occur in the middle of downloading and, unfortunately, the only way I could debug this further is if I manage to replicate the situation on my machine. I'm afraid there is nothing more that can be done right now.

[Opadera]

closed as the issue is continued in https://github.com/ibamacsr/sentinelsat/issues/91