Download doesn't work anymore

[Rot90]

Hello,
I used sentinelsat until 4 days ago when this error showed up:

Traceback (most recent call last):
File "c:\users\crossi\appdata\local\continuum\anaconda2\lib\runpy.py", line 17
4, in _run_module_as_main
"main", fname, loader, pkg_name)
File "c:\users\crossi\appdata\local\continuum\anaconda2\lib\runpy.py", line 72
, in run_code
exec code in run_globals
File "C:\Users\crossi\AppData\Local\Continuum\Anaconda2\Scripts\sentinel.exe_
main.py", line 9, in
File "c:\users\crossi\appdata\local\continuum\anaconda2\lib\site-packages\clic
k\core.py", line 716, in call
return self.main(_args, *_kwargs)
File "c:\users\crossi\appdata\local\continuum\anaconda2\lib\site-packages\clic
k\core.py", line 696, in main
rv = self.invoke(ctx)
File "c:\users\crossi\appdata\local\continuum\anaconda2\lib\site-packages\clic
k\core.py", line 1060, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "c:\users\crossi\appdata\local\continuum\anaconda2\lib\site-packages\clic
k\core.py", line 889, in invoke
return ctx.invoke(self.callback, *_ctx.params)
File "c:\users\crossi\appdata\local\continuum\anaconda2\lib\site-packages\clic
k\core.py", line 534, in invoke
return callback(_args, *_kwargs)
File "c:\users\crossi\appdata\local\continuum\anaconda2\lib\site-packages\sent
inelsat\scripts\cli.py", line 83, in search
api.query(get_coordinates(geojson), start, end, *_search_kwargs)
File "c:\users\crossi\appdata\local\continuum\anaconda2\lib\site-packages\sent
inelsat\sentinel.py", line 130, in query
self.query_raw(query)
File "c:\users\crossi\appdata\local\continuum\anaconda2\lib\site-packages\sent
inelsat\sentinel.py", line 138, in query_raw
_check_scihub_response(self.content)
File "c:\users\crossi\appdata\local\continuum\anaconda2\lib\site-packages\sent
inelsat\sentinel.py", line 89, in _check_scihub_response
raise SentinelAPIError(response.status_code, code, msg, response.content)
sentinelsat.sentinel.SentinelAPIError: (HTTP status: 404, code: None) # HTTP Sta
tus 404 -

---

type Status report

message __

description The requested resource is not available.

---

Apache Tomcat/7.0.47

Any suggestions?

[kr-stn]

I suspect this is a regular service interruption due to maintenance activities throughout October. Basically they have a hard time handling the increased input data stream (Sentinel-1B) and the product format change for Sentinel-2.

As this is a regular 404 you can always check manually if the SciHub can be accessed through the web-interface and just retry after it is back online - or is this a persistent error?

[kalxas]

I can confirm this error here too

[valgur]

If you suspect that the error is on the sentinalsat side, can you share the command or a relevant part of the code?

[kalxas]

sorry, I did not mean that the error is in the library side, just confirming that the API is still down for me too.

[Opadera]

same here

[kr-stn]

Can you give us more info about your Python Environment, OS and the command you ran? Without that it is quite hard to reproduce the error. Unless the error was caused by the API only, in which case you could close the issue.

I was trying to reproduce the error on Win7 and Ubuntu 14.04 with Python 2.7 and 3.5 with the following environment:

edit: The download worked for me on all 4 environments.

click (6.6)
geojson (1.3.3)
homura (0.1.3)
html2text (2016.9.19)
humanize (0.5.1)
pip (8.1.2)
pycurl (7.43.0)
requests (2.11.1)
sentinelsat (0.7)
setuptools (27.2.0)
six (1.10.0)
tqdm (4.8.4)
wheel (0.29.0)

[Opadera]

Python 2.7
Win 10

sentinel search --sentinel2 -c 75 -s 20160918 user password aoi.json -d -p D:\aoi

packages were the same except for:

requests (2.10.0)
setuptools (23.0.0)

I had no errors with this last week.
I upgraded as available to

requests (2.11.1)
setuptools (26.1.1.post20160901)

same error using the command above.
@Fernerkundung does the download work for you currently?

[kalxas]

I am still getting this response from the API:
`

Traceback (most recent call last):
File "/home/user/bin/sentinel/bin/sentinel", line 11, in
sys.exit(cli())
File "/home/user/bin/sentinel/local/lib/python2.7/site-packages/click/core.py", line 716, in call
return self.main(_args, *_kwargs)
File "/home/user/bin/sentinel/local/lib/python2.7/site-packages/click/core.py", line 696, in main
rv = self.invoke(ctx)
File "/home/user/bin/sentinel/local/lib/python2.7/site-packages/click/core.py", line 1060, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/home/user/bin/sentinel/local/lib/python2.7/site-packages/click/core.py", line 889, in invoke
return ctx.invoke(self.callback, *_ctx.params)
File "/home/user/bin/sentinel/local/lib/python2.7/site-packages/click/core.py", line 534, in invoke
return callback(_args, *_kwargs)
File "/home/user/bin/sentinel/local/lib/python2.7/site-packages/sentinelsat/scripts/cli.py", line 83, in search
api.query(get_coordinates(geojson), start, end, *_search_kwargs)
File "/home/user/bin/sentinel/local/lib/python2.7/site-packages/sentinelsat/sentinel.py", line 130, in query
self.query_raw(query)
File "/home/user/bin/sentinel/local/lib/python2.7/site-packages/sentinelsat/sentinel.py", line 138, in query_raw
_check_scihub_response(self.content)
File "/home/user/bin/sentinel/local/lib/python2.7/site-packages/sentinelsat/sentinel.py", line 89, in _check_scihub_response
raise SentinelAPIError(response.status_code, code, msg, response.content)
sentinelsat.sentinel.SentinelAPIError: (HTTP status: 404, code: None) # HTTP Status 404 -

---

type Status report

message __

description The requested resource is not available.

---

Apache Tomcat/7.0.47

`

My environment is Ubuntu Xenial 16.04.1 and virtualenv has the following installed:
click==6.6
geojson==1.3.3
homura==0.1.3
html2text==2016.9.19
humanize==0.5.1
pkg-resources==0.0.0
pycurl==7.43.0
requests==2.11.1
sentinelsat==0.7
six==1.10.0
tqdm==4.8.4

[kr-stn]

@UNNIC

With that command I can reproduce your error and I think I found the culprit. It looks like the url needs to be explicitly set with -u for now. Will try to find the bug in the code.

In the meantime this should work sentinel search --sentinel2 -c 75 -s 20160918 -u "https://scihub.copernicus.eu/apihub" user password aoi.json -d -p D:\aoi

[Opadera]

awesome, it's running again.

[j08lue]

@Fernerkundung
The error is in this line: https://github.com/ibamacsr/sentinelsat/blob/d606ca91b3ceef8d1489ea8b26c1084c8b6c2782/sentinelsat/sentinel.py#L152

change that to

if api_url[-1] != '/':

and it will work. The problem is that api_url for some reason is of type unicode, while '/' is str. So is will give False, while == will give True.

[j08lue]

Do we need that _url_trail_slash at all? I'd think urljoin in line 132 will take care of that?

[kr-stn]

@j08lue _url_trail_slash is a function we used before we joined the URLs with urljoin. Well spotted on the incorrect type comparison - it can be removed w/o issue.

[j08lue]

I thought so @Fernerkundung. :-) Good to know you have no future plans with it. Then my rather brute approach of just removing it is perhaps OK. :-)

[kr-stn]

Solved with release of v0.7.1. Issue can be closed @willemarcel