onda: error with the geometry parameter

[maximlt]

The geometry passed to onda doesn't seem to work anymore, try for instance the following request (cuont request) made to onda in its end to end test:

https://catalogue.onda-dias.eu/dias-catalogue/Products?productType=S2MSI1C&$format=json&$search=%22footprint:%22Intersects%28POLYGON%20%28%280.2564%2043.1956%2C%200.2564%2043.9078%2C%202.3798%2043.9078%2C%202.3798%2043.1956%2C%200.2564%2043.1956%29%29%29%22%20AND%20productType:S2MSI1C%20AND%20beginPosition:[2021-03-12T00:00:00.000Z%20TO%20%2A]%20AND%20endPosition:[%2A%20TO%202021-03-19T00:00:00.000Z]%22

The returned error is:

"Error from server : Unable to parse shape given formats \"lat,lon\", \"x y\" or as WKT because java.text.ParseException: java.lang.UnsupportedOperationException: Unsupported shape of this SpatialContext. Try JTS or Geo3D. input: POLYGON ((0.2564 43.1956, 0.2564 43.9078, 2.3798 43.9078, 2.3798 43.1956, 0.2564 43.1956))"

[sbrunato]

https://www.onda-dias.eu/cms/updates-on-major-incident-at-ovhcloud-infrastructure-12-march-2021/

a limited subset of metadata is exposed and searchable for the time being: Id, Name, ContentLength, CreationDate, ContentDate – Start, ContentDate – Stop, offline, size. The full set of metadata will be made accessible gradually

Onda has been impacted by the incident on OVHcloud datacenter. We may wait that all catalog features are operational again

[maximlt]

Ok, I'll keep this issue open if someone else stumbles upon the same error, until the problem is fixed by onda,

[remi-braun]

I got a (maybe) a linked problem :)

• when querying with a point (like with S2 tiles in your tuto):
  Skipping error while counting results for onda ODataV4Search instance: [b'{"error":{"code":null,"message":"Exception during tokenizer creation with message \'Forbidden character in state PHRASE->{\'."}}']
  This error comes from the point geometry:
  
• When querying with a polygon, I got an empty query every time

[maximlt]

Would you happen to have the URL that returned this error? It's displayed in the logs when you set it (set_logging(verbose=3)).

It seems that onda hasn't yet completely recovered from the OVH issue. But we also fixed a bug in EODAG where the locations parameter was wrongly propagated as a special query kwarg (#186).

[remi-braun]

• Point geometry:

2021-03-23 08:07:55,412-15s eodag.core                       [INFO    ] (core             ) Searching product type 'S2_MSI_L1C' on provider: onda
2021-03-23 08:07:55,412-15s eodag.core                       [DEBUG   ] (core             ) Using plugin class for search: ODataV4Search
2021-03-23 08:07:55,412-15s eodag.plugins.search.qssearch    [DEBUG   ] (qssearch         ) Mapping eodag product type to provider product type
2021-03-23 08:07:55,412-15s eodag.plugins.search.qssearch    [DEBUG   ] (qssearch         ) Getting provider product type definition parameters for S2_MSI_L1C
2021-03-23 08:07:55,412-15s eodag.plugins.search.qssearch    [DEBUG   ] (qssearch         ) Building the query string that will be used for search
2021-03-23 08:07:55,412-15s eodag.plugins.search.qssearch    [DEBUG   ] (qssearch         ) Retrieving queryable metadata from metadata_mapping
2021-03-23 08:07:55,412-15s eodag.plugins.search.qssearch    [INFO    ] (qssearch         ) Sending count request: https://catalogue.onda-dias.eu/dias-catalogue/Products/$count?productType=S2MSI1C&$search=%22footprint:%22Intersects%28POINT%20%287.0456%2048.2417%29%29%22%20AND%20productType:S2MSI1C%20AND%20beginPosition:[2021-01-13T00:00:00.000Z%20TO%20%2A]%20AND%20endPosition:[%2A%20TO%202021-02-25T00:00:00.000Z]%20AND%20locations:%7B%27s2_tiles_centroids%27:%20%2732ULU%27%7D%22
2021-03-23 08:07:55,529-15s eodag.plugins.search.qssearch    [ERROR   ] (qssearch         ) Skipping error while counting results for onda ODataV4Search instance: [b'{"error":{"code":null,"message":"Exception during tokenizer creation with message \'Forbidden character in state PHRASE->{\'."}}']
Traceback (most recent call last):
  File "C:\Users\rbraun\Anaconda3\envs\eodownload\lib\site-packages\eodag\plugins\search\qssearch.py", line 610, in _request
    urllib_response = urlopen(prep.url)
  File "C:\Users\rbraun\Anaconda3\envs\eodownload\lib\urllib\request.py", line 222, in urlopen
    return opener.open(url, data, timeout)
  File "C:\Users\rbraun\Anaconda3\envs\eodownload\lib\urllib\request.py", line 531, in open
    response = meth(req, response)
  File "C:\Users\rbraun\Anaconda3\envs\eodownload\lib\urllib\request.py", line 641, in http_response
    'http', request, response, code, msg, hdrs)
  File "C:\Users\rbraun\Anaconda3\envs\eodownload\lib\urllib\request.py", line 569, in error
    return self._call_chain(*args)
  File "C:\Users\rbraun\Anaconda3\envs\eodownload\lib\urllib\request.py", line 503, in _call_chain
    result = func(*args)
  File "C:\Users\rbraun\Anaconda3\envs\eodownload\lib\urllib\request.py", line 649, in http_error_default
    raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 400: 

• Polygon geometry:

2021-03-23 08:06:13 - [INFO] - Querying L8_OLI_TIRS_C1L1 products.
2021-03-23 08:06:13,154-15s eodag.core                       [INFO    ] (core             ) Searching product type 'L8_OLI_TIRS_C1L1' on provider: onda
2021-03-23 08:06:13,154-15s eodag.core                       [DEBUG   ] (core             ) Using plugin class for search: ODataV4Search
2021-03-23 08:06:13,154-15s eodag.plugins.search.qssearch    [DEBUG   ] (qssearch         ) Mapping eodag product type to provider product type
2021-03-23 08:06:13,154-15s eodag.plugins.search.qssearch    [DEBUG   ] (qssearch         ) Getting provider product type definition parameters for L8_OLI_TIRS_C1L1
2021-03-23 08:06:13,154-15s eodag.plugins.search.qssearch    [DEBUG   ] (qssearch         ) Building the query string that will be used for search
2021-03-23 08:06:13,154-15s eodag.plugins.search.qssearch    [DEBUG   ] (qssearch         ) Retrieving queryable metadata from metadata_mapping
2021-03-23 08:06:13,155-15s eodag.plugins.search.qssearch    [INFO    ] (qssearch         ) Sending count request: https://catalogue.onda-dias.eu/dias-catalogue/Products/$count?platform=Landsat-%2A&$search=%22footprint:%22Intersects%28POLYGON%20%28%28-0.9570%2043.6241%2C%20-1.2143%2043.6305%2C%20-1.2127%2043.6866%2C%20-0.8368%2043.8658%2C%20-0.7765%2043.8667%2C%20-0.7765%2043.7476%2C%20-0.9570%2043.6241%29%29%29%22%20AND%20platformName:Landsat-%2A%20AND%20beginPosition:[2021-02-01T00:00:00.000Z%20TO%20%2A]%20AND%20endPosition:[%2A%20TO%202021-03-23T08:06:12.000Z]%22
2021-03-23 08:06:13,355-15s eodag.plugins.search.qssearch    [INFO    ] (qssearch         ) Sending search request: https://catalogue.onda-dias.eu/dias-catalogue/Products?platform=Landsat-%2A&$format=json&$search=%22footprint:%22Intersects%28POLYGON%20%28%28-0.9570%2043.6241%2C%20-1.2143%2043.6305%2C%20-1.2127%2043.6866%2C%20-0.8368%2043.8658%2C%20-0.7765%2043.8667%2C%20-0.7765%2043.7476%2C%20-0.9570%2043.6241%29%29%29%22%20AND%20platformName:Landsat-%2A%20AND%20beginPosition:[2021-02-01T00:00:00.000Z%20TO%20%2A]%20AND%20endPosition:[%2A%20TO%202021-03-23T08:06:12.000Z]%22&$top=500&$skip=0
2021-03-23 08:06:13,559-15s eodag.plugins.search.qssearch    [DEBUG   ] (qssearch         ) Adapting 0 plugin results to eodag product representation
2021-03-23 08:06:13,559-15s eodag.core                       [INFO    ] (core             ) Found 0 result(s) on provider 'onda'

[sbrunato]

@remi-braun you've got the error when searching using POINT GEOMETRY because locations:{'s2_tiles_centroids': '32ULU'} is still in the search URL. As mentions @maximlt , this issue has been fixed (see #186) on develop branch

[maximlt]

The usage of passing the locations query directly as kwargs, introduced in version 2.0, was still preserved for backwards compatibility, even if I think we'd like to remove it at some point and we have removed it from the docs. So until we release the next version, instead of:

dag.search(locations=dict(s2_tiles_centroids="xxx"))

you could use:

dag.search(s2_tiles_centroids="xxx")

[remi-braun]

Yes OK I will do that.
But do you manage to query sth with ONDA and any polygon ?

[maximlt]

I just tried the link in my original post and it still doesn't work. I also tried to to search for products on their catalog by drawing a polygon and got an "Invalid Search Text" . I guess they're still down

[sbrunato]

The problem seems now to be related to products availability, and not a problem with geometry search:

dags_L1C, _ = dag.search(productType="S2_MSI_L1C",  start='2015-01-01', end='2021-02-26')

2021-03-23 11:32:26,464-15s eodag.core [INFO ] (core ) Found 0 result(s) on provider 'onda'

dags_L1C, _ = dag.search(productType="S2_MSI_L1C",  start='2021-02-26', end='2021-02-27')

2021-03-23 11:36:48,833-15s eodag.core [INFO ] (core ) Found 431 result(s) on provider 'onda'

dags_L1C, _ = dag.search(productType="S2_MSI_L1C",  start='2021-02-26', end='2021-02-27', 
                             locations=dict(country="FRA"))

2021-03-23 11:45:32,170-15s eodag.core [INFO ] (core ) Found 6 result(s) on provider 'onda'

[remi-braun]

https://www.onda-dias.eu/cms/updates-on-major-incident-at-ovhcloud-infrastructure-12-march-2021/

Please also note that the following services are currently not available: Advanced API (ENS), View, Planned Acquisition View, CLEOPE. Restart of these services will take place during week 12, starting next Monday, 22 March.

Wait and see, then 😄

[sbrunato]

Onda search on old products works know well:

dags_L1C, _ = dag.search(productType="S2_MSI_L1C",  start='2015-01-01', end='2021-02-26')

2021-08-17 10:19:09,650-15s eodag.core [INFO ] Found 15786129 result(s) on provider 'onda'

And so does POINT geometry search:

dags_L1C, _ = dag.search(productType="S2_MSI_L1C",  start='2015-01-01', end='2021-02-26', geom="POINT(123.45 43.21)")

2021-08-17 10:19:35,205-15s eodag.core [INFO ] Found 547 result(s) on provider 'onda'

See https://www.onda-dias.eu/cms/updates-on-major-incident-at-ovhcloud-infrastructure/