clear_scroll failing on 2.4.0

[slydon-apixio]

Just upgraded to 2.4.0 and clear_scroll is failing.

I see in the traceback:
client.clear_scroll(body={'scroll_id': [scroll_id]}, ignore=(404, ))

But the warning log I see says a 400 was returned.

The following is the exception type:
elasticsearch.exceptions.RequestError

Did the usage change for elasticsearch.helpers.scan? Is this a problem with my application or with the implementation of clear_scroll?

Edit: I should have said that I upgraded from 2.1.0. I downgraded for now.

[honzakral]

Thank you for the report, I am, however, having trouble replicating this exception. Could you please provide more information on how to replicate this?

Ideally also a full content of the exception obtained from it's .info attribute.

Also, what elasticsearch version are you running?

Thanks!

[rshkv]

Hi,

I'm having the same issue. I upgraded my client from 2.3 to 2.4. The elasticsearch version is 1.5 (can't update).

The .info attribute is:

{
 'status': 400, 
 'error': 'ElasticsearchIllegalArgumentException[Failed to decode scrollId]; nested: IOException[Bad Base64 input character decimal 123 in array position 0]; '
}

Doesn't look like the error lies with the parameters I pass to helpers.scan().
This is the whole stack trace I'm getting:

Thank you!

[rafacustodio]

I'm using the ES 1.6 and I'm having the same issue (can't update either)

W 2016-09-29 17:00:03,843 elasticsearch DELETE /_search/scroll [status:400 request:0.003s]
E 2016-09-29 17:00:03,843 tornado.application Exception in callback <_functools.partial object at 0x00000001069d9018>
Traceback (most recent call last):
  File "/Users/rafaelcustodio/opt/rask_00k/site-packages/tornado/ioloop.py", line 604, in _run_callback
    ret = callback()
  File "/Users/rafaelcustodio/opt/pypy/lib_pypy/_functools.py", line 42, in __call__
    return self._func(*(self._args + fargs), **fkeywords)
  File "/Users/rafaelcustodio/opt/rask_00k/site-packages/tornado/stack_context.py", line 275, in null_wrapper
    return fn(*args, **kwargs)
  File "/Users/rafaelcustodio/alpha/correios_tracking_cronpoke/correios_tracking_cronpoke/listener.py", line 25, in iterate
    hit = hits.next()
  File "/Users/rafaelcustodio/opt/rask_00k/site-packages/elasticsearch/helpers/__init__.py", line 314, in scan
    client.clear_scroll(body={'scroll_id': [scroll_id]}, ignore=(404, ))
  File "/Users/rafaelcustodio/opt/rask_00k/site-packages/elasticsearch/client/utils.py", line 69, in _wrapped
    return func(*args, params=params, **kwargs)
  File "/Users/rafaelcustodio/opt/rask_00k/site-packages/elasticsearch/client/__init__.py", line 688, in clear_scroll
    params=params, body=body)
  File "/Users/rafaelcustodio/opt/rask_00k/site-packages/elasticsearch/transport.py", line 327, in perform_request
    status, headers, data = connection.perform_request(method, url, params, body, ignore=ignore, timeout=timeout)
  File "/Users/rafaelcustodio/opt/rask_00k/site-packages/elasticsearch/connection/http_urllib3.py", line 109, in perform_request
    self._raise_error(response.status, raw_data)
  File "/Users/rafaelcustodio/opt/rask_00k/site-packages/elasticsearch/connection/base.py", line 113, in _raise_error
    raise HTTP_EXCEPTIONS.get(status_code, TransportError)(status_code, error_message, additional_info)
RequestError: <unprintable RequestError object> 

PS: In the DOC it's different to delete for old versions https://www.elastic.co/guide/en/elasticsearch/reference/1.6/search-request-scroll.html#_clear_scroll_api

[rafacustodio]

Sorry, you can ignore my comment, I saw that I have to install an old version of the package.

[honzakral]

@rshkv elasticsearcgh-py 2.x doesn't support Elasticsearch version 1.x, it is not expected to work because behavior of scan/scroll has changed between those versions. You need to use the appropriate version - 1.9.0 in this case. You can see the details here - http://elasticsearch-py.readthedocs.io/en/master/#compatibility

@slydon-apixio what elasticsearch version are you using? Thanks!

[slydon-apixio]

Yeah 1.7. Sorry about the confusion on this.

[jaisharma639]

What's the fix for this issue? I'm getting
RequestError: TransportError(400, u'illegal_argument_exception', u'Failed to decode scrollId')
@honzakral ,i'm running ES 2.4

[honzakral]

@jaisharma639 and what version of elasticsearch-py are you running?