[BUG] - Error in JSON Parsing (RequestError 400) when Using OpenSearch

[ibalampanis]

Description:

I encountered an issue when using OpenSearch in my flask (gunicorn) project. It appears to be related to JSON parsing, resulting in a RequestError(400, 'json_parse_exception'). This error occurs when trying to execute a search query. Note that this error is not showing up each time and I am making always the same request.

Error Message:

Exception: RequestError(400, 'json_parse_exception', 'Illegal character ((CTRL-CHAR, code 31)): only regular white space (\\r, \\n, \\t) is allowed between tokens\n at [Source: (org.opensearch.core.common.io.stream.InputStreamStreamInput); line: 1, column: 2]')
Traceback: Traceback (most recent call last):
  File "/home/user/project/data-module/api/routes.py", line 263, in gw_recv_account_details
    response = search_document(os_cli, OS_INDEX_IN, query)
  File "/home/user/project/data-module/api/utils/opensearch.py", line 32, in search_document
    response = os_cli.search(
  File "/home/user/anaconda3/lib/python3.9/site-packages/opensearchpy/client/utils.py", line 179, in _wrapped
    return func(*args, params=params, headers=headers, **kwargs)
  File "/home/user/anaconda3/lib/python3.9/site-packages/opensearchpy/client/__init__.py", line 1553, in search
    return this_transport.perform_request(
  File "/home/user/anaconda3/lib/python3.9/site-packages/opensearchpy/transport.py", line 409, in perform_request
    raise e
  File "/home/user/anaconda3/lib/python3.9/site-packages/opensearchpy/transport.py", line 370, in perform_request
    status, headers_response, data = connection.perform_request(
  File "/home/user/anaconda3/lib/python3.9/site-packages/opensearchpy/connection/http_urllib3.py", line 266, in perform_request
    this_transport._raise_error(
  File "/home/user/anaconda3/lib/python3.9/site-packages/opensearchpy/connection/base.py", line 301, in _raise_error
    raise HTTP_EXCEPTIONS.get(status_code, TransportError)(
opensearchpy.exceptions.RequestError: RequestError(400, 'json_parse_exception', 'Illegal character ((CTRL-CHAR, code 31)): only regular white space (\\r, \\n, \\t) is allowed between tokens\n at [Source: (org.opensearch.core.common.io.stream.InputStreamStreamInput); line: 1, column: 2]')

Python Code:

# Function to search a document from an index.
def search_document(os_cli, index, query):
    response = os_cli.search(
            index=index,
            body=query
        )
    return response

OS_INDEX_IN = RISK_VALUES_IDX

request_id = str(data_type_22['requestId']).encode('utf-8')
query = {
    'query': {
        'match': {'requestId': request_id.decode('utf-8')}
    }
}


response = search_document(os_cli, OS_INDEX_IN, query)

Environment Details:

• Operating System: Ubuntu 18.04
• Python Version: 3.9.13
• OpenSearch Version: 2.11
• opensearch-py Version: 2.4.2

Thank you for your assistance in resolving this issue. Let me know if you need any more information from my end to address this problem.

[dblock]

I believe this is opensearch-project/OpenSearch#10802 and it was fixed in 2.11.1. Can you please try that? Reopen if that didn't fix the problem please.