Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Elasticsearch connection got problem when click test connection #8923

Closed
mikolaje opened this issue Jan 4, 2020 · 11 comments
Closed

Elasticsearch connection got problem when click test connection #8923

mikolaje opened this issue Jan 4, 2020 · 11 comments
Labels

Comments

@mikolaje
Copy link

@mikolaje mikolaje commented Jan 4, 2020

A clear and concise description of what the bug is.

Expected results

what you expected to happen.
elasticsearch+http://elastic:changeme@device1:9200
I added the above url as the ES connection

Actual results

But it failed with an Error:

ERROR: {"error": "Connection failed!\n\nThe error message returned was:\ncan only concatenate str (not \"NoneType\") to str", "stacktrace": "Traceback (most recent call last):\n  File \"/data/software/miniconda3/lib/python3.7/site-packages/superset/views/core.py\", line 1704, in testconn\n    with closing(engine.connect()) as conn:\n  File \"/data/software/miniconda3/lib/python3.7/site-packages/sqlalchemy/engine/base.py\", line 2206, in connect\n    return self._connection_cls(self, **kwargs)\n  File \"/data/software/miniconda3/lib/python3.7/site-packages/sqlalchemy/engine/base.py\", line 103, in __init__\n    else engine.raw_connection()\n  File \"/data/software/miniconda3/lib/python3.7/site-packages/sqlalchemy/engine/base.py\", line 2306, in raw_connection\n    self.pool.unique_connection, _connection\n  File \"/data/software/miniconda3/lib/python3.7/site-packages/sqlalchemy/engine/base.py\", line 2275, in _wrap_pool_connect\n    return fn()\n  File \"/data/software/miniconda3/lib/python3.7/site-packages/sqlalchemy/pool/base.py\", line 303, in unique_connection\n    return _ConnectionFairy._checkout(self)\n  File \"/data/software/miniconda3/lib/python3.7/site-packages/sqlalchemy/pool/base.py\", line 760, in _checkout\n    fairy = _ConnectionRecord.checkout(pool)\n  File \"/data/software/miniconda3/lib/python3.7/site-packages/sqlalchemy/pool/base.py\", line 492, in checkout\n    rec = pool._do_get()\n  File \"/data/software/miniconda3/lib/python3.7/site-packages/sqlalchemy/pool/impl.py\", line 238, in _do_get\n    return self._create_connection()\n  File \"/data/software/miniconda3/lib/python3.7/site-packages/sqlalchemy/pool/base.py\", line 308, in _create_connection\n    return _ConnectionRecord(self)\n  File \"/data/software/miniconda3/lib/python3.7/site-packages/sqlalchemy/...

This is the log:

TypeError: can only concatenate str (not "NoneType") to str
2020-01-04 18:44:27,311:INFO:werkzeug:127.0.0.1 - - [04/Jan/2020 18:44:27] "POST /superset/testconn HTTP/1.1" 500 -
2020-01-04 18:48:03,388:INFO:root:Database.get_sqla_engine(). Masked URL: elasticsearch+http://elastic:XXXXXXXXXX@device1:9200
2020-01-04 18:48:03,389:ERROR:root:can only concatenate str (not "NoneType") to str
Traceback (most recent call last):
  File "/data/software/miniconda3/lib/python3.7/site-packages/superset/views/core.py", line 1704, in testconn
    with closing(engine.connect()) as conn:
  File "/data/software/miniconda3/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 2206, in connect
    return self._connection_cls(self, **kwargs)
  File "/data/software/miniconda3/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 103, in __init__
    else engine.raw_connection()
  File "/data/software/miniconda3/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 2306, in raw_connection
    self.pool.unique_connection, _connection
  File "/data/software/miniconda3/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 2275, in _wrap_pool_connect
    return fn()
  File "/data/software/miniconda3/lib/python3.7/site-packages/sqlalchemy/pool/base.py", line 303, in unique_connection
    return _ConnectionFairy._checkout(self)
  File "/data/software/miniconda3/lib/python3.7/site-packages/sqlalchemy/pool/base.py", line 760, in _checkout
    fairy = _ConnectionRecord.checkout(pool)
  File "/data/software/miniconda3/lib/python3.7/site-packages/sqlalchemy/pool/base.py", line 492, in checkout
    rec = pool._do_get()
  File "/data/software/miniconda3/lib/python3.7/site-packages/sqlalchemy/pool/impl.py", line 238, in _do_get
    return self._create_connection()
  File "/data/software/miniconda3/lib/python3.7/site-packages/sqlalchemy/pool/base.py", line 308, in _create_connection
    return _ConnectionRecord(self)
  File "/data/software/miniconda3/lib/python3.7/site-packages/sqlalchemy/pool/base.py", line 437, in __init__
    self.__connect(first_connect_check=True)
  File "/data/software/miniconda3/lib/python3.7/site-packages/sqlalchemy/pool/base.py", line 639, in __connect
    connection = pool._invoke_creator(self)
  File "/data/software/miniconda3/lib/python3.7/site-packages/sqlalchemy/engine/strategies.py", line 114, in connect
    return dialect.connect(*cargs, **cparams)
  File "/data/software/miniconda3/lib/python3.7/site-packages/sqlalchemy/engine/default.py", line 453, in connect
    return self.dbapi.connect(*cargs, **cparams)
  File "/data/software/miniconda3/lib/python3.7/site-packages/es/elastic/api.py", line 32, in connect
    return Connection(host, port, path, scheme, user, password, context, **kwargs)
  File "/data/software/miniconda3/lib/python3.7/site-packages/es/elastic/api.py", line 109, in __init__
    **kwargs,
  File "/data/software/miniconda3/lib/python3.7/site-packages/es/baseapi.py", line 57, in __init__
    self.url = parse.urlunparse((scheme, netloc, path, None, None, None))
  File "/data/software/miniconda3/lib/python3.7/urllib/parse.py", line 475, in urlunparse
    return _coerce_result(urlunsplit((scheme, netloc, url, query, fragment)))
  File "/data/software/miniconda3/lib/python3.7/urllib/parse.py", line 487, in urlunsplit
    url = '//' + (netloc or '') + url
TypeError: can only concatenate str (not "NoneType") to str

what actually happens.

Screenshots

If applicable, add screenshots to help explain your problem.

How to reproduce the bug

  1. Go to '...'
  2. Click on '....'
  3. Scroll down to '....'
  4. See error

Environment

(please complete the following information):

  • superset version: superset version
    Superset 0.35.1
  • python version: python --version
    Python 3.7.3
  • node.js version: node -v
  • npm version: npm -v

Checklist

Make sure these boxes are checked before submitting your issue - thank you!

  • I have checked the superset logs for python stacktraces and included it here as text if there are any.
  • I have reproduced the issue with at least the latest released version of superset.
  • I have checked the issue tracker for the same issue and I haven't found one similar.

Additional context

I already installed pip install elasticsearch-dbapi

Add any other context about the problem here.

@issue-label-bot

This comment has been minimized.

Copy link

@issue-label-bot issue-label-bot bot commented Jan 4, 2020

Issue-Label Bot is automatically applying the label #bug to this issue, with a confidence of 0.98. Please mark this comment with 👍 or 👎 to give our bot feedback!

Links: app homepage, dashboard and code for this bot.

@issue-label-bot issue-label-bot bot added the #bug label Jan 4, 2020
@mikolaje mikolaje closed this Jan 4, 2020
@mikolaje mikolaje reopened this Jan 4, 2020
@MarcusSorealheis

This comment has been minimized.

Copy link
Contributor

@MarcusSorealheis MarcusSorealheis commented Jan 4, 2020

You would have to check with the maintainer of the sqlalchemy library for elasticsearch, but I would be surprised if this was tested using miniconda/anaconda on python 3.7. I personally don't know the basics of it. Can you post the environment versions as described in the issue template?

@mikolaje

This comment has been minimized.

Copy link
Author

@mikolaje mikolaje commented Jan 4, 2020

@MarcusSorealheis sure, I updated. I guess it is a bug from elasticsearch-dbapi?

@villebro

This comment has been minimized.

Copy link
Contributor

@villebro villebro commented Jan 4, 2020

@dpgaspar might be able to help

@dpgaspar

This comment has been minimized.

Copy link
Contributor

@dpgaspar dpgaspar commented Jan 4, 2020

Please try with the following connection string instead:

elasticsearch+http://elastic:changeme@device1:9200/

Notice the trailing slash, I'll improve this on elasticsearch-dbapi

@mikolaje

This comment has been minimized.

Copy link
Author

@mikolaje mikolaje commented Jan 5, 2020

@dpgaspar Thanks for your support.

but I have another issue, I'm using elasticsearch 5, so I installed the package with pip install elasticsearch==5.5.3, after that , superset raised:

pkg_resources.VersionConflict: (elasticsearch 5.5.3 (/data/software/miniconda3/lib/python3.7/site-packages), Requirement.parse('elasticsearch>7'))

do I have to run with the elasticsearch 7 package?

@dpgaspar

This comment has been minimized.

Copy link
Contributor

@dpgaspar dpgaspar commented Jan 5, 2020

Yes, the library is tested against Elasticsearch 7.X. Should work with Elasticsearch > 6.3

@dpgaspar dpgaspar added #question and removed #bug labels Jan 5, 2020
@mikolaje

This comment has been minimized.

Copy link
Author

@mikolaje mikolaje commented Jan 6, 2020

@dpgaspar So, it is not compatible with the elasticsearch 5?
Once I use the elasticsearch package > 7, It raises:

Traceback (most recent call last):
  File "/data/software/miniconda3/lib/python3.7/site-packages/superset/views/core.py", line 1705, in testconn
    conn.scalar(select([1]))
  File "/data/software/miniconda3/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 920, in scalar
    return self.execute(object_, *multiparams, **params).scalar()
  File "/data/software/miniconda3/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 988, in execute
    return meth(self, multiparams, params)
  File "/data/software/miniconda3/lib/python3.7/site-packages/sqlalchemy/sql/elements.py", line 287, in _execute_on_connection
    return connection._execute_clauseelement(self, multiparams, params)
  File "/data/software/miniconda3/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 1107, in _execute_clauseelement
    distilled_params,
  File "/data/software/miniconda3/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 1253, in _execute_context
    e, statement, parameters, cursor, context
  File "/data/software/miniconda3/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 1475, in _handle_dbapi_exception
    util.reraise(*exc_info)
  File "/data/software/miniconda3/lib/python3.7/site-packages/sqlalchemy/util/compat.py", line 153, in reraise
    raise value
  File "/data/software/miniconda3/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 1249, in _execute_context
    cursor, statement, parameters, context
  File "/data/software/miniconda3/lib/python3.7/site-packages/sqlalchemy/engine/default.py", line 552, in do_execute
    cursor.execute(statement, parameters)
  File "/data/software/miniconda3/lib/python3.7/site-packages/es/baseapi.py", line 25, in g
    return f(self, *args, **kwargs)
  File "/data/software/miniconda3/lib/python3.7/site-packages/es/elastic/api.py", line 139, in execute
    results = self.elastic_query(query)
  File "/data/software/miniconda3/lib/python3.7/site-packages/es/baseapi.py", line 220, in elastic_query
    resp = self.es.transport.perform_request("POST", path, body=payload)
  File "/data/software/miniconda3/lib/python3.7/site-packages/elasticsearch/transport.py", line 350, in perform_request
    timeout=timeout,
  File "/data/software/miniconda3/lib/python3.7/site-packages/elasticsearch/connection/http_urllib3.py", line 257, in perform_request
    self._raise_error(response.status, raw_data)
  File "/data/software/miniconda3/lib/python3.7/site-packages/elasticsearch/connection/base.py", line 182, in _raise_error
    status_code, error_message, additional_info
elasticsearch.exceptions.TransportError: TransportError(500, 'null_pointer_exception', None)

my server is running Elasticsearch 5.5.

@dpgaspar

This comment has been minimized.

Copy link
Contributor

@dpgaspar dpgaspar commented Jan 6, 2020

You really need elasticsearch (the backend itself) > 6.3 because the dbapi uses X-Pack SQL

@mikolaje

This comment has been minimized.

Copy link
Author

@mikolaje mikolaje commented Jan 7, 2020

@dpgaspar alright, thanks :)

@dpgaspar

This comment has been minimized.

Copy link
Contributor

@dpgaspar dpgaspar commented Jan 7, 2020

no problem

@dpgaspar dpgaspar closed this Jan 7, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
4 participants
You can’t perform that action at this time.