Support more NoSQL databases

[btzo]

I don't know if this topic has already been discussed, but I was thinking on the idea of including a NoSQL database option in Caravel.
Whereas currently the interface for connection is made using SQLAlchemy and, as its own name says, it doesn't deal with anything that is not SQL, I think this could be a nice upgrade.

PS: I imagine that this change could bring changes on the current interface as well.

[mistercrunch]

We support Druid which we query through a REST API. There's a common interface to both, though it's non trivial (but certainly doable!) to add support for a new DB backend. We were discussing adding support for ElasticSearch for instance as we have some use cases at Airbnb.

[mistercrunch]

Any specific backend you had in mind?

[btzo]

CouchDB

[hugo-pires]

I would like to suggest MongoDB.

[saitx]

Mongodb would be great !

[hugo-pires]

Acting like a REST API client could be used to connect to several databases

[mistercrunch]

Well the REST APIs to different NOSQL dbs are all very different. I don't think there's a standard there... Different calls to get column metadata, different APIs for queries, ...

[hugo-pires]

Ok. Just a suggestion.

[jgbarah]

I'm thinking about ElasticSearch. Has a nice REST interface, and Caravel could be a nice alternative to Kibana...

I wonder if porting Caravel would be mainly a matter of reimplementing what I see for Druid in views.py, or it would be much more than that?

[jgbarah]

I see that topic of integrating with ElasticSearch is covered in some detail in #241.

[SalehHindi]

@jgbarah, @mistercrunch, any update on this issue? Has there been work done on this that I could see?

[fnzv]

ElasticSearch support +1

[danmaz74]

+1 for ElasticSearch

[eroncastro]

+1

[jerry-sjtu]

+1 for ElasticSearch

[mineca]

+1 for ElasticSearch

[feiquanbifeng]

+1 for ElasticSearch

[liukaitj]

+1 for ElasticSearch

[xrmx]

PSA: Please make use of github reactions instead of the annoying +1 comments, thanks.

[detectivebag]

+1 for ElasticSearch

[QiXuanWang]

Anyone actually working on this?
Is there any standard Python API so user could implement certain interface for specific NoSQL database when officially it's not done yet?

[xrmx]

@QiXuanWang yes, you should write a sqlalchemy dialect to be pluggable with superset

[mistercrunch]

Here's the spec for Python's DBAPI:
https://www.python.org/dev/peps/pep-0249/

[frosiere]

Would it be possible to know the list of NoSql databases currently supported by Superset? Is there a list or additional information somewhere. By the way, also in favour of an Elasticsearch support.

[xycloud]

+1 for elasticsearch

[davidattw]

+1 for elasticsearch...

[lilila]

+1 for bigquery

[xrmx]

People, backends support won't automagically appear by themselves by adding +1. Again: #600 (comment)

[mistercrunch]

I've done some work to clarify the connectors interface, making it easier to add new connectors to the platform.

[mistercrunch]

FYI I started the work to integrate with ElasticSearch. Seems like it's going to be fairly straightforward. Let me know if you want to get involved.
#2953

[mistercrunch]

#8441