New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Elastic Search Connector #1665

Open
sorenbs opened this Issue Jan 17, 2018 · 16 comments

Comments

Projects
None yet
@sorenbs
Copy link
Member

sorenbs commented Jan 17, 2018

Prisma supports two kinds of database connectors:

  • Active connectors - manage and migrate the database schema
  • Passive connectors - read and write data only

This is a living document outlining all considerations for an active SQL connector. This is not a final spec - please contribute by commenting below.

Simple query

{
  Match: {
    last_name: "Smith"
  }
}

Combined filter and query

{
  filtered: {
    filter: {
      range: {
        age: { gt: 30 }
      }
    }
    query: {
      match: {
        last_name: "Smith"
      }
    }
  }
}

returns

{
  ...
  "hits": {
    "total": 1,
    "max_score": 0.30685282,
    "hits": [
      {
        ...
        "_source": {
          "first_name": "Jane",
          "last_name": "Smith",
          "age": 32,
          "about": "I like to collect rock albums",
          "interests": [ "music" ]
        }
      }
    ]
  }
}

Questions

  • Should we keep the _ prefixes used by elastic search in many places?
@siddharthlatest

This comment has been minimized.

Copy link

siddharthlatest commented Mar 13, 2018

Love that you are thinking about the Elasticsearch adaptor.

I am not sure how this would map to the full Elasticsearch Query DSL? Even if you narrowly consider match, term and bool as three clauses to support - there are numerous parameters each of them can support. Can you share how an example spec might look like?

Also, if the idea isn't to fully support the DSL (imo really hard to do), what is the fallback mechanism?

@sorenbs

This comment has been minimized.

Copy link
Member Author

sorenbs commented Mar 13, 2018

@siddharthlatest With all connectors we aim to support 90+ % of the underlying functionality. The reason we don't aim for 100% is that adding the last bit of functionality would add significant clutter to the API degrading the developer experience. We will support arbitrary queries defined as a string in the data model as fallback. While this is more tedious, it enables the full power of Elastic while keeping most common functionality easily accessible. We will expand on the spec in the next couple weeks and look forward to your feedback :-)

@siddharthlatest

This comment has been minimized.

Copy link

siddharthlatest commented Mar 13, 2018

@sorenbs Looking forward to the expanded spec. In the meanwhile, is there any example spec (may be for a different DB) that can give an idea on how this might work?

@nimi

This comment has been minimized.

Copy link

nimi commented May 3, 2018

Any updates on the spec and/or progress on this feature?

@thebetterjort

This comment has been minimized.

Copy link

thebetterjort commented Jun 4, 2018

Does anybody have an example?

@dan-kez

This comment has been minimized.

Copy link

dan-kez commented Oct 4, 2018

Hi all! Just wanted to check in and see if this is still on the 2018-Q4 roadmap as we're about to enter Q4

@Disturbing

This comment has been minimized.

Copy link

Disturbing commented Oct 11, 2018

Yes, looking forward to this - is it still on track?

@mrimran

This comment has been minimized.

Copy link

mrimran commented Nov 3, 2018

Looking forward for this feature. Please update on this.

@siddharthlatest

This comment has been minimized.

Copy link

siddharthlatest commented Nov 5, 2018

I am interested in contributing and/or collaborating on this in some way - do we have any initial plans / spec?

@sorenbs

This comment has been minimized.

Copy link
Member Author

sorenbs commented Nov 14, 2018

Thank you all for the interest in this! We have not yet started work on the Elastic connector, and we will not be able to start this year. I will update this thread when we have a clear roadmap.

If you are interested in the general concept on non-sql database connectors you can already check out the Mongo Preview

We would like critical feedback from the community on the proposed API spec above. If anybody would be willing to jump in and create a complete specification, that would also be tremendously helpful. Feel free to open a separate issue and link it here.

@stale

This comment has been minimized.

Copy link

stale bot commented Jan 8, 2019

This issue has been automatically marked as stale because it has not had recent activity. It will be closed in 10 days if no further activity occurs. Thank you for your contributions.

@ali-habibzadeh

This comment has been minimized.

Copy link

ali-habibzadeh commented Jan 11, 2019

Will this still happen?

@dzcpy

This comment has been minimized.

Copy link

dzcpy commented Jan 27, 2019

Any updates?

@Disturbing

This comment has been minimized.

Copy link

Disturbing commented Feb 18, 2019

Anyone want to take this up on a bounty? KintoHub will pay for it :)

@dzcpy

This comment has been minimized.

Copy link

dzcpy commented Feb 20, 2019

@Disturbing I might give a try, how do I contact you?

@Disturbing

This comment has been minimized.

Copy link

Disturbing commented Feb 20, 2019

To refrain from putting my email address up - please join our kintohub slack - reach out to me directly (Same alias as here)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment