Better GraphQL Integration

[maaft]

Hi @krotik!

EliasDB looks very promising to become a one-stop GraphQL-DB solution.

However, I think there are some points that could increase the current experience tremendously:

1. Better GraphQL API

For this, you might want to look at how Dgraph handles their GraphQL API.

The API should hide as much backend/query knowledge from the frontend-user as possible. Currently, when using GraphQL, the frontend developer has to have domain knowledge on eliasdb specific syntax like traversal arguments.

Most clients do not care about such power. They want to do the following:

• Filter by fields
• Order by fields
• Paginate

Thats pretty much all IMHO. More complex operations could of course be still available, but maybe optional.

Example GraphQL query:

query {
  queryDatasets(filter: {owner: {in: ["Alice", "Bob"]}}) { // get all datasets whos owner is either alice or bob
    owner
    name
    images(filter: {extension: {eq: "jpg"} }, first: 1) { // retrieve only the first jpg image of these datasets
      filename
      thumbnail
      metadata(order: {asc: key}) { // order metadata alphabetically by their keys
         key
         value
      }
    }
  }
}

The above query is 100% human readable and the client does not need to care about how the underlying DB traverses the graph.

2. Interface/Union Support

Interfaces and Unions are defined on the GraphQL standard. They are important as they reduce query complexity and boilerplate.

Example Query:

query {
  queryFoldersOrFile {
    name
    ... on File {
       extension
    }
    ... on Folder {
        childs {
            __typename
        }
    }
  }
}

3. Bonus Points: Create Database From GraphQL Schema:

Again, this example comes from how dgraph handle things.

Given a GraphQL type schema, eliasdb could bootstrap its whole CRUD-API:

interface UUID {
   id: String! @id //ids are automatically searchable
}

interface Timestamped {
  createdAt: DateTime! @default(add: "now") //default values for different CRUD Actions
  updatedAt: DateTime! @default(add: "now", update: "now")
}

interface Ownable {
  owner: String!
}

type Dataset implements UUID & Timestamped & Ownable {
   name: String! @index // build indices for search
   images: [Image!]! @hasInverse(field: dataset, cascadeDelete: true) // type image has an inverse edge to Dataset, delete all images when a dataset is deleted
}

type Image implements UUID & Timestamped & Ownable {
   filename: String! @index
   extension: String! @index 
   dataset: Dataset! // inverse edge to dataset type required
}

Using the above Info you could completely bootstrap and layout an EliasDB instance and generate a GraphQL CRUD API as described as in 1.

Why not use Dgraph after all? Good Question. Here are the answers:

• Partially Licensed / not Open-Source
• Discontinued / Development has stalled
• Linux Only
• huuuuge Memory requirements
• Missing a lot of features that will not be developed because its discontinued.

But, Dgraph has made a lot of things right and was very close in becoming the GraphQL-DB no-code solution. So we could borrow a lot of concepts to make EliasDB better in that regard.

Happy to hear your thoughts about this!

[krotik]

Hey there,

Thank you for the suggestions and feedback (I am missing this the most in this project 😄 )

I think you raised some very good and interesting points.

1. Sounds straight forward and I would hope not too much effort.

2. I am not sure I understand completely from the example. I'll need to read again in the GrapQL spec and see.

3. I can see the power of creating a DB schema but implementing a proper type system is a significant effort. If I remember the spec correctly it was seen as optional and I chose at the time not to implement it because it is such a significant effort (though my PhD was actually about a type system 😅 )

All your points make sense to me. I cannot see anything in the moment I would disagree with. Unfortunately, I can't give a timeline for this as it is a significant effort.

[av8ta]

I too would like a graphql design like above. Found it confusing in graphiql even with the docs there to help. Normally the graphiql docs are all you need to learn the api but I agree the underlying database abstraction has leaked too much into graphql. It does seem like a big effort to change but I'd be willing to help.

I wonder if a shortcut could be compiling graphql queries to EQL? Although I haven't yet looked in the code to see how you're doing it. Just want to spark a line of enquiry 😄

[beoran]

Since we are all interested in this, let's actually help out by writing some pull requests to fix this step by step. :)

[av8ta]

Not sure where to start though...

[krotik]

I've looked a bit more into this. Here are my current thoughts.

In general I am careful with hiding knowledge because it means you have to make default assumptions. Basically for every piece of knowledge you hide, you will need to make a one-fits-all choice. The balance is between readability/easy of use and potential performance issues/unexpected behavior. In this case, I can see the point of adding a simple interface for "simple" cases and leave the more complex option for the rest.

So the query in point 1. can already be done currently:

{
    queryDatasets(matches : {owner: "^(Alice|Bob)$"}) {
        owner
        name
        images(traverse:":::images", matches: {extension: "jpg"}, items: 1) {
          filename
          thumbnail
          metadata(traverse:":::metadata", ascending: key) {
             key
             value
          }
        }
    }
}

I would agree that this query can be hard to read so I've added some "syntactical sugar" (in 1.3.0) to make it more palatable for basic scenarios. Semantically the following is equivalent to the query above.

{
    queryDatasets(matches : {owner: [Alice, Bob]}) {
        owner
        name
        images(matches: {extension: "jpg"}, items: 1) {
          filename
          thumbnail
          metadata(ascending: key) {
             key
             value
          }
        }
    }
}

The main difference to the example presented in point 1. is that the operators "in" or "eq" do not exist explicitly. I think the current filtering is powerful enough for the vast majority of cases. If it becomes necessary to do some more advanced calculations or do type specific operations when filtering selection sets then another directive could be added in the future.

Part of the suggestion in point 2 is already implemented in EliasDB - I just failed to document it 😄 - EliasDB supports "Fragments" of the GraphQL standard together with the @skip and @include directives. I've updated the documentation .

Although it is not possible to define interfaces as such, it is possible to use fragments with their type condition matching on node kinds. For example in the tutorial I could write:

{
  Station(ascending:key) {
    name
    StationAndLine(traverse: ":::") {
      ... on Line {
        kind
        name
      }
      ... on Station {
        kind
        name
        zone
      }
    }
  }
}

I can also use variables in combination with a directive to modify a query based on input parameters:

query Stations($expandedInfo: boolean=true){
  Station(ascending:key) {
    key
    name
    ... on Station @include(if: $expandedInfo) {
      zone
    }
  }
}

Interface and Unions are part of the GraphQL Type system which is not implemented in EliasDB. In the moment I am not sure that implementing this would add enough value to EliasDB to justify the significant effort.

BTW re: where stuff is: The GraphQL parser can be found here: https://github.com/krotik/common/tree/master/lang/graphql/parser
The interpreter can be found here: https://github.com/krotik/eliasdb/tree/master/graphql/interpreter