Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Product Roadmap #1

Closed
56 of 65 tasks
manishrjain opened this issue Nov 30, 2015 · 70 comments
Closed
56 of 65 tasks

Product Roadmap #1

manishrjain opened this issue Nov 30, 2015 · 70 comments
Labels

Comments

@manishrjain
Copy link
Contributor

manishrjain commented Nov 30, 2015

  • Low Latency
  • High Throughput
  • RDF Parsing
  • Rocks DB Badger DB for persistence
  • Commit Logs (Replaced by RAFT logs)
  • Query Language - GraphQL-like
    • Query
    • Root Arguments
    • Fields
    • Response in JSON
    • Field Arguments
    • Field Alias
    • Mutations Mutations via GraphQL #23 [v0.2]
    • Fragments Support GraphQL Fragments #8
    • Variables
    • Type System
      • Scalar Types
      • Object Types
    • Mutation validation for scalar types
    • String matching / Name search
    • Sort by attribute
    • Limit number of results Limit number of results #9
    • Filter
      • anyof
      • allof
      • eq (equal)
      • inequality (>=, <=, >, <)
    • Aggregate Functions
      • count
      • sum
      • max
      • min
    • Geospatial Queries
      • Nearby
      • Within
      • Contains
      • Intersects
  • Official Clients
    • Javascript
    • Go [v0.3]
    • Java
  • Distributed Transactions
  • Distributed Make DGraph distributed #14 [v0.2]
    • Distributed Loader [v0.2]
    • Distributed Server [v0.2]
  • Clustering
    • Node discovery and membership via Dgraph Zero
  • High Availability
    • Raft
    • Automatic Data Replication
    • Automatic Failover for reads
    • Read linearizability
  • Resilience
    • Shard moves to handle server failure
  • Export
  • Backup

After v1.0 / Proprietary Plugins

  • Multi-homing support
  • Cypher Support
  • Access Control Lists
  • Query Graphical User Interface
  • User authentication
  • Cluster Management
  • SPARQL [maybe]
  • Tinkerpop Support [maybe]
  • Distributed transactions [maybe]
@KyleAMathews
Copy link

It'd be cool if the DB could serve GraphiQL. This would be perfect for demos + internal spelunking through data.

@manishrjain
Copy link
Contributor Author

I'm running a DGraph server holding Freebase Film data, which you can query like so: curl dgraph.xyz/query -XPOST -d '{}'. An example of the query is in the README.

@manishrjain manishrjain modified the milestone: v.future Feb 17, 2016
@PhilAndrew
Copy link

Support tinkerpop? https://tinkerpop.apache.org

@manishrjain
Copy link
Contributor Author

Tinkerpop would happen with Gremlin support. We'll do that after v1.0 gets released. This roadmap is largely focused on v1.0.

@manishrjain manishrjain removed this from the v.future milestone Jun 3, 2016
@F21
Copy link
Contributor

F21 commented Oct 19, 2016

For gossip discovery, I've found hashicorp/memberlist to work quite well.

@manishrjain
Copy link
Contributor Author

@F21: Instead of using Gossip protocol, we used the already implemented RAFT protocol for keeping track of membership. In terms of discovery, our cluster requires knowing at least one node in the cluster; any healthy node would do.

@julsdelatierra
Copy link

I suggest also support cypher for queries, to avoid the queries migration from neo4j users.

@manishrjain
Copy link
Contributor Author

@jcebXD -- Yeah, Cypher is something we'll fully support.

@pmualaba
Copy link

Cypher support will be great indeed for the application layer. I also see that Tinkerpop support is on the Roadmap after 1.0, but I would have prefered to see SPARQL instead. Since the data model for Dgraph is already Triple/Nquad based, it would be logic and convenient to expose a SPARQL 1.1 endpoint out of the box, in addition to the GraphQL endpoint. SPARQL support is very important to fully interact with the semantic web, without the need to take a full Dgraph (triples) backup which then again needs to be imported in a separate TripleStore such as BlazeGraph to be able to run SPARQL queries against the Dgraph data.

@ghost
Copy link

ghost commented Dec 30, 2016

Is there any interest in supporting mobile ?
Running a dgrpah DB on mobiles using gomobile is possible. Even with the CGO aspect of rocksdb.
I currently run boltDB on the servers and the clients with golang. It is nice to have that reuse.

For example i build apps for mobile, desktop using golang with this: https://github.com/therecipe/qt
It uses CGO heavily, and works well.

@manishrjain
Copy link
Contributor Author

@gedw99 -- We're not currently thinking about mobile. All our efforts are focused on production for large scale deployments.

@pmualaba
Copy link

Aggregations are not on the roadmap for 1.0, and neither after 1.0 ?
The ability to perform aggregations (COUNT, SUM, AVG, MIN, MAX, GROUP BY, DISTINCT,...) is clearly one of the main reasons you store data in a database in the first place and all acient and modern databases and database languages (SQL, CYPHER, SPARQL,...) support it. Why are they missing from the roadmap for DGraph?

Also for a new product anno 2017 it also make sense to have built in support for subscriptions (changefeeds ala RethinkDB). since many of the applications we built today needs support for real-time and push features...

DGraph is a great and promising database project, but without these features one would still be forced to move to other open-source alternatives such as ArangoDB which at this moment is the only serious open-source "competitor" for DGraph.

@ashwin95r
Copy link
Contributor

@pmualaba: We're working on aggregate functions already! #532 It was missing on the roadmap. Thanks for pointing it out. We've added it now.

@pmualaba
Copy link

You guys are lightning fast ;-) Great to hear that you're already working on this.

@pmualaba
Copy link

pmualaba commented Feb 20, 2017

While playing with DGraph, there is one thing that doesn't feel quite right to me :
DGraph returns JSON format (which is great!), GraphQL queries also uses JSON format (without values) which is also great! But GraphQL mutations only supports NQuad format?
I would like to see mutations also to be consistent in using JSON. As a matter of fact, there is already a W3C compliant RDF serialization format called JSON-LD: http://json-ld.org/

Accepting valid JSON-LD in mutations will also greatly improve developer experience using DGraph, since application logic these days all generate or serialize into JSON before persisting to a database. Manual JSON to NQuad conversion can then be eliminated from the application layer and moved to the database layer.

I would not suggest to remove support for NQuad mutations enterily (sometimes it can come in handy), But to add additional support for JSON-LD. I believe NQuad is still the way to go for massive batch imports, but JSON-LD is preferable for use in mutation queries, i guess.

@manishrjain
Copy link
Contributor Author

Thanks for the feedback, @pmualaba. JSON-LD seems like a nice format for mutations. We have @ashishnegi working on it.

@joeblew99
Copy link

@manishrjain Are the proprietry plugins going to be closed source.
Sorry to aks this, but ACL's for example not being part of the open source offering ? Its pretty harsh.

@manishrjain
Copy link
Contributor Author

manishrjain commented Mar 14, 2017

Hey @joeblew99 , others,

It is very important for us to figure out a revenue model, if we want to continue working on Dgraph three years down the lane. That's the only way we can keep supporting the salaries of people who are dedicated their lives on Dgraph. Without money, Dgraph would stop spinning (think RethinkDB).

Keeping that in mind, we have divided the project into what we consider open core and what we consider commercial. Anything that a startup needs to run their services and scale is part of the open core. This includes all of the current functionality and running Dgraph distributedly (which is not for e.g., allowed by Neo4j).

Anything that's an advanced feature would then be put into the commercial, closed-source version of Dgraph. This means: ACLs, cluster management, multi-tenancy, auto-scaling on Amazon, Google, etc.

This is a delicate balance between giving startups the same (I know it's better but I want to stay humble) level of technology built within the walls of big companies, like Google, Facebook, etc.; while also finding a way to monetization to make the company profitable and keep hiring the top engineers we can find.

Note that for small companies, which are pre-revenue, we would most likely make the enterprise version free of cost (free, but not open). But, if a company is making revenue and benefitting from Dgraph, we do expect them to pay. That's what is going to allow us to keep our engineers working for Dgraph, in a very competitive market where giants can pay up a huge sum to entice engineers. So, their salaries have to be competitive and they must feel that this company is not going to vaporize, and is going to be around 10 years from now.

@foobatman
Copy link

foobatman commented Mar 14, 2017 via email

@manishrjain
Copy link
Contributor Author

manishrjain commented Mar 14, 2017

One thing that's clear to me: Dgraph can only survive if we are 10x better than Neo4j and others. If we genuinely didn't believe that, we'd be instead working for some big company, 9-5 and getting a nice fat check at the end of the year. So, making Dgraph the best graph database out there is the top priority. And for that, we need to hire and retain the best engineers.

I think Firefox is a bad example. They were thriving when Google was paying (donating?) them a big sum of money; until Google decided to stop that and focus on Chrome. Since then, Firefox revenue has steadily declined, while Chrome has taken over the world. At this point, Firefox can't afford to get the top talent working for them -- while Google (/Chrome) continues to recruit the best and brightest, which in turn makes Chrome even better.
http://howdoesitmakemoney.com/how-does-mozilla-firefox-make-money/

Consultancy is great, and we intend to do that, but consultancy alone is not going to make a company significantly profitable (more consultancy requires more human power). All top companies today are profitable due to selling software (Google, Microsoft, Facebook, etc.).

P.S. I don't intend this to become a thread of its own; so feel free to email me if anyone has concerns: manish, dgraph.io.

@joeblew99
Copy link

joeblew99 commented Mar 14, 2017

@manishrjain i agree with you.
Many startups that were making new database systems failed due to lack of revenue. We dont want another rethinkdb, and the open core model makes sense.

But, everyone needs ACLS. I would think about if ACLS are in the open core of not.
The other stuff is for commercial for sure - so i agree with that.

But ACLS is so fundamental to any system you run always.
I mean any open source project that wants to build a forum or a chat system needs ACLS.

@ghost
Copy link

ghost commented Apr 8, 2017

What are your thoughts on subscriptions? Do you plan on supporting libraries for languages other than Go (and Python and Java)?

@manishrjain
Copy link
Contributor Author

We have community run libraries for Python and Java. You can find their repos in dgraph-io org in Github.

Not sure what you mean about subscriptions.

@MichelDiz
Copy link
Contributor

@gedw99 The backup feature still not exist. So, for that there's no restore. But you have "Export" and ou can re-import this exported file into a new instance. See more details here: https://docs.dgraph.io/deploy#export-database

take a look on:
https://docs.dgraph.io/deploy#bulk-loader
https://docs.dgraph.io/deploy#live-loader

@ghost
Copy link

ghost commented May 8, 2018 via email

@manishrjain
Copy link
Contributor Author

All Dgraph code base is open. We currently don't have any enterprise features.

@lynic
Copy link

lynic commented Jul 12, 2018

I really like this project, but there're two missing features block me from using it at our development environment, one is user authentication, the other is Tinkerpop/gremlin support. If you could prioritize these two features, that would be very helpful!

@manishrjain
Copy link
Contributor Author

Gremlin is on the roadmap. We have ideas around user auth, but for now, you could use the password type to store user passwords and use that to authenticate them.

@yiouyou
Copy link

yiouyou commented Jul 13, 2018

I'm learning Dgraph. A smarter "Query Graphical User Interface" would be very helpful when exploring the graph database, like Grakn.ai has. And moreover, if there is a "How to do reasoning" section in Help Doc, that would be wonderful. One more question: does dgraph support hypergraphs? Thanks!

@pierreis
Copy link

Two questions:

  1. Multi-homing support: Does this relate to datacenter / rack awareness for geo-distributed setups?
  2. Any update on a hosted Dgraph service? Q1 2018 still is the period I'm hearing about.

@manishrjain
Copy link
Contributor Author

Thanks for all your comments! It gives me a lot of joy to see how far Dgraph has progressed since this first issue on Github. Think it's time to close this issue. We're still listening to what our users want, so go on and tell us what you'd like to see built in Dgraph in the new 2019 roadmap here: #2894 .

Superseded by #1966 and #2894 .

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Development

No branches or pull requests