Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Product Roadmap 2020 #4724

Closed
4 of 24 tasks
manishrjain opened this issue Feb 3, 2020 · 62 comments
Closed
4 of 24 tasks

Product Roadmap 2020 #4724

manishrjain opened this issue Feb 3, 2020 · 62 comments
Labels

Comments

@manishrjain
Copy link
Contributor

manishrjain commented Feb 3, 2020

Here's the product roadmap for 2020.

  • Official GraphQL spec compliance
    • Queries (Q1)
    • Mutations (Q1)
    • Subscriptions (Q2)
    • Live Queries (Q2)
  • SaaS (Q2)
  • Scalability, performance, and reliability
    • Single predicate sharded across groups (TBD)
    • Distributed bulk loader (Q3)
    • Rolling upgrades (Q4)
    • Ludicrous mode (Q1)
    • Query Planner (TBD)
  • Enterprise features
    • Change Data Capture (Q3)
    • Multi-tenancy (Q3)
    • Point-in-time recovery (TBD)
    • Audit logs (Q3)
    • ACL integration with AD, LDAP (stretch goal)
  • Integrations
    • Kubernetes
      • Operator (TBD)
      • Bulk Loader (TBD)
    • Kafka (TBD)
    • Others. Let us know in comments.

We have mentioned the features we are planning to focus on in Q1 and Q2 (first half of 2020). For the rest, we'll assess them for their ETA as reach mid-year. Tell us what more you'd like to see happen in 2020!

@manishrjain
Copy link
Contributor Author

manishrjain commented Feb 3, 2020

I know a bunch of folks have been looking for Gremlin support. We're currently focused on GraphQL, but if you need Gremlin to work with Dgraph, show your support by upvoting this comment and we'll consider prioritizing it in a few months.

@smkhalsa
Copy link

smkhalsa commented Feb 3, 2020

@manishrjain Thanks for the update and for the wonderful work you and the dgraph team are doing.

I'm glad to see that a fully managed SaaS option is on the roadmap.

One question: I don't see any specific mention of exposing full GraphQL+- functionality in spec-compliant GraphQL queries (something like what neo4j-graphql-js offers). There was an indication from the dgraph team shortly after the graphql api was launched that this was coming soon. If that's the case, can you give a preview of how that might work?

@manishrjain
Copy link
Contributor Author

manishrjain commented Feb 3, 2020

exposing full GraphQL+- functionality in spec-compliant GraphQL queries

We're considering multiple ways of doing this:

  1. Figuring out ways in which we can port over +- functionality in GraphQL spec compliant way, so a GraphQL user can just construct these queries directly (fuzzy matching, full-text search, has function).

  2. Adding automatically generated GraphQL functions for the complex +- functionality (like, say email: string @index(exact) @upsert could have upsertEmail func automatically generated).

  3. If a feature can't be translated into GraphQL, then having a way for a user to specify a function name along with the +- query it maps to. This is the closest to the resolver pattern exposed via Apollo. This would be done at schema level, instead of involving any particular programming language (Apollo's resolvers are typically javascript based) for maximizing compatibility among various languages.

For now, we're going with easy wins, that is 1 and 2. Once we have covered and exhausted ways to port +- into 1 or 2, we can look into 3.

@vikram-ma
Copy link

vikram-ma commented Feb 5, 2020

@manishrjain
Where can I find more details to understand what these goals really mean?

  • For example, what does "Ludicrous mode (TBD)" do? and what's the plan to achieve this is.
  • Also can you elaborate Query Planner (TBD)?
    Do you guys have any checks in place to control GraphQL (and graphql+-) query complexity? To prevent friendly fire scenario? Will this be part of Query Planner?

@Relaxe111
Copy link

Relaxe111 commented Feb 6, 2020

Hello, it looks like multi-tenancy support is added to Enterprise feature. I would kindly ask to reconsider this feature to make it open source or to make a more flexible pricing plan. Locking this feature in Enterprise version could be a big barrier in widely adoptoption of dgraph, at least in the EU it will be for sure.

@manishrjain
Copy link
Contributor Author

manishrjain commented Feb 6, 2020

All the enterprise features (including multi-tenancy) would be automatically included in the SaaS offering. That should allow for flexible, pay as you go sort of pricing.

Ludicrous mode -- Idea is to allow a mode of Dgraph, which gives up on some "correctness" things to achieve maximum performance. For a lot of people, if Dgraph doesn't give them the needed speed, they revert to a NoSQL database, which provide you very little in terms of consistency / transactional guarantees. This mode would allow Dgraph to run with lower guarantees, but at a faster speed.

Query Planner: Dgraph doesn't do much query planning right now. It executes the queries in the same order they're given. Of course, we could do a better job by having an internal query planner which can alter the ordering of tasks to achieve better performance.

Do you guys have any checks in place to control GraphQL (and graphql+-) query complexity

Not sure what you mean.

@vikram-ma
Copy link

vikram-ma commented Feb 6, 2020

Clients can do arbitrarily complex GraphQL queries.i.e clients control it the query complexity.
What it means is that clients can issue very very complex queries i.e queries could take minutes/hours to complete, and impact dgraph's ability to serve other requests.

A valid client is making very complex query, that significantly slows down or impacts dgraph's ability to serve other client requests.

Are there checks in place to detect/prevent/handle this?

@willem520
Copy link

willem520 commented Feb 6, 2020

Hello, it looks like multi-tenancy support is added to Enterprise feature. I would kindly ask to reconsider this feature to make it open source or to make a more flexible pricing plan. Locking this feature in Enterprise version could be a big barrier in widely adoptoption of dgraph, at least in the EU it will be for sure.

hi, multi-tenancy is a popular feature, many developer like us need this feature to improve our project. Personally, I hope it could be free. thx

@manishrjain
Copy link
Contributor Author

manishrjain commented Feb 6, 2020

Are there checks in place to detect/prevent/handle this?

A client can specify a context with timeout, which can shut the query down once if it runs too long. But, apart from that, nothing avoids that right now. Once we have a query planner and can calculate the cost of running a query, we can do a better job of rejecting expensive queries.

@nmabhinandan
Copy link

nmabhinandan commented Feb 6, 2020

  • A startup program offering evaluation license of enterprise edition. (or better yet, SaaS credits).
  • A reactive Spring Data client (Along the lines of SDN/RX)
  • GeoJson support

@manishrjain
Copy link
Contributor Author

manishrjain commented Feb 6, 2020

A startup program offering evaluation license of enterprise edition. (or better yet, SaaS credits).

Already there. Every Dgraph instance comes with a month of free enterprise trial.

GeoJson support

Already there. Dgraph has been supporting geo queries since the early days. In fact, some users have said that Dgraph's geo support is better than PostGis (we haven't verified).

@Relaxe111
Copy link

Relaxe111 commented Feb 6, 2020

Saas including multi-tenancy I think is not what majority of developers will really searching for. I mean by more flexible pricing plan to make gradual licencing. For example a company could choose and pay only for Enterprise feature in which is interested, if I need only one feature I would not be happy to pay full license but using only one feature.

@ghost
Copy link

ghost commented Feb 7, 2020

My company is in the position of exploring different graph DBs to modernize our stack, and Dgraph is the best fit in every category (especially looking at this 2020 roadmap), except for lack of Gremlin support (we would like to port our existing queries, and eventually switch). Excited to see how this roadmap progresses!

@manishrjain
Copy link
Contributor Author

manishrjain commented Feb 7, 2020

Generally asking the community here, for Gremlin support, I'm curious if that's really a deal breaker. The port of queries from Gremlin to GraphQL is probably a one-time effort -- and you get the benefit of new, easy to use tech, JSON support, with a growing ecosystem of tools and editors to support creating queries, exploring data, etc. (GraphQL has so many editors).

@ghost
Copy link

ghost commented Feb 7, 2020

Oh porting is definitely something we would be opening to do. Its that we are moving under time pressures and we know that what we have works, and that a few of our members are experienced with gremlin. Not to mention we can switch backends as needed with the support. So that is very specific to us, but it would help teams who are exploring dgraph decide early on if it is a good fit for them by allowing them to use what they have.

@brianbroderick
Copy link

brianbroderick commented Feb 9, 2020

I mentioned this in #2693 but wanted to make sure people see this.

First, I want to say that I love Dgraph and am an evangelist for your product. I've given several talks and am constantly trying to get people interested in Dgraph.

There's one last hurtle to start getting actual adoption, and that's the ability to have multiple schemas in a dev & test environment.

Most companies in my area use MySql, Maria, or Postgres. Therefore having the ability to have many schemas is something people take for granted and is something people are not willing to pay for.

It's a challenge to get people to switch from something they are comfortable with. Making this as painless as possible is the only way to get widespread adoption.

There are many reasons to have multiple schemas; for example, it's typical to have a dev, test, and prod environment with their respective schemas. This makes it so the test database can be recreated before each test run. Right now, the only way to accomplish this is to either have multiple instances of Dgraph running, or to add a prefix to all predicates.

If I only want to clear test environment predicates, adding a prefix complicates queries like this: &api.Operation{DropAll: true}, which I run before any tests. It would also complicate Go structs when determining the right predicate values in JSON.

It's also typical to work on many micro services at a time, but these micro services should not have any chance of data colliding with each other; they should be completely isolated. It doesn't seem realistic to have 10+ instances of DGraph running at the same time on a laptop (5+ micro services, each with a dev and test environment)

Therefore, I want to second everyone's comments about supporting multiple schemas in the free version. The Saas offering isn't going to help my dev and test environments on my laptop.

@Relaxe111
Copy link

Relaxe111 commented Feb 9, 2020

This is absolutely true! That's why I kindly asked to reconsider multi-tenancy to be open-source! I try for last year to introduce dgraph in my company but the biggest barrier to convincing my boss is support for multi-tenancy. Last week we had a new discussion and again everyone in my company is skeptical against dgraph because of lack of multi-tenancy in the community version. I can confirm that for multi-tenancy my company isn't willing to pay. Having multiple instances of DB for different Environments dev/ test is an issue in dgraph. I just hope that owners of such amazing product will understand that multi-tenancy will not be an argument for companies to buy a license to use dgraph enterprise. But having multi-tenancy open-source will be an argument to adopt it. It is more likely to switch to Enterprise for a company that uses already dgraph than to switch from well known traditional DBS to dgraph enterprise or dgraph open-source without multi-tenancy. I truly believe that bigger adoption of dgraph will be, bigger Enterprise mass dgraph will have. But without open-source multi-tenancy, the majority of potential future Enterprise users will ignore dgraph to adopt it now as open source.

@marvin-hansen
Copy link

marvin-hansen commented Feb 10, 2020

My company isn't going to pay for multi-schema / multi-tenancy because any other OSS DB brings it already to the table. Charging for something you get in most OSS DB's for free and that is legally mandated in certain industries or countries is just completely ridiculous. Please start listening to your customers!

Please add GPU acceleration or support for in-DB machine learning to bring at least some tangible value to the enterprise version that would justify a purchase.

#4678

#4608

@bronzels
Copy link

bronzels commented Feb 11, 2020

Hope Gremlin support in Q1 pls.

@jorroll
Copy link

jorroll commented Feb 13, 2020

Generally asking the community here, for Gremlin support, I'm curious if that's really a deal breaker. The port of queries from Gremlin to GraphQL is probably a one-time effort -- and you get the benefit of new, easy to use tech, JSON support, with a growing ecosystem of tools and editors to support creating queries, exploring data, etc. (GraphQL has so many editors).

@manishrjain I imagine most people currently investigating dgraph are people with existing graph db needs and, historically, many existing graph dbs use gremlin. Personally, having used Cypher, Gremlin, and SQL (GraphQL too, though I've never seen it used for directly querying a db), as well as some proprietary APIs like Firestore, I'd say that Gremlin is by far the worst (and one of the reasons why graph dbs are a nich product). I can appreciate someone asking for support because porting an app over to a new language can be a huge undertaking, but, long term, I really hope Gremlin dies in favor of other languages (e.g. upcoming GQL standard). Providing tooling to help port existing Gremlin apps to a newer query language might be a compromise.

I'm speculating, but I think one challenge for dgraph could be that, historically, graph database usage is mostly confined to backend engineers. It seems likely that most of the current dgraph users are backend folks as well. This would contrast with GraphQL which is mostly a frontend query language (tho obviously it can be used server side as well). From my perspective, one of the most exciting aspects of dgraph is the idea that maybe in the future, I can use Apollo Client to query the backend directly from the frontend, eliminating a huge chunk of work in building out an API server (similar to what Firestore or Hasura can accomplish). This is probably not something that has any appeal to backend folks though.

@shekarm
Copy link

shekarm commented Feb 14, 2020

Hi, I investigated different databases and it appears that multi-tenancy is not something you get free with other databases either. Any implementation of multi-tenancy will require access control lists and other security-related features and most databases require an enterprise license for the same. In someways, Dgraph is following those models.

@Relaxe111
Copy link

Relaxe111 commented Feb 14, 2020

Hello @shekarm could you please give concrete examples of such databases?
Thnks.

@Relaxe111
Copy link

Relaxe111 commented Feb 14, 2020

I respectfully disagree with you. Multi-tenancy and ACL are different features which can't be put together. In my experience, most (if not all) dbs multi-tenancy is open source. But ACL not so many dbs offers that feature either free or Enterprise.

@seanlaff
Copy link

seanlaff commented Feb 15, 2020

Data isolation (which is falling under the multi-tenancy bullet) is critical for dgraph to see success in our company. We have many interested parties, but lacking that feature makes it a non-starter.

If dgraph found a place in our stack, I could see us growing into the enterprise tier (e.x needing granular ACL, fancier snapshot/restore, etc), however lacking rudimentary data isolation in the free-tier hampers our ability to start the journey/build PoCs.

Specifically- risk of schema collision is the real blocker

@shekarm
Copy link

shekarm commented Feb 18, 2020

Currently, Dgraph implements multi-tenancy and user authentication as part of our ACL implementation, to validate users and their access credentials. We will look at this implementation and see if it makes sense to isolate the credential authorization required for multi-tenancy.

@shekarm
Copy link

shekarm commented Feb 18, 2020

On the issue of GPU acceleration, there is a separate issue opened by @marvin-hansen and it is being tracked separately here.

@seanlaff
Copy link

seanlaff commented Feb 18, 2020

@shekarm Thanks for your consideration- I think it maps to the elasticsearch pattern of supporting multiple indices in the free tier, and then supporting document (and field) level security in the enterprise tier.

@marvin-hansen
Copy link

marvin-hansen commented Mar 11, 2020

@manishrjain @shekarm @MichelDiz

Please support CSV data import in Dgraph.

Details in ticket #4920

@fpattyn
Copy link

fpattyn commented Mar 18, 2020

Can you reconsider adding multi-tenancy to the open source distribution? Being able to define different graphs in one database helps to solve the 'provenance' issue when integrating data from different sources. Every source adds data to a separate graph. It's a cool feature to be able to show where each data source contributed to a the complete knowledge graph.

@ganisback
Copy link

ganisback commented Apr 1, 2020

If does not support multi-tenancy in open source distribution, I have to back to janusgraph.
I think it's a basic feature for a graph database.

@willem520
Copy link

willem520 commented Apr 2, 2020

Hi,I want to know when the multi-tenancy will be supported in open source distribution. it is really important to me.I have used in product. single-tenancy. it means that I have to allocate server resources to each business. if I have 100 business, it will took a lot of server resources

@ganisback
Copy link

ganisback commented Apr 2, 2020

@willem520, from this roadmap, they do not plan support this feature in community edition, it will be included in enterprise edition.

@Relaxe111
Copy link

Relaxe111 commented Apr 2, 2020

Well this is not exactly that. According to comments written earlier, they will consider if it will make sense to open source multi-tenancy. So I think than to speculate around this issue, will be better to wait for an official announcement from Dgraph team. )

@willem520
Copy link

willem520 commented Apr 9, 2020

@willem520, from this roadmap, they do not plan support this feature in community edition, it will be included in enterprise edition.

I think they will reconsider this feature

@emregency
Copy link

emregency commented Apr 10, 2020

Hey all,

  • Would NATS Streaming integration be something interesting in relation with Kubernetes integrations? Event-driven k8s is gaining traction with serverless k8s frameworks e.g. OpenFAAS , Fission

  • Would a dgraph-graphql-js client (as mentioned by @smkhalsa) not be interesting? Gateway+federated services is a more common architectural pattern for enterprises and sophisticated applications than JAMStack-ish Server Side Rendered WebApps connecting directly to a dGraph /graphql endpoint.

@grizzly-monkey
Copy link

grizzly-monkey commented Apr 27, 2020

We were evaluating dgraph for our SaaS application. The planned feature that we would need is Multi-Tenancy
Most of the DBs give this as part of OSS code and is a basic necessity these days.
It would be great if you give it a thought to have this as part of open source road map.

Thanks
Jeet

@willem520
Copy link

willem520 commented May 22, 2020

it would be great for dgraph to support ingesting data from hive or hdfs and other similar big data stores. now, bulk loader or live loader only support local files

@iliketowritecode
Copy link
Contributor

iliketowritecode commented May 31, 2020

Most interested in "Single predicate sharded across groups" out of all of the things planned! I think it would be great for scalability and performance :)

@liveforeverx
Copy link
Contributor

liveforeverx commented Jun 9, 2020

Hi, @manishrjain !

I would like to join all other people here (at least 5) with a question about multitenancy.

I think, that almost everyone, who uses DGraph is interested or can profit with this feature or get a better user experience by using DGraph. Even, who just runs pet projects and not able to buy a full licence would be interested in this feature. Just not to run different Dgraph's instances for different pet-projects and mostly 2 Dgraph instances per pet project (for example, because dataset in dev and test is different and test is cleaned, so instead of running one Postgres, you need most probably to run multiple DGraphs per any project).

I know people, who simply do not take a database as a serious database without this feature (and it was most major no in adaptation on my previous company, as I remember and one of the important points in my current company) and it was one of the most oft complaints I heard from people on meetups.

It would be great, if you would think about making this feature accessible to everyone, it would be great, if it would be Open Source.

Just some examples:
Neo4j offers it for free: https://neo4j.com/developer/manage-multiple-databases/
ArangoDB offers it for free: https://www.arangodb.com/docs/stable/data-modeling-databases-working-with.html
OrinetDB offers it for free: http://orientdb.com/docs/last/OrientDB-REST.html#post---database

I personally do not know another database, which doesn't have this feature or one, which offers this feature in Enterprise edition.

But, even, if it would be a one-time payment, for an acceptable price for the sole developer for this feature, I would like to consider buying it even for personal use on my development machine to get a better UX from DGraph usage, as I know it from any other database I used before personally and professionally. There was an opinion that other databases offer it not for free, I don't know any other database, which has multiple databases, has an open-source version and doesn't offer it for free.

My personal example: at the moment, I reset DGraph by every switch between developing an adaptor for DGraph and my pet-project and after the switch, I refill my pet-project with data every time. My tests on the pet-project carefully designed to clean every rubbish they create by themself to avoid this problem and be able to use dev + test on the same database.

P.S. I gave talks in meetup about DGraph and Elixir, I'm the maintainer of the most advanced Elixir driver for Dgraph.

@liveforeverx
Copy link
Contributor

liveforeverx commented Jun 9, 2020

@larvinloy Wouldn't it be better to suggest enhancements for GraphQL+- with not supported use cases and challenge the enhancements for GraphQL+- to support this cases?

I think, it would be great to add some features to DGraph, which makes Gremlin not a requirement for any new greenfield project, so that GraphQL+- just covers this.

@sorenhoyer
Copy link

sorenhoyer commented Jun 12, 2020

dgraph looks dope, but if multiple databases / multi tenancy support is only planned for enterprise customers I think most people doing SaaS applications, at least startups will just look elsewhere (eg. ArangoDB) which could be a real shame and possibly a lot of lost revenue (eventually). I'm sure people will be willing to pay once they get a decent amount of customers on, so if you don't want to give it away for free, maybe make a limit of say 200, 500 or 1000 databases (1 for each customer/tenant) for the Community version. If you need more I'm sure you can also pay for it. I for one is in that situation right now. I'd definitely have started my first SaaS on dgraph if only you had planned this as a community feature, but in the end decided to go with Arango due to this :/

@abhijit-kar
Copy link

abhijit-kar commented Jun 23, 2020

@manishrjain,

With my limited knowledge of the back-end, some fantasy requirements for the website I am making & a few pages of High-scalability blog: I present to you my humble wishlist.

  1. Rate limiting
  2. Archiving or making older data read-only (Read only data can be compressed with higher compression ratio: e.g. using Z Standard)
  3. Input validation for each predicate
  4. NATS Streaming as buffer
  5. Integration with anything from CNCF
  6. LZ4 for compression instead of Snappy for writable data, as it is faster than Z Standard in both read & write, and uses less CPU power.
  7. ApolloGraphQL Plugin to support GraphQL+-
  8. Subscriptions & Live Queries

@mac2000
Copy link

mac2000 commented Jun 27, 2020

Wondering if there is any chance to see apollo federation support in future? e.g. ability to add new services backed by dgraph into existing ecosystem with other services and beeing able to extend existing entities and so on.

From implementation perspective there is really few things need to be made:

  • Add Query._service.sdl - which will return current schema as a string with user defined directives (@key, @extends, etc)
  • Add Query._entity - aka generic resolver which will take an array of resolve references ({__typename, id}) as input and resolve desired entitites

@StevenACoffman
Copy link

StevenACoffman commented Jun 30, 2020

@mac2000 gqlgen (which dgraph uses) supports Apollo federation (i.e. being federated ) out of the box as described here. However, graphql-go (which dgraph also uses) does not support being federated, see graphql-go/graphql#492 so it's not clear whether DGraph already has this capability or not.

Have you tested whether _service and _entity already work?

Clarification for others, Apollo Federation is made up of two parts:

  • Federated services, which are standalone parts of the graph (DGraph could support being federated)
  • A gateway which composes the overall schema and executes federated queries (out of scope for DGraph)

To be part of a federated graph, a microservice implements the Apollo Federation spec which exposes its capabilities to tooling and the gateway. The federated service can extend types from other services and add its own types that can be extended.

Collectively, federated services form a composed graph. This composition is done by a gateway which knows how to take an incoming operation and turn it into a plan of fetches to downstream services. The gateway orchestrates requests, merges the data and errors together, and forms the overall result to send back to the client.

@mac2000
Copy link

mac2000 commented Jul 1, 2020

@StevenACoffman indeed, seems like gqlgen supports it but if understand everything correct it also means that I gonna need to recompile dgraph itself and somehow enable this feature.

Did check given docker images and there is not _service, _enitity query fields.

Wondering if federation might be shipped by default? Reasoning: federation itself wont break any existing servicess because it does not do anything specific to service only I as a user can write down some directives and still until being connected to gateway it does not affect graph in any way

PS: probably I should create separate issue for this one, to get it up and running and add some docs around this topic
PPS: also I'm not sure how it will work then in slash

@MichaelJCompton
Copy link
Contributor

MichaelJCompton commented Jul 2, 2020

Hi @mac2000 and @StevenACoffman,

Thanks for the interest in Dgraph GraphQL and for the ideas around Apollo Federation. This is a bit of an intricate one, so sorry for the long reply.

A few technical points first. We don't take a dependency on graphql-go. Nor do we really take a dependency on gqlgen - I think we use one bit to help respond to GraphQL introspection. So supporting Apollo Federation would mean us implementing it. It's not as simple as flicking it on.

That said, we did cut a branch at one stage to test building it out, which worked fine, but we weren't convinced it should be built directly into Dgraph for these reasons:

  • It's not part of the GraphQL spec, it's a cool feature built by Apollo. Which is nice, but meant it wasn't on our core roadmap for our spec compliant GraphQL support. It's also not the only suggestion around for federation: for example, at GraphQL java, they are building a federation engine nadel that works the other way around by the federation saying what parts of the services it is federating, rather than the services telling the federation how to federate them. Which is a cool idea because it makes the federated services independent of the federation.
  • Because it's part of another product, rather than the GraphQL spec, it's likely to make it more of a moving target for us to support.
  • It's able to be done already by putting an Apollo server as a proxy in front of your Dgraph instance. That'll take a small amount of boilerplate, but I currently feel that if you are building an Apollo federation, it's likely that you have the capability for that, and it's a more Apollo native solution than us baking it in. (we have a blog coming soon around this)
  • There are other things arriving around federating GraphQL that I expect are likely to be more flexible than us building in one solution for Apollo federation. I've mentioned Nadel, but also I think (but haven't yet tried a working example) that GraphQL Mesh (https://graphql-mesh.com/) could be a really cool way to get federations (including Apollo Federation) without it having to be hard coded into our service.

So ... yes it's a cool idea; no it's not on the roadmap for the next quarter; yes please put up a feature request if you really want it built in (that'll help us gauge interest).

We are also watching this space with interest and will build out some examples around the products mentioned above over the next few months so we can give some guidance on how to use what's already available, and work out if we need to take something on, say Q4 2020 or beyond.

@StevenACoffman
Copy link

StevenACoffman commented Jul 2, 2020

In the Go community, it might also be worth noting these other two approaches:

Nautilus Gateway

Nautilus is based around the assumption that all your services are GraphQL Services. It uses the Node interface (Relay Gloabl Object Identification) to automatically federate between multiple services. This approach is quite similar to the approach Apollo Federation took. So all services have to comply to the Relay Global Object spec and you're ready to go. Nautilus gateway will analyze your services via introspection at startup time and generate the final gateway schema.

Tyk's Graphql Gateway

In contrast Tyk's Graphql Gateway (and I believe this tyk gateway as well? ) goes with a complete different approach. The basic assumption is that your public gateway schema should not be an artifact. The gateway schema is the contract between gateway and clients. The gateway doesn't make any assumptions on your services other than complying to some protocol and some spec. It's a lot more manual work at the beginning but this gives a lot of advantages. Because the gateway is the single source of truth regarding the schema you cannot easily break the contract. With federation an upstream service can directly break the contract. Additionally you're not limited to GraphQL upstreams. While GraphQL upstreams are supported it's only a matter of implementing another DataSource interface to support more upstream protocols, e.g. SOAP, MQTT, KAFKA, Redis etc.. On top of that, because the gateway schema is the single source of truth you'll get two additional benefits. First, you can swap DataSources for a given schema without changing the contract. E.g. you could replace a REST User-Service with a GraphQL User-Service without changing the contract between client and gateway. This could help to easily transition from legacy to more modern architectures. Second, because the gateway owns the schema you can apply features like rate limiting, authorization etc. at the gateway level.

@mac2000
Copy link

mac2000 commented Jul 6, 2020

@MichaelJCompton thank you for such explanation I have few thoughts if you will:

  • as about graphql spec - it also does not have DateTime scalar for example, but dgraph has, so the question then why DateTime is ok to add and _Service is not - what I'm trying to say here is that it does not correlate with spec or its change in any way it is just about few additional prebuild helpers in api
  • totally agree about third party, but it is preatty easy to implement and can be just turned off at any time because it is just extension and it is pretty generic but after all you are right here and there is no way i can convice on that in any way
  • indeed dgraph can be put behind gateway as is but it wont achieve anything from federation
  • proxies like mesh can be used (I still wait for PR on underlying library to make it more usable) but it will be hacky because we will be forced to deploy two pieces also graph manager wont work nice with such setup, e.g. I wish to be able to check schema changes right inside continuous integration process

Did go thru this in dotnet which also have no federation support, we were forced to implement it - it requires some time but at the end it becomes clear that it is just few more pieces which can be added/removed at any given time, so yes I agree with your point that it will be hard to keep track on that but might be if it will be some kind of pluggable turn on/off piece with support from community might be there is a chance it will work (but really not sure whether it is doable in golang world)

In either case dgraph seems to be pretty cool project hope will be able to give it a try

@pranaypratyush
Copy link

pranaypratyush commented Jul 15, 2020

Subscriptions and live queries are marked Q2. What's the status?

@pawanrawal
Copy link
Contributor

pawanrawal commented Jul 15, 2020

Hey @pranaypratyush

Subscriptions have been implemented and are available in master. They allow you to subscribe to changes in a query result i.e. they are implemented as live queries. They would also be part of the upcoming 20.07.0 release which should be out soon. We are currently working on getting the docs in place on how to use them. The docs should be available on https://graphql.dgraph.io/ in a week. We'll mark them as done once we release 20.07.0.

If you want to give it a try right now, you can try the dgraph master docker image. You can use the @withSubscription directive on any type that you want to generate subscription queries for. Here you can see an example of an end to end test for a subscription query. Let us know what do you think about it.

@minhaj-shakeel
Copy link
Contributor

minhaj-shakeel commented Jul 16, 2020

Github issues have been deprecated.
This issue has been moved to discuss. You can follow the conversation there and also subscribe to updates by changing your notification preferences.

drawing

@mac2000
Copy link

mac2000 commented Sep 2, 2020

BTW personally I just discovered that hasura.io has its own SaaS solution which is awesome, with such products like yours and theirs building new services will be fast and fun

But there is absolutely same request to support Apollo federation and what is fun almost the same answers, e.g. we have remote joins and hasura might be used as a gateway, but what if I want to use both dgraph and hasura which one should I use as a gateway?

In ideal world I wish to be able just go to site, press order button, configure federation, fields, done

But it seems that every project tries to be one tool for everything

PS: just wanted to share observations I do respect your position on this topic but without us, end users, talking/asking about this it will never happen

@iyinoluwaayoola
Copy link

iyinoluwaayoola commented Sep 16, 2020

Change Data Capture planned for Q3, any updates?

@manishrjain
Copy link
Contributor Author

manishrjain commented Sep 16, 2020

Unfortunately, no update on CDC yet. We had a feature freeze to focus on GraphQL. But, we'll reopen the feature work in October 2020. So, will ensure that CDC goes out.

@iyinoluwaayoola
Copy link

iyinoluwaayoola commented Nov 4, 2020

Unfortunately, no update on CDC yet. We had a feature freeze to focus on GraphQL. But, we'll reopen the feature work in October 2020. So, will ensure that CDC goes out.

It's past October, any update?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests