New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Namespaces #163

Open
OlegIlyenko opened this Issue Apr 10, 2016 · 69 comments

Comments

Projects
None yet
@OlegIlyenko
Copy link

OlegIlyenko commented Apr 10, 2016

At the moment all GraphQL types share one global namespace. This is also true for all of the fields in mutation/subscription type. It can be a concern for bigger projects which may contain several loosely-coupled parts in a GraphQL schema.

I tried to describe this feature in detail in following article (which also includes motivation and different use-cases):

GraphQL Namespaces Proposal

After posting this it, I saw interest from different people within the GraphQL community, so I decided go one step further and open this issue in order to suggest inclusion of this feature in the spec and start a discussion around it.

I suggested particular syntax & semantics for the namespaces, but I myself still not 100% sure that it can be used as-is. I hope, that as we discuss it, we would come up with better syntax and semantics (or maybe even entirely different feature) that better fits existing language features and covers described use-cases.

@hepin1989

This comment has been minimized.

Copy link

hepin1989 commented Jun 7, 2016

I think the / syntax is great ,with the current GraphQL spec,we need mount the on GraphQL ObjectType to another bigger GraphQL object as a field and a new name,like Mount Dummy type to a bigger graph as dummyService field.

And still I think #174 have something similar,just a little namespace.

@leebyron

This comment has been minimized.

Copy link
Collaborator

leebyron commented Jul 2, 2016

Thanks for writing this up, Oleg!

I'm a bit confused as to what this is enabling that isn't already possible or what the immediate goal of using namespaces would be.

I also expressly want to avoid namespaces being included and then people naively namespacing everything just because the feature is there, since that would result in a lot of bloated queries.

For example, in your post you explain that we could write queries like this:

{
  person(id: 12345) {
    id
    firstName

    schema.org/parent {
      givenName
    }

    schema.org/knows {
      givenName
      taxID
    }
  }
}

but I'm curious why you would prefer that over:

{
  person(id: 12345) {
    id
    firstName

    parent {
      givenName
    }

    knows {
      givenName
      taxID
    }
  }
}

The latter seems to represent the same thing, but just requires less typing. Is the goal to just describe what the fields relate to? Perhaps we could have metadata about the fields in the introspection system?

@IamCornholio

This comment has been minimized.

Copy link

IamCornholio commented Jul 2, 2016

I think that the namespaces here enable one to "import" another schema. One could potentially import an schema and build on top of that rather than starting from scratch every time. Implementation wise it looks tough to do though. It might just be easier to import a library (js, ruby, python, java depending on the graphql implementation language) and then build schema using types available from that library.

Another reason why namespaces could be useful is so one can filter a introspection query with a namespace string. In a large system the number of types can make the introspection query results quite unwieldy. But then one could always write a custom introspection query that accepted a filter on a field that's common to all types.

I can think of another use of namespace which is that the number of top level fields (especially those with parameters) and mutations can become quite unwieldy and so one would have trouble viewing them in a graphql api browser like GraphiQL. Having top level fields and mutations further filtered by a namespace would make things easier on the eye.

@OlegIlyenko

This comment has been minimized.

Copy link
Author

OlegIlyenko commented Jul 3, 2016

@leebyron Thanks a lot for looking into this and giving your feedback!

I think we can approach this from 3 different perspectives. First is a semantic meaning of types and fields. This is where schema.org example came from. I think for this one it should be enough to have a user-defined metadata on a field and type level which is exposed via introspection. I think community then can come up with conventions for how to encode this semantic information for use in different tools.

The second is the fact that the mutation (as well as subscription) fields share global namespace. Mutation type is the only place where mutation can be defined for a whole GraphQL schema. This is where my original motivation came from. I went into details about this particular point in the original post ("Mutation Type" section).

There are several factors to why I see it as a concern. At the moment mutation type should be an ObjectType, which means there that all mutation fields should be defined in it. This makes it hard to work with loosely-coupled parts of the schema which potentially may have name conflicts. In my opinion, one of the contributing factors is also the lack of polymorphism in input objects. This greatly limits amount of logic one can encode in one particular mutation field, which means that different variations of updates need to be represented as top-level mutation fields.

Namespaces can provide a solution for this particular problem, but I'm pretty sure that there are also other approaches we can take. For instance, let's imagine that it is possible to use a union type as a mutation type:

type AuthorMutation {
  create(firstName: String!, lastName: String!): Author
  changeName(id: String!, firstName: String!): Author
  delete(id: String!): Author
}

type ArticleMutation {
  create(title: String!, authorId: String!): Article
  changeText(id: String!, text: String): Article
  delete(id: String!): Article
}

union Mutation = AuthorMutation | ArticleMutation

schema {
  mutation: Mutation
}

a query like this would not make much sense because field name is ambiguous:

mutation {
  create(firstName: "John", lastName: "Doe") { id }
  delete(id: "1234") { id }
}

but we can use an inline fragment to disambiguate it:

mutation {
  ... on AuthorMutation { 
    create(firstName: "John", lastName: "Doe") { id }
  }

  ... on ArticleMutation { 
    delete(id: "1234") { id }
  }
}

If we provide a shorthand syntax for inline fragments, it can become a bit more cleaner:

mutation {
  AuthorMutation/create(firstName: "John", lastName: "Doe") { id }
  ArticleMutation/delete(id: "1234") { id }
}

This code violates the semantics of inline fragments and union types - they normally represent 2 mutually exclusive fields, still I use them here just to disambiguate the field names, so both mutation fields would be resolved. For the sake of example I use this approach simply because there is already syntax available for it in the language. The point I would like to make is that this example does not involve namespaces, still it addresses my issue with the global namespace for the mutation fields.

The third perspective is the the global namespace for the types. I recently had a few conversations with different people who are thinking on introducing GraphQL in their projects and companies (from small to pretty big). One question that came up several times in the discussion is whether GraphQL should serve as a facade for different internal services managed by different teams and if yes, then how does DDD fit into this picture. I do believe that GraphQL is a perfect fit for a facade API. Many client applications often have cross-cutting concerns, so it is very helpful to provide a facade for these client through a single GraphQL API, especially if clients are constrained in some ways (like mobile clients or embedded devices). On the other hand I find it helpful to think about different internal services as a bounded contexts. Each bounded context comes with it's own ubiquitous language. Given a system designed with these principles in mind, name conflicts are actually desired, but ubiquitous language within different bounded contexts can define very different semantics for the same terms (like User, Account, Product, etc.). I don't want to say that it's the best way to model system or anything like this, it's just some people use it and find it helpful for their own systems (including myself). This is where namespaces can be a very helpful tool when designing a GraphQL facade that aggregates several bounded contexts. I would be really interested to know your opinion on this topic. Do you think that GraphQL is suitable as facade in these kind of systems and if yes, then will you try to define the single model for all bounded contexts or you will try to preserve the ubiquitous language of every bounded context and minimize a common/shared area of the schema? (/cc @dschafer, @schrockn)

I also expressly want to avoid namespaces being included and then people naively namespacing everything just because the feature is there, since that would result in a lot of bloated queries.

Indeed, I do share this concern. I guess if one asks himself whether he/she needs to use namespaces, then the answer is probably no. On the other hand, as I described above, namespaces may be a useful tool to solve particular challenges (although, not the only possible tool in every given case).

The latter seems to represent the same thing, but just requires less typing. Is the goal to just describe what the fields relate to? Perhaps we could have metadata about the fields in the introspection system?

Indeed, in this particular example query I would normally skip the namespaces. This is the nice thing about the namespaces in contrast to naming conversions and other namespace emulation techniques which have quite a few disadvantages: even in presence of namespaces one can just skip them if they are not necessary (some implicit namespacing rules will apply in this case). There are also a few reasons why I may use explicit namespace in this query:

  • As you mentiond, I may use it to make it clear, for people who view this query, that this field is defined in schema.org spec and has particular semantics associated with it (best type of documentation:))
  • I anticipate change to this type in future and would like to keep my query forward-compatible
  • After some time we decided to implement schema.org interface in this type, but this cased name collisions. I will then use namespace to disambiguate the field. This is especially important in case of mutations (as i described above), where name collisions are intentional.

That said, I don't really want push concept of namespaces per se. My main goal is to address some of the issues, like global namespace for a mutations fields. Whether it is solved with namespaces or not does not really matter. I hope what I described here reveals a bit my original motivations for this proposal :)

@aschrijver

This comment has been minimized.

Copy link

aschrijver commented Jul 31, 2016

Hi @OlegIlyenko ,

Nice elaboration and spec you`ve written down!

In general I see a clear need for a namespaces mechanism. I am currently writing microservices that communicate using graphql. With microservices architecture each microservice represents a separate bounded context (in DDD terminology) and you can easily get type conflicts when there are many of them around. I.e. same name, different semantics, etc. Also - like you said - there will be a need for common (base) types to be defined.

Your proposal certainly has its charms, especially the notation. Yet I also have a number of reservations:

  • Additional complexity in server implementation while namespaces support could be optional
  • Less visual appealing, harder to grasp query language (visual clutter, readability)
  • It is opinionated. It offers a single way to define namespaces, while there are many existing ones around
  • You will either stay opinionated and have people stick other namespace implementations on top of their implementation, or you will have to gradually adopt all the nooks and crannies that are also in other namespaces to take care of special cases ('my namespace string has a slash', 'mine is an object, not a string', etc.).
  • It is a breaking change! Or at least it will be in environments where the server cannot dictate the query format and there are multiple spec flavours around (old and new ones)

In another issue I proposed a more general metadata mechanism, that is an extension and not breaking. See #200
The namespace could be defined in this metadata using any standard mechanism that is around (well, there should be some compatibility with JSON actually, but still).
The structure of the metadata is defined like any type and there could be introspection support with a __METADATA type.

Using this metadata proposal for namespaces, however, is far less powerful than what you propose. But it can be adjusted. What I like in this is the freedom it leaves to implementers.
Having metadata support we could wait and see how the community use it, what metadata formats are popular, before adopting a specific approach (Hell, there could be a Metadata Profiles feature in future and a 'marketplace' for profiles and other extensions. Yeah more marketplaces..)

Finally, I would be in favour of having a lean and mean base spec, and then have an extension mechanism that allows you and others to offer the kind of cool stuff you propose. It could be a standard extension. And I could consult a server to see if it supports this feature before sending my query (Note this is similar to the extension mechanism in OASIS CMIS spec and others).

Arnold Schrijver

@yarax

This comment has been minimized.

Copy link

yarax commented Aug 16, 2016

Just a note: I'm currently writing a wrapper around existing REST approach based on microservices (with plenty of endpoints) and namespaces are very wanted, otherwise all type names look like fact_reports_pp_campaigns_delivered

@DomKM

This comment has been minimized.

Copy link

DomKM commented Aug 23, 2016

I would also appreciate namespaces for all of the reasons stated above by @OlegIlyenko.

@leebyron Given that Facebook has nearly 10,000 types, how does it keep track of type names and query/mutation fields (which I am assuming there are hundreds if not thousands of as well)? Are there any best practices for merging schemas from multiple sources?

@xuorig

This comment has been minimized.

Copy link

xuorig commented Nov 23, 2016

As our mutation root grows we've been asking the same questions internally.

Given that Facebook has nearly 10,000 types, how does it keep track of type names and query/mutation fields (which I am assuming there are hundreds if not thousands of as well)?

Anyway you can share alternative solutions or insight on how you've been handling that problem @leebyron 🙏 ?

@smolinari

This comment has been minimized.

Copy link

smolinari commented Nov 24, 2016

This might be totally off the wall and possibly even a stupid comment showing my lack of understanding, but couldn't different URLs and a routing middleware be used like namespaces? One could split up a larger app among different endpoints that way.

So, a bigger site would have

https://some-site.com/user
https://some-site.com/blog
https://some-site.com/admin

etc.

And if things are getting too involved even with this "namespacing", you could also break it down more.

https://some-site.com/user/profile (or register)
https://some-site.com/blog/page (or /gallerie)
https://some-site.com/admin/blog

I know, I know. We are heading towards something RESTy-like again with that idea, but it isn't really REST, right?

This might not solve the microservice communication issue, which I believe this idea came from. But, I think it solves the "big schema" and "name collisions" issues, since each endpoint would have its own schema. You'd also get the decoupling, which was asked for. It would allow for better asynchronicity too, because you could hit different endpoints at the same time, if needed, which I don't think would be too often. It would allow for parallelism, because breaking up into separate endpoints, theoretically, would allow a system to be scaled out much more easily later on too. And, if you refine the URLs, it isn't a breaking change.

The cons would be duplication. But really, if an app gets that big, I would imagine creating schema would be automated in some way anyway, right? Or does Facebook really have their devs go into the schema and alter or add to it all the time? I find that hard to believe.

I'm sure this blows away some other general concept that makes GraphQL so great, which I am completely missing due to my own lack of understanding. So, please do give me hell. All I can do is learn. 😄

Scott

@khamusa

This comment has been minimized.

Copy link

khamusa commented Jan 9, 2017

I'd like to note that allowing namespaces to be optional as long as they're unambiguous may difficult future developments of the schema, since you cannot safely add new types that would add namespace ambiguity having already allowed clients to write unnamespaced queries.

Regarding syntax, / seems an interesting choice, but I'd also propose . as a separator (in my opinion it is slightly less cluttering).

@IvanGoncharov

This comment has been minimized.

Copy link
Collaborator

IvanGoncharov commented Jan 9, 2017

As @khamusa point out:

you cannot safely add new types that would add namespace ambiguity having already allowed clients to write unnamespaced queries.

So you force all consumers of your API to bloat their queries by prefixing every type and field.
I really like the idea that GraphQL force you to define a consistent vocabulary for your system.

And the biggest concern I have is that people starting to auto-generate one huge GraphQL proxy to expose a bunch of microservices without even try to reuse common types. And instead of solving communication problem in your organization you forcing your API clients to work around this by using prefixes and reading a ton of docs to understand the difference between: schema.org/Person, Accounting/Person, CRM/Person, etc.
Moreover, I imagine a lot of developers will abuse this to implement versioning and I don't want to constantly use v1, v2, v3 namespaces.

It certainly possible to do the same stuff right now by making v1_Accounting_Person but it looks ugly and not encourage by any existing GraphQL feature.

Finally, as it was pointed out previously Facebook has nearly 10,000 types without any name clash.
And I don't think many other API will ever approach this number.

@leebyron

This comment has been minimized.

Copy link
Collaborator

leebyron commented Jan 9, 2017

@IvanGoncharov illustrated a number of API design abuse patterns that worry me as well.

I've still yet to hear a compelling argument for why namespaces are necessary. The only argument I've heard so far is naming collision. Frankly, that's been a non-issue for us at Facebook and our schema continues to grow well over 10,000 types.

We avoid naming collisions in two ways:

  1. integration tests.

We don't allow any commit to merge into our repository that would result in a broken GraphQL schema. This test is pretty simple and fast to run, we just boot the server and if it fails to boot with a GraphQL error, we fail the test. Two types or fields on a type of the same name would cause such a failure. This kind of test ensures our master branch isn't constantly broken, and protects against name collision problems. If someone accidentally choose the same name for a new type, they can resolve the problem. Often, resolving it actually means using the existing type rather than renaming the new one. That keeps our API internally consistent and avoids duplicate information.

  1. Common naming patterns.

We have common patterns for naming things which naturally avoid collision problems. For example, mutations often follow a "verbNoun" pattern such as "likeStory" assuming that the noun is the type it operates on and your types also don't have conflicts as well, this naturally avoids conflict. I suppose with namespaces you could have written "Story/like" but that doesn't offer clients anything they didn't already have, and it might force them to handle multi-part names when reading the result. For types, we try to generalize as much as possible so that we have types shared across products. That makes future product integrations easier as well. For products that shouldn't integrate, or have similar concepts that are rightfully different, like a Facebook Event and an advertising logging Event, we alter the name to clear the ambiguity for the reader.

@mvpmvh

This comment has been minimized.

Copy link

mvpmvh commented Jan 16, 2017

Say I'm looking at your API in graphiql, I'm either a new fb hire or some dev looking to integrate with a part of your API. I pick 1 of your 10,000 types and I want to dig into the mutations that fall under this type. How do I, as a new hire or third-party dev, easily consume this information?

@maciejkrol

This comment has been minimized.

Copy link

maciejkrol commented Feb 21, 2017

For products that shouldn't integrate, or have similar concepts that are rightfully different, like a Facebook Event and an advertising logging Event, we alter the name to clear the ambiguity for the reader.

@leebyron How do you not end up with horrible caterpillar names like @xuorig said? I have not so very big schema and this is already a problem.

I tried to look for examples of big schemas or best practices. Nothing. Every article I could find is based on a schema with ~3 simple types. If you have anything please consider answering this question.

@leebyron

This comment has been minimized.

Copy link
Collaborator

leebyron commented Feb 21, 2017

Say I'm looking at your API in graphiql, I'm either a new fb hire or some dev looking to integrate with a part of your API. I pick 1 of your 10,000 types and I want to dig into the mutations that fall under this type. How do I, as a new hire or third-party dev, easily consume this information?

Typically our types do not correspond 1:1 with mutations. We typically do not have CRUD style mutations, instead we often have RPC style mutations that correspond to user events. In the cases that there is a closer relationship, the most common mutations are typically mentioned in the description of the type. For example the Story type includes a mention in it's description of the likeStory mutation, since it's a common question.

As I mentioned in the prior comment, naming conventions can help a lot. If you're looking at GraphiQL, the ideal experience is that you start typing what you expect to work and get typeaheads with helpful suggestions. In addition to this, both search in the documentation explorer, and helpful descriptions with recommendations for common mutations are all part of the strategy of providing information in the right places at the right time.

@leebyron

This comment has been minimized.

Copy link
Collaborator

leebyron commented Feb 21, 2017

@maciejkrol - our type system has many types, but with some clear and easy to follow naming conventions we balance between clarity, conciseness, and expandability.

The primary goal when naming things in a type system expected to expand in the future is to have clear answers for these two questions:

  1. Does this name introduce ambiguity? (e.g. if a type is called X, do you find yourself asking "what kind of X is this?")
  2. Can we imagine future expansions of this API in which this name will become too generic and therefore ambiguous? This question is much more difficult to answer, but forward-thought is always helpful when designing long-lived systems.

For example at Facebook we have a type called User, which in the beginning was clear via context that this meant a User of facebook.com. As our schema expanded, we introduced other types like InstagramUser, ThirdPartyUser, and other more specific types unique to certain circumstances or areas of the schema. Interface and Union types then help use related types together. For example, we have an interface called Actor which represents the entity which performed some action in the product, which is often a User but could be a handful of other types. We also have some type names that are longer and much more specific like EventsCalendarDateTime which belong to specific sub-products and have product-centric nuance that would be inappropriate for more generic shared types. In general this balance has worked for us: terse names for broadly used types and specific unambiguous names for specific or single-purpose types.

@Cyberlane

This comment has been minimized.

Copy link

Cyberlane commented Mar 9, 2017

@leebyron my only concern with your suggestions is that the Docs link within the GraphiQL will show a massive list of mutations under Mutation. Is there perhaps a way to decorate each item with some type of Category which will only affect the generated docs, whilst leaving the actual schema alone?

Our schema has two entry points, which is really nice.

Our mutations however are in the hundreds... and without being able to split them up a bit, the generated documentation becomes useless to our developer teams.

@syrusakbary

This comment has been minimized.

Copy link

syrusakbary commented Mar 9, 2017

@Cyberlane this issue might be related #252 :)

@jhclouse

This comment has been minimized.

Copy link

jhclouse commented Jun 14, 2017

The ideal situation is one in which you have complete control over the design of your schema and can justify a well-crafted set of type names. Unfortunately, many of us are in a different situation: we have an ever-growing and ever-changing set of disparate backend data sources described with something like Swagger and would love to be able to import them on the fly without curating them. Maybe namespaces aren't the best idea. But it would be great to have some kind of support for this scenario.

@isaksky

This comment has been minimized.

Copy link

isaksky commented Jun 30, 2017

@leebyron For cases where you know the object you would like to mutate, but don't know what cutesy name your colleagues decided to give the mutation (updateStory, changeStory, likeStory), I think this would get annoying. Think about this: why do we organize our functions into modules and namespaces as programmers? Don't those reasons also apply here?

Also, what about allowing it only for top level fields? A lot of what I see in this thread looks messy/overkill, but allowing it just for the top level would solve most of the problem. Fields after the first level already have a context, so further namespacing after that may cause more problems than it solves.

@KieronWiltshire

This comment has been minimized.

Copy link

KieronWiltshire commented Jul 19, 2017

https://github.com/Quture/app/issues/1

This is exactly what I need for ^^

@RickMoynihan

This comment has been minimized.

Copy link

RickMoynihan commented Sep 11, 2017

Hi,

I'm pretty new to GraphQL, so please take my comments with a pinch of salt, but I see the need for namespaces for some usecases, and I think there are several distinct reasons to want them.

  1. If you're generating your schemas dynamically for example from a database schema then the possibility of collisions is undesirable, and having to prefix fieldnames/typenames/enums etc is pretty ugly.
  2. More importantly without namespaces it's hard to see how it's possible to share schemas and target them to create portable clients across different services. Without them how do I know that the field droids contains the rebel droids I'm looking for? Namespaces would help turn schemas into contracts and enable extensibility and discovery.

You might say why do we need portable clients? But we already have the need for one as the graphiql IDE is used across many implementations and it appears to rely on __schema being a contract with the server. The use of the __ prefix here seems to be a tacit acknowledgement of the role of namespaces as __schema really means graphql/schema (to borrow the syntax of this issue). The problem is that __schema is effectively now reserved, embracing namespaces would be embracing 3rd party extensibility and would turn a closed system into an open one.

Namespace & module systems often allow importing a namespace as an alias, e.g. you could imagine something like the following (inspired by SPARQL), though I suspect you could find a prettier syntax:

ALIAS myns: <http://foo.com/mynamespace> 
ALIAS schema: <http://schema.org/>

{
  schema/name 
  myns/name
}

Unnamespaced names with the exception of __schema could then default to being in an anonymous namespace for backwards compatibility.

@leebyron's idea of putting this data into the introspection system sounds like an interesting proposal, though I'm not yet sure if it would be equivalent to first class namespace support.

@thecaddy

This comment has been minimized.

Copy link

thecaddy commented Sep 29, 2017

This should happen. Lee may not support it but the standard is bigger than FB now.

👍

@alloy

This comment has been minimized.

Copy link

alloy commented Nov 23, 2017

I don’t want to really double post and definitely don’t think my thoughts on the topic are crystallised well enough to actually partake in this debate, so for now I wanted to just link to some rough thoughts on this subject in the context of ‘stitching’ schemas from various services apollographql/graphql-tools#495 (comment)

@leebyron

This comment has been minimized.

Copy link
Collaborator

leebyron commented Nov 20, 2018

@acron0 that tone is not helpful. Please keep discussion civil. Also, I haven’t left.

I believe @mjmahone was trying to reiterate previous feedback that while namespaces were proposed to handle “large” schema, Facebook’s is probably the largest by quite a lot and never felt any growing pains that suggested namespaces were necessary. I also appreciated his advice and feedback on the gateway concept.

As for news, all work is pretty well captured here. I’ve applied the appropriate labels, but there hasn’t been much forward momentum on this concept in quite some time. The next steps are either to decide that GraphQL does not need namespaces and close this issue, or to have a champion take on this work and develop it to meet RFC 1.

@cruhl

This comment has been minimized.

Copy link

cruhl commented Nov 20, 2018

@leebyron I plan on creating a tool which "compiles" my module-enabled schema definition to spec-compliant GraphQL. Here's my initial plan...

module Time

scalar Date
scalar Duration

...would produce...

scalar Time__Date
scalar Time__Duration

You can import/reference modules like so...

import Time

type Event {
  start: Time.Date!
  duration: Time.Duration!
}

...or...

module Event

import Time (Date, Duration)

type Event {
  start: Date!
  duration: Time.Duration! # Qualified name still allowed
}

...or...

module Event

import Time as CoolTime (Date, Duration)

type Event {
  start: Date!
  duration: CoolTime.Duration! # Qualified name alias still allowed
}

Those would both produce...

type Event__Event {
  start: Time__Date!
  duration: Time__Duration!
}

I plan on following Elm's module schematics pretty closely. Is building this kind of usable-preprocessor tool a good start toward more serious discussion?

@rmosolgo

This comment has been minimized.

Copy link

rmosolgo commented Nov 20, 2018

I'm not leebyron but I'll answer anyways 😆 I think YES that would be a great first step: you could demonstrate a working, valuable example as a starting point for the specification.

That's something I really appreciate about the GraphQL change process: only proven features are added, that way, we don't end up accidently adding complexity without any value.

So, if you prove the value and smooth out the bumps, that's a big step towards standardizing! Please do chime in here when you get that project started, I'd love to see how it plays out :)

@acron0

This comment has been minimized.

Copy link

acron0 commented Nov 20, 2018

@cruhl I think some of your examples are wrong: start seems to be duplicated in the Event type.

@xuorig

This comment has been minimized.

Copy link

xuorig commented Nov 20, 2018

I've re-read this issue today to try and get a feel of why we want namespaces.

From what I'm reading, most concerns are in regards to type conflicts and ambiguity. I've been thinking a lot about GraphQL modules/composition/stitching/gateway/etc recently. I've also been interested in knowing how we can represent Bounded Contexts and preserve a ubiquitous language across different parts of the schema.

To me, whether your schema is distributed on n services or found in one big schema, there could still be a need for defining bounded contexts in the GraphQL Schema. You may have two representations of User within a single GraphQL schema just like you would if you had to merge a 3rd party schema or a schema from your other services. So in that sense, I don't think the argument that people are starting to use GraphQL in a distributed way is necessarily a great one for adding namespaces.

Now it also means I do see value in having something like namespaces, but as many have said before me, it is already possible to remove ambiguity and conflict using naming right now. The only downside I see right away with the approach is sometimes over-complicated or "uglier" names for types.

Having namespaces also brings a whole set of other questions. Can a field from one namespace reference any other type from other namespaces? Is there a concept of entry points to other namespaces? Can I have private types within a namespace? If not, it almost seems like an illusion of namespaces, to have better looking naming?

As far as stitching and conflict handling goes, I have some opinion that maybe the schema should not be distributed in this way. In fact I can see a gateway tool allowing something like this, it looks quite similar to what others have tried (cc @cruhl). Slightly different than auto-prefixing modules because depending on where your GraphQL module is used, maybe we'd like the gateway/merge point to decide on final names.

schema.graphql

# import "User" from "service-a" as "UserA"
# import "User" from "service-b" as "UserB"

type Query {
  userA: UserA!
  userB: UserB!
}

This example looks like an SDL extension, but it could just be code also. In this scenario, some services define their schemas, but the final schema is generated statically as a "monolithic" schema using imports.

So basically, I've yet to see something that needs to be in the spec, besides the argument that naming becomes a little bit less optimal than if we had some kind of namespace notation. I see value in namespaces but I'd like to see whether they would have other functions. I can see that a spec maintainer would prefer the status quo here since almost everything seems possible right now given the right tooling, and adding such a feature to the spec comes with huge implications. To me the balance is still a bit more on the status quo side right now.

Maybe my thinking comes from using a "Schema as Code" approach for so long vs the very used SDL first approach out there, but I've always had no problem using code / extensions and treating the GraphQL Schema or SDL as an artifact/result of the schema build. Maybe that's why some of us see more value in namespacing if the SDL is used directly as the way to build a GraphQL API.

@cruhl

This comment has been minimized.

Copy link

cruhl commented Nov 20, 2018

I personally think the addition of modules over namespaces might be a better direction for GraphQL. An ecosystem of commonly shared modules for things like dates/times/searches/images/etc. would be great. For instance, I use these types in most of my schemas...

  scalar Time__Date
  scalar Time__Duration
  scalar Time__Milliseconds

  type Time__FormattedDate {
    iso: String!
    unix: Time__FormattedDuration!
    humanized: String!
    formatted(template: String = "h:mm A, dddd, MMMM Do, YYYY"): String!
  }

  type Time__FormattedDuration {
    humanized: String!
    milliseconds: Time__Milliseconds!
    seconds: Float!
    minutes: Float!
    hours: Float!
    days: Float!
    weeks: Float!
    months: Float!
    years: Float!
  }
@mjmahone

This comment has been minimized.

Copy link
Contributor

mjmahone commented Nov 20, 2018

To be clear: I'd love to see someone champion this. It would include building out, as much as possible, something on top of the existing spec. This might involve using directives on type definitions that, in the future (provided they give a lot of value), we might want to fold back into the language. It's also quite possible after implementing it, the implementer will find what they really want isn't quite namespaces but instead modules or something else entirely. I would be really excited to see that happen.

My main point from above: even "at scale", namespaces are not required. Given that, the barrier to adding them is not "many people say they want this" but instead "we have a champion who has built a working implementation that's gone through a few iterations and the value of folding the implementation into the spec as opposed to as a layer on top of the spec is high". I'm not convinced that namespaces will meet that bar (therefore I'm not going to try being the champion), but I'd be really happy to be proven wrong.

@leebyron

This comment has been minimized.

Copy link
Collaborator

leebyron commented Nov 20, 2018

I really agree with @xuorig and @mjmahone's views above so I want to show support for them.

@cruhl - I think it would be really valuable to see something real built out that implements namespaces/modules. It gives us concrete ground to test out these ideas and ask real questions of them. I applaud and encourage your effort.

However I do want to make sure the bar for admission into the spec is clear. As @rmosolgo pointed out, we move concepts into the spec once they've proven value. My immediate naive reaction to your illustrative example is that the compiled output is preferable to the modularized input - this is because I value having fewer concepts.

module Time

scalar Date
scalar Duration

# ...would produce...

scalar Time__Date
scalar Time__Duration

In this example above, the modularized version requires the additional concept of a module while the compiled result works today without any changes and is equally descriptive (at the minor cost of longer type names).

The bar for admission into the spec is that this must provide some value that was not possible before, e.g. "Beyond syntax sugar, it is not possible to do ______ without namespaces/modules" - and this newly created value must at least offset the cost of added concepts and complexity.

One particular concern I have for complexity cost is how the GraphQL type system projects into other existing languages - both for building servers and for using code generation on clients. Right now types are always named and named with identifiers that can be directly used by almost every common language.

TL;DR - my concern here isn't if it's possible or not or if we can come up with syntax we're happy with - it is whether the entire concept adds enough value to offset its costs.

@zcourts

This comment has been minimized.

Copy link

zcourts commented Nov 20, 2018

@leebyron et al, we've been watching this issue to see where it'd go because we'd like to have it.
On our platform two independent developers can submit GraphQL definitions for their apps that have overlapping types used for totally different things, we ourselves provide a common/shared set of definitions and we allow an app to depend on other apps where we then pull all definitions into one big schema for that app.

So far we've been able to get away with just asking them to prefix their types but it's becoming more painful.
So we've been watching this because we didn't want to roll our own, this evolved and the community took it in a different direction and suddenly all the time our customers spent building their apps resulted in non-standard GraphQL models we'd either have to deprecate or continue being non-standard. So we've got a catch-22

I prefer the syntax @yordis used above

namespace IAM {
  type Account { ... }
}
namespace CRM {
  type Account { ... }
}

IAM.Account
CRM.Account

Since it's becoming more of an issue for us, I'm happy for us to pursue a minimal implementation, try it with a few customers, get their feedback, iterate and the come back here with findings.

Given we're already doing schema stitching for app dependencies, we can probably get it in there with just a little more effort.
We're on the JVM and use graphql-java and will need to modify its parser for the syntax but should be limited to this.

Assuming we did this, is that initial syntax good to run with for now? Although I say it's not "a lot", when we make changes from the feedback we get, I'd rather it was additions/modifications rather than throw it all away and start again and to minimise that I think getting a roughy idea on syntax down goes a long way.

Does that sound like a plan to try and move this forward? I suspect we can have this live and customers testing in Q1 2019

@yordis

This comment has been minimized.

Copy link

yordis commented Nov 21, 2018

I definitely prefer to go with my proposal.

Two main reasons.

Simplicity and Explicitness

The way we merge schemas, or inject types, or do any mutation to the final schema is agnostic of GraphQL specification, pretty much.

Do whatever and however, you want when it comes to that part of the specification.

Adding complexity on how to load parsers (like XML), or depending on the network layer will introduce another level of indirection and the specification will suffer from it.

Also if you go for the route of doing import X for example, you will make something simple, complex.

How do you resolve type now?

Since you introduce import now you need to figure out for each type where the type came from.

Did it came from the current namespace scope or do I need to resolve the type by parsing the current block and figure out the namespace used?

How do you deal with namespace collisions between imports?

I don't believe is worth to pursue this, this route introduces complexity without justifying the needs (in my opinion), typing less shouldn't justify introducing complexity like this one into the specification.

Real problem

The needs of namespace come from the lack of namespace (Ba Dum Tss) no anything else, at least from my understanding.

So.

Going for this route

namespace IAM {
  type Account { ... }
}
namespace CRM {
  type Account { ... }
}

type Companies {
   accounts: StrawHat.Account
} 
  1. We need the namespaces for avoiding collisions
namespace StrawHat {
  type Account { ... }
}
namespace AWS {
  type Account { ... }
}
  • Any definition inside the namespace block will be scoped by the namespace name.
  1. We keep the explicitness and stateless definition
type Companies {
   accounts: StrawHat.Account
} 
@leebyron

This comment has been minimized.

Copy link
Collaborator

leebyron commented Nov 21, 2018

I’m happy to hear you’ll test this out with real scenarios to learn more.

Again, I think the real question that will continue to come up with this proposal is “what does the namespaced AWS.Account provide which the simply named AWSAccount does not?” - that has been the prevailing question for this issue and to my memory has not yet been satisfactorily answered.

I’m not sure I buy the argument of avoiding collisions. An api designer which pulls in an AWS schema should know not to name their own types starting with “AWS” - in fact tools could be built to assert this.

@leebyron leebyron closed this Nov 21, 2018

@leebyron leebyron reopened this Nov 21, 2018

@leebyron

This comment has been minimized.

Copy link
Collaborator

leebyron commented Nov 21, 2018

Another point I want to make sure is clear. Let’s focus all problem statements on the API consumer. One of GraphQL’s core values is that it is driven by the needs of the client. The majority of the discussion so far has been about enabling remote schemes, schema stitching, and other schema authoring details. Any solution which results in leaking implementation details of how a schema is put together to an API client is not ideal. Namespaces, if we include them, should have a very clear value proposition to the API client in terms of new capability.

@yordis

This comment has been minimized.

Copy link

yordis commented Nov 21, 2018

One of GraphQL’s core values is that it is driven by the needs of the client.
@leebyron

Well, if this is the case then GraphQL works as it is since you could prefix all your types and that is how you solve the issue of naming collision.

Except that with true namespacing dev-tools (for documentation especially) could benefit from this but, since GraphQL allow extensions we could solve the metadata issue with it.

So, if that is your argument, definitely we do not need to do anything (and we live with long names but whatever)

@zcourts

This comment has been minimized.

Copy link

zcourts commented Nov 21, 2018

@leebyron ok. Well, our case is all about schema authoring and stitching (which for us is very important, the less annoying things are to our customers the better it is for us).

I'll get it into our roadmap anyway because while our case right now is all about stitching and authoring I'm sure once we and our customers start building real stuff with it we'll get to find out if it really brings anything to clients.

I'll see if we can get it in there and available to customers in Jan so we'll have answers to the client question soon.

I presume by client you don't just mean the JavaScript client but in fact any GraphQL client?

@freiksenet

This comment has been minimized.

Copy link

freiksenet commented Nov 21, 2018

I want to weight in as the original author of schema stitching. I agree with @xuorig that schema stitching has lots of issues. I regret many things in its design and I especially regret the fact that it became so popular within the community, because I feel it's been way overused.

It also feels that the fact that SDL-driven development became so popular in JS, we are trying to too much to solve problems that were created by using that format. Current solutions don't allow easily mixing SDL and programmatic schemas, even using SDL to create parts of your schema isn't possible (there is code inside schema stitching that does that, but no public code). It feels that maybe we wouldn't even have the namespace issue if our approach to making GraphQL schemas in node.js would have been less SDL driven.

@freiksenet

This comment has been minimized.

Copy link

freiksenet commented Nov 21, 2018

Also note to schema stiching users - there are ways to namespace stuff through schema transforms, I think that would be another way to test if this idea is viable. Have you tried using schema transforms to do that? How is that working for you, what issues are you having and which of those issues would be solved via namespaces?

@jhclouse

This comment has been minimized.

Copy link

jhclouse commented Nov 21, 2018

It sounds like the real problem here is what Facebook intended GraphQL to be—and what people are trying to do with it—are two totally different, and possibly incompatible, things.

I think what you guys wanted it to be is a curated abstraction layer for data. You take the complexities of various backend systems and thoughtfully present one single easy-to-use schema to exactly one client with a single set of requirements.

What a lot of people are using it for is an automated schema aggregation system that acts as a proxy for multiple backend data systems and presents them in a flexible way to any number of clients with any number of requirements.

These two things are mostly orthogonal. Maybe tools can help bridge that gap somewhat. But if we’re expecting it to be an example of the latter, we’re “gonna have a bad time.”

Is that a fair analysis of the situation?

@leebyron

This comment has been minimized.

Copy link
Collaborator

leebyron commented Nov 21, 2018

I presume by client you don't just mean the JavaScript client but in fact any GraphQL client?

Correct. Any client.

@leebyron

This comment has been minimized.

Copy link
Collaborator

leebyron commented Nov 21, 2018

@jhclouse you are exactly right, I think.

The SDL was not originally intended to be a human written document, at least not beyond writing examples and using it to communicate ideas. The SDL was originally intended to be a human read document. There are plenty of other design decisions that favor readability over writability.

Your description of graphql as a whole is apt. I really like your choice of word “curated” - I think those of us who worked on graphql in the early years believe you need to think about the design of an API to produce something that can be easily consumed. This has always been my biggest philosophical issue with schema stitiching. It always felt to me like giving up space for design decisions, which felt like a net loss.

At the end of the day, schema stitching and remote schema don’t bother me much as long as they remain transparent to someone actually using the resulting API. More choices for API designers is a good thing as long as it doesn’t come at the cost of less choices for API consumers.

The one thing I’m not sure I agree with you on is about a single client vs many clients. A big early motivation for graphql was supporting a potentially infinite set of clients with different needs. This is why graphql let’s you select the data you need rather than pre-defining a shape to be returned. I think having support for a diversity of client will remain important to the future design process

@xuorig

This comment has been minimized.

Copy link

xuorig commented Nov 21, 2018

The one thing I’m not sure I agree with you on is about a single client vs many clients. A big early motivation for graphql was supporting a potentially infinite set of clients with different needs. This is why graphql let’s you select the data you need rather than pre-defining a shape to be returned. I think having support for a diversity of client will remain important to the future design process

💯 I'd like to double down on this. One Schema doesn't mean one requirement. One of the most powerful thing about a GraphQL API is that supporting multiple use cases, behaviors, or clients comes with no costs (cognitive, or in terms of performance) to other client teams, or API teams. A lot of different solutions (Backend for Frontends, Netflix's Server Adapters) have been created in the past to solve similar problems and GraphQL is another great one to me.

Whether its schema stitching or remote schemas, it shouldn't change anything to the client. I still don't see how it's not currently possible to build tooling around it, the fact parts of the schema are network separated should not matter at all.

@zcourts

This comment has been minimized.

Copy link

zcourts commented Nov 21, 2018

@leebyron great.
To the other comments, I acknowledge all the views here on the matter, we watched the issue for a long time without contributing because we didn't want to get caught up in a battle of philosophy that ends in stalemate.

For our case we were doing the equivalent of schema stitching before we adopted GraphQL, it wasn't a case of something cool we picked up from the community, we designed with it from the get go without know what anyone else was doing (and later found it was being called stitching). It met a requirement that our product doesn't work without.

I've spoken to everyone I needed to on my side, I've gotten consensus to proceed with my proposal from above. As such, it'll be in our sprints from Dec. I'll go back to being a silent observer on the matter until after we've delivered this, if anyone has questions or comments I'm happy to be contacted offline courtney.robinson [at here] hypi dot io. I've reached out to some customers we know are starting new projects soon as well and I think they'll be open to testing this when it becomes available.

I'll close by saying this, successful open source projects tend to evolve far beyond their original aims.

An under dog feature that was initially dropped as a nice to have can easily evolve into being an indispensable part of the project. SDL and Schema stitching are both totally in that situation here I think. When this happens I think the best course of action is to let it play out and then figure out how to best fit it into the rest of the ecosystem without trampling over the original beliefs and aims of the project.

I'm glad @leebyron et al are open to seeing if this brings real value once it's in the hands of real users, I think that's a fair way to let this play out. It could well be that at the end of this it's added nothing valuable...we'll deal with that then but if the opposite happens I think it changes the conversation completely.

Then we need to talk about how to go about making underdog features have a place to grow as part of the ecosystem. I think the last thing we want is for every implementor to start add random features that they have no way of reconciling with upstream. One approach to avoiding this may be some well defined mechanism by which any implementor can add vendor specific extensions with some compatible migration path into the upstream spec after the extension has proven to be useful/gained adoption and then I guess some community process by which these extensions get promoted upstream into the spec.

@acjay

This comment has been minimized.

Copy link

acjay commented Nov 21, 2018

@zcourts

Then we need to talk about how to go about making underdog features have a place to grow as part of the ecosystem. I think the last thing we want is for every implementor to start add random features that they have no way of reconciling with upstream. One approach to avoiding this may be some well defined mechanism by which any implementor can add vendor specific extensions with some compatible migration path into the upstream spec after the extension has proven to be useful/gained adoption and then I guess some community process by which these extensions get promoted upstream into the spec.

I think there's a good story for this in the Scala community. A section of the community particularly interested pushing the boundaries of the powerful type system grew frustrated with features they wanted never being prioritized. Eventually they forked the compiler.

A lot of people thought this would split the community, but it didn't. It was designed to be strictly binary compatible with mainline Scala, such that libraries could use the Typelevel compiler but application code could still use the mainline compiler, even in the same build. All developments from mainline Scala were rapidly backported. All innovations were released with corresponding PRs to mainline Scala. A good number of these have been merged.

I guess you could call it a "pilot fork". It's not an easy path, but it can work, if really thoughtfully done.

@KieronWiltshire

This comment has been minimized.

Copy link

KieronWiltshire commented Dec 3, 2018

I'll give a use case for where namespaces would be super useful for me.

I'm building a modular API router in nodejs that in the end will take in plugins. These plugins will handle their own routes using graphql through the user of a namespace such as /plugin/<pluginName/?query.

The reason for namespacing plugins is that plugins can't be named the same name, therefore hiding each plugins route handle behind a namespace prevents conflicts.

However... my issue with this is that how does someone now use all the benefits of querying graphql?
What if I want to get something from plugin A and plugin B, I'm performing two separate queries. Wasn't the idea behind graphql to prevent that? Wouldn't it be easier to query against just /?query but then namespacing the schema provided by each plugin?

It's a very niche use case, but perhaps my issue with graphql shines light on the subject.

@raderio

This comment has been minimized.

Copy link

raderio commented Jan 11, 2019

@aethys256 aethys256 referenced this issue Feb 9, 2019

Open

feat: implement ResolverPrefix decorator #122

3 of 3 tasks complete
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment