2024-02.md

Composite Schemas WG - February 2024

Watch the replay: Composite Schemas Working Group Meetings on YouTube

• Michael covers the github repo and issues, we are at stage 0 and making many changes
  • He covers the spec language and core principles, and shows the defined terms in the spec and how we are extending the GraphQL core spec
  • Open invitation to raise issues, we should agree before committing
• The term "source schema" is under discussion - get involved via the issues. #11
• Structure of composition has received a lot of discussion. We've worked to define the scope: schema composition and distributed execution.
• We need to do like the GraphQL spec: describe the fundamental execution, but everyone can have a better execution algorithm for it. It's important to understand how it works, but most advanced servers deviate from that.
  • Martijn has a nice paragraph that we might add to make this clear.
  • Space for innovation is important.
• The spec text is very much in flux, but if you don't agree with Section 1 - Overview, please do raise issues soon!
• Simon: is the execution algorithm outside of the normative specification, or do we require the observable behavior remains the same?
• Martijn: the latter; we must not specify exactly how the query plans work, but the observable behavior must be the same.
• Michael: I want to ensure that regular OSS devs can easily get started; e.g. the GraphQL Spec can be implemented directly and you get an okay result - not the fastest, but it works.
• We're discussing names and parameters for directives, but the set of the directives is quite good - it's currently a mix of the directives from Fusion and Federation and should cover most needs.
• Section 4 - composition has received a lot of work (and had quite a bit removed). It talks about validating and merging source schemas.
• Martijn: it goes beyond the GraphQL spec validation; it ensures source schemas are syntactically and semantically valid (e.g. that the directives reference sensible things)
• Shane: Do we need directives to make it work? Or is a basic GraphQL schema valid? Like Typescript model?
• Martijn: we're favoring explicit over implicit right now.
• Yaacov (in chat): In general composition may require transformations to avoid validation failure and prune the merged schema…. I would overall suggest calling out any transformations as separate pieces, because the main goal here is composition but transformations and a separate nomenclature for that might be useful……
• John: the schemas must be valid GraphQL schemas, but we only require additional data in order to interact with the other schemas
• Michael: if we want to have inference we can have a pre-processing tool that infers the directives, and then ties into the main implementation. We're specifying the core spec, how people do work on top of that is up to them.
• Jeff: regarding "core specification" - what are the bounds of a common specification. Do we share a common understanding of what is in scope versus out of scope? What's going to be in the first draft versus later proposals.
• Michael: the general outline are the directives that we cover, there's no authn/authz/policies - those are "plus things" that we shouldn't dictate to everyone. The other main effort is describing the composition: how do we get from a couple of subgraphs to an executable composed schema. Also how does the execution work in terms of behavior.
• Pascal: composition and execution overlap.
• Benjie (in chat): Makes sense; e.g. helpers to build connections are common in GraphQL but aren't part of the spec.
• Benjie (in chat): My opinion on the spec scope: "if in doubt, leave it out"
• Martijn:
  • Composition: ensuring there's at least one possible way of getting each field
  • Execution: Generating an efficient way of getting each field
• Michael: "satisfiability" - ensuring the result of the merge process can be executed with all the different ways of getting to certain fields.
• We don't cover authz/authn/enterprise features that solutions can be put on top.
• Martijn: also we'll leave out all the transform directives/preprocessing like were in the original Fusion spec.
• Jeff: how do we decide what terms to use, how do we figure out the trade-offs and how to govern users? We need to keep in mind what's already running, and ensure people can migrate easily.
• Michael: I think with Apollo's link spec (with provides) should be possible. I think most of the directives we've taken from Apollo so far we've kept one-for-one. But the spec isn't implementable yet.
• Jeff: just want to ensure we share a common expectation of where we're going.
• Michael: I think we have a rough understanding, and there will be a lot of discussions on naming things. What we call things in the spec text you don't need to use those in marketing.
• Martijn: but of course directive names are critical for compatibility.
• Yaacov (in chat): A major transformation kind of necessary for composition is when an argument on a field in one subschema is actually a field in another subschema which may dictate execution order and cause the argument to be dropped in the merged schema (aka requires) / So that’s a specific transformation which alludes to potentially broader features
  • Reply from John (in chat): Do you have an example of what you mean? (Maybe a GH issue would be easier to elaborate, since chat is limited)
• Martijn: there's a lot of existing usage out there, so we must weigh the cost of changes against the migration costs - e.g. using the ECMAScript to model this.
• Pascal: seems like a good time to drop legacy concepts too.
• Michael: as Benjie mentioned in chat: there may be a preprocessing step that we can do to translate old schemas into new ones. If migration is difficult then that's something we should definitely discuss when the problems arise.
• Jeff: this seems backwards from a specification process - figuring out what is optimal and then making this compatible is exciting, but it might be better to capture what's out there and iterate from that towards an optimal solution.
• John (in chat): I agree. We should consider feasibility for migration, but ideally only after we come to the design we're seeking.
• Michael: I feel like that's what we've broadly done; there's things like _entities that we've not handled yet because it removes type security at runtime because it has an Any input, and it makes the schemas fundamentally insecure to expose. Other than this, everything else is very much aligned.
• Martijn: I agree, another difference is that there's a different syntax for requires; but we need a new way to bring in those arbitrary dynamic fields. Maybe we should establish a design principle of removing _entities and adding entity resolvers as a base, but changing other things would require more discussion.
• Pascal: definitely right to start by looking at what customers are using in production; great way to figure out what works and what doesn't.
• Benjie: the original reason for this WG was to capture the commonalities between all the different solutions and solidify that, so building off of production usage is the right way to do it (but attacking from both directions can be useful).
• Benjie (In chat, in response to the directive in Section 4- Composition): Maybe @entitySource rather than @entityResolver - as in "heres where you can get the entity from"?
• John (in chat): My hope is while we're iterating we don't unintentionally exclude ideas because they don't match an existing production solution. Sounds like everyone's on the same page there, so should be good 🙂
• Yaacov (in chat): ill try to post a discussion on the wg about what i meant about transformation for composition....this is apollo's example for requires:

  • type Product @key(fields: "id") {
      id: ID!
      size: Int @external
      weight: Int @external
      shippingEstimate: String @requires(fields: "size weight")
    }

• Martijn: we want to describe what the composition stages are, why, and really explain before going on to the details - so there's a lot of content left to be written!
• Pascal: as you start writing algorithms, your understanding improves massively and that can really help you find problems in the spec - so getting more people involved in writing validation rules would be great.
• Michael: please read section 1; if you don't agree with something, please open an issue. As Martijn says, first we want to specify the idea before we write the algorithms around the ideas.
• ACTION - everyone: read Section 1: Overview - outlines design principles