GQL Mesh Improvements - Summary Issue [trivago feedback]

[angelsvirkov]

This is an ongoing issue which is used to keep track of potential GQL Mesh improvements. The feedback comes from use cases in trivago.

Most of the issues or ideas are related to the following use case:

• programmatic usage of GraphQL Mesh
• combining data from several sources (GQL, MySQL, etc.)
• using various schema transforms
• providing authorisation layer on the final schema
• integrating the final schema into an existing server

Example Repository: https://github.com/Kiesen/graphql-mesh-express

Tasks:

• 1) The GraphQLHandler cannot use a ready-made introspection schema. -> GraphQLHandler - Cannot use the introspection property #2783
• 2) Provide a documentation recipe for using 1 schema with 2 different sources
• 3) Try using envelop instead of express-graphql. Might be useful to replace the post execution of extensions from express-graphql with some module from envelop in order for the persistChanges layer to work smoother.
• 4) Try moving the full schema building process to compile time instead of runtime. Identify blockers.
• 5) Provide documentation for various programmatic modules used in the repo

[santino]

Hey @angelsvirkov,
happy to read a conversation about feedback and potential improvements.

Probably people watching this repo can share several thoughts on the points you have raised.
But I wonder how you'd like this conversation to progress.

Is there any specific action item you'd like GraphQL Mesh to start looking into?

[angelsvirkov]

Hey @angelsvirkov,
happy to read a conversation about feedback and potential improvements.

Probably people watching this repo can share several thoughts on the points you have raised.
But I wonder how you'd like this conversation to progress.

Is there any specific action item you'd like GraphQL Mesh to start looking into?

Hi @santino , I have discussed that with @Urigo and @ardatan . The main idea is that we together work on specific things to improve. I will be adding different issues for every task where we can continue a more specific discussion. Also I might continue adding points to this overall issue summary. I can try solving some of those myself or else I will ask for help

[JustusNBB]

Hi! I'll be interested in your ideas on future work considering introspection (caching), but since this discussion doesn't have an acceptance criteria, it fits better there ("discussion" not "issue")?

2/ there are two strategies to combine multiple sources, and we have examples:

• federation: https://github.com/Urigo/graphql-mesh/tree/master/examples/federation-example/
• stitching: https://github.com/Urigo/graphql-mesh/blob/master/examples/openapi-location-weather/.meshrc.yaml

3/ before you build your own gateway you should consider SDK generation solutions too

4/ that's like it already works for SDK generation?

[angelsvirkov]

Hi! I'll be interested in your ideas on future work considering introspection (caching), but since this discussion doesn't have an acceptance criteria, it fits better there ("discussion" not "issue")?

@JustusNBB , I would agree that a discussion could be a better place. Should we convert it?

2/ there are two strategies to combine multiple sources, and we have examples:

What I was talking about here is having 2 GraphQL Data sources which use the same schema. Only the endpoints and the actual data content are different. We did make it work by using encapsulation and reusing the same schema for both data sources. I see this as a problem which others might also face, so I was wondering if a short recipe / documentation would be something useful. I could write that if you believe it makes sense.

3/ before you build your own gateway you should consider SDK generation solutions too

We have considered using sdk generation solutions but we decided to go our own way and using your APIs programmatically. We have this for almost a year now and we are happy with the solution as it is flexible and we could integrate it into our existing service and reuse some functionality from it.

4/ that's like it already works for SDK generation?

Yes, it works for it but we would like to do it programmatically and not via the provided mesh commands because we cannot be flexible with them.

[JustusNBB]

Yeah seeing how this branches the discussion forum should be a better place imo.

2/ there are two strategies to combine multiple sources, and we have examples:

What I was talking about here is having 2 GraphQL Data sources which use the same schema. Only the endpoints and the actual data content are different. We did make it work by using encapsulation and reusing the same schema for both data sources. I see this as a problem which others might also face, so I was wondering if a short recipe / documentation would be something useful. I could write that if you believe it makes sense.

Ah now I understand better what you mean, I saw related issues too and I am still looking for a great solution myself, documentation is not rounded out for this yet since the build/runtime split imho.
Please leave a comment from your point of view: #2618

3/ before you build your own gateway you should consider SDK generation solutions too

We have considered using sdk generation solutions but we decided to go our own way and using your APIs programmatically. We have this for almost a year now and we are happy with the solution as it is flexible and we could integrate it into our existing service and reuse some functionality from it.

Can you explain more what you have been doing exactly?

4/ that's like it already works for SDK generation?

Yes, it works for it but we would like to do it programmatically and not via the provided mesh commands because we cannot be flexible with them.

Environment configuration does not work for you?

[angelsvirkov]

@JustusNBB , I think I have no rights to convert this issue to a discussion. Regarding points 3 and 4 - You can check out the example repo we have set up. It contains a small scale replica of how we use the mesh. Regarding the discussion in #2618 about Multi Mesh vs Mega Mesh I am still not quite sure what that means and how it relates. I'd think more about it.

[JustusNBB]

You are using lerna, see this related nx topic: #2103

https://github.com/Kiesen/graphql-mesh-express/blob/master/packages/backend/src/mesh/mesh.ts

That reminds me of myself half a year ago, switching from the Gateway approach to SDK generation, back then I started to programmatically composing the mesh like you do here, without a .meshrc.yml (workaround findAndParseConfig...) but I got stopped by frequent internal mesh changes, and then graphql-mesh was refactored and split into build & runtime with introspection cache - I currently have more that one .meshrc.yml file for smaller (validation-only) use cases, while trying to get TypeScript (ts-node) dependencies removed from the built artifacts, maybe not related - just fyi.

@santino @ardatan can you recommend the code-only approach (without .meshrc.yml) or is it more preferable to rely on one or more .meshrc.yml files in general and for specific use cases?

[ardatan]

Sorry for late response. I'd recommend to keep using CLI if possible. We recently did a few improvements by removing ts-node dependency etc.

Try using envelop instead of express-graphql. Might be useful to replace the post execution of extensions from express-graphql with some module from envelop in order for the persistChanges layer to work smoother.

We're currently working on Envelop/Yoga integration of Mesh. I'd recommend to check https://www.graphql-mesh.com/docs/recipes/custom-server Custom Server feature to provide your own server that uses Envelop in favor of Mesh's.

Try moving the full schema building process to compile time instead of runtime. Identify blockers.

We are gradually moving schema generation process from runtime to compile time. It also needs some work on Schema Stitching side.

Provide a documentation recipe for using 1 schema with 2 different sources

Are you referring to dynamic endpoint URLs while using the same introspection?

The GraphQLHandler cannot use a ready-made introspection schema.

I created a PR on your repo that seems to fix the issue you had.

[charlypoly]

Hi @angelsvirkov!

Thank you for your complete feedback!
Please know that we took notes of your suggestions when rewriting the GraphQL Mesh documentation: https://www.graphql-mesh.com/docs/introduction

We now provide more advanced examples/tutorials (single sources, multiple sources, sources with no definition):

• programmatic usage of GraphQL Mesh: https://www.graphql-mesh.com/docs/guides/extending-unified-schema
• combining data from several sources (GQL, MySQL, etc.): https://www.graphql-mesh.com/docs/getting-started/combine-multiple-sources
• using various schema transforms: https://www.graphql-mesh.com/docs/getting-started/combine-multiple-sources
• providing authorisation layer on the final schema: https://www.graphql-mesh.com/docs/getting-started/customize-mesh-server
• integrating the final schema into an existing server: https://www.graphql-mesh.com/docs/getting-started/customize-mesh-server

Let us know if you have any feedback on the improved documentation 👀