Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Caching in-memory postgraphile objects for performance. #1671

Closed
krish7919 opened this issue Oct 8, 2022 · 1 comment
Closed

Caching in-memory postgraphile objects for performance. #1671

krish7919 opened this issue Oct 8, 2022 · 1 comment

Comments

@krish7919
Copy link

krish7919 commented Oct 8, 2022

Refer #693 for some context, as this seems to be an extension of the other issue and specifically this comment from @benjie in 2018:

Are you trying to load it up as an executable schema? If so you cannot do that via JSON because the schema is generated in memory. We don't have a way for doing that currently - it's highly dynamic.

Thing that works:

  1. A process generates the SDL in postgraphile using writeCache option and stores this Graphql SDL.
  2. API call triggers the lambda, which then reads this SDK using the readCache option, which then handles the query and returns the response.

Problem:
Our instrumentation shows that that createPostgraphileSchema function takes ~2-2.5 seconds of time to read the cache file and then make the call to database to resolve the query. Our response times during has changed from 1-2 secs overall to >4 seconds now. One problem might be the size of the SDL, which on an average for us is ~2.5MB as JSON.

Question:
I try to use createPostgraphileSchema in the first lambda run, and then try to cache the postgraphile object (in some caching system like memcached or redis or such; memcached response times are in milliseconds) and then reload it in a subsequent lambda run. A serialisation operation of the schema and resolvers (in the first run) and a comparatively faster deserialisation of the object in memory should be much faster than reading the schema repeatedly and creating the object in memory for every Lambda API call.

Apparently, it seems to not work as thought as serialising the GraphQL schema object seems to be impossible. Is there a way the community has handled this issue when using Graphql on serverless and tuning for performance?

@benjie benjie added the 🔁 revisit-in-v5 Not happy with this behaviour, but changing it is a breaking change. label Oct 8, 2022
@benjie benjie added 🔮 fixed-in-v5 and removed 🔁 revisit-in-v5 Not happy with this behaviour, but changing it is a breaking change. labels Sep 29, 2023
@benjie
Copy link
Member

benjie commented Sep 29, 2023

In V5 you can export your schema as executable code, then you simply run that code (no PostGraphile, no Graphile Build hooks system, just new GraphQLObjectType(...) and similar calls) directly - completely eradicating this issue.

Read more here: https://dev.to/graphile/intro-to-postgraphile-v5-part-6-excellent-executable-exports-1150

@benjie benjie closed this as completed Sep 29, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants