Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Parallelize loading included objects #7981

Open
3 tasks done
mtrezza opened this issue May 7, 2022 · 3 comments · May be fixed by #6501
Open
3 tasks done

Parallelize loading included objects #7981

mtrezza opened this issue May 7, 2022 · 3 comments · May be fixed by #6501
Labels
bounty:$100 Bounty applies for fixing this issue (Parse Bounty Program) type:feature New feature or improvement of existing feature

Comments

@mtrezza
Copy link
Member

mtrezza commented May 7, 2022

New Feature / Enhancement Checklist

The following description was taken from @noahsilas who is the original author of the related (but stale) PR #6501.

Current Limitation

When preloading multiple fields, we can get some extra performance by doing multiple fetches simultaneously.

Feature / Enhancement Description

The solution here is to build what is basically a dependency graph out of promises; each include path blocks while it is waiting for whatever path it depends on to finish loading. Finally, once we have that graph, we return a promise that depends on every node in it.

Aside: Technically we only need to depend on leaf nodes, but there shouldn't be any meaningful difference between waiting on the leafs and waiting on the entire graph, and this way we don't have to do any analysis to find the leafs)

Example Use Case

Consider a query fetching comments, and requesting that ParseServer preload "post" and "author" pointers:

GET /classes/Comment?include=post,author

In this case, we first need to fetch the resulting "Comment" documents, but after that we should be able to fetch the documents referenced by the post and author fields of the results simultaneously, as they don't depend on each other at all.

Things get a little trickier when we have nested fields:

GET /classes/Comment?include=post,post.author,post.category

To resolve this query, we first need to fetch the related posts, and only once we've added the data about those posts into the results tree can we scan it for the post.author and post.category pointers. But, once that first fetch is completed, we can unblock both of those nested queries!

Alternatives / Workarounds

n/a

@parse-github-assistant
Copy link

parse-github-assistant bot commented May 7, 2022

Thanks for opening this issue!

  • 🎉 We are excited about your ideas for improvement!

@mtrezza mtrezza added the type:feature New feature or improvement of existing feature label May 7, 2022
@mtrezza mtrezza linked a pull request May 7, 2022 that will close this issue
@mtrezza mtrezza added the bounty:$100 Bounty applies for fixing this issue (Parse Bounty Program) label May 7, 2022
@mtrezza mtrezza changed the title Parallelize loading include objects Parallelize loading included objects May 7, 2022
@mtrezza mtrezza pinned this issue May 7, 2022
@Moumouls
Copy link
Member

Moumouls commented May 8, 2022

This feature will also drastically improve GraphQL API. This PR is challenging and interesting to develop. I'll see one of my colleagues and I can dedicate a coding pair session to effectively address this issue in a readable manner.

The dependency graph is the right way to go.

@mtrezza mtrezza unpinned this issue Oct 12, 2022
@dblythy dblythy mentioned this issue Oct 12, 2022
31 tasks
@Moumouls
Copy link
Member

@dblythy an update on the subject, if you are curious: #6501 (comment)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bounty:$100 Bounty applies for fixing this issue (Parse Bounty Program) type:feature New feature or improvement of existing feature
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants