Skip to content

run withDescendents with filters#8014

Merged
filfreire merged 4 commits into
Kong:developfrom
jackkav:git-performance-1
Sep 30, 2024
Merged

run withDescendents with filters#8014
filfreire merged 4 commits into
Kong:developfrom
jackkav:git-performance-1

Conversation

@jackkav
Copy link
Copy Markdown
Contributor

@jackkav jackkav commented Sep 26, 2024

motivation: large git sync collection block the main event loops by running withDescentants 15 times on each tab navigation

rather than fetching all descendants 15 times
we just fetch the types we need, the problem is we need to also fetch all the folders

ideas:

  1. stop doing this altogether
    1a. just read and write to files and not a fake filestore-nedb adapter
  2. do it in a more optimized way
    2a. run this in a different conetxt so it doesn't need to be re-evaluated on navigation.
    2b. run it faster by fetching a list of all folders in a workspace and iterations over the types using an $in query

Related: INS-4503

filfreire
filfreire previously approved these changes Sep 26, 2024
@jackkav jackkav changed the title stop running withDescendents for every folder run withDescendents with filters Sep 26, 2024
Comment on lines +159 to +161
const modelTypesWithinFolders = [models.request.type, models.grpcRequest.type, models.webSocketRequest.type];
const typeFilter = modelTypesWithinFolders.includes(type) ? [models.requestGroup.type, type] : [type];
const children = await db.withDescendants(workspace, null, typeFilter);
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

perhaps consider an excludive option here isntead, eg [models.ApiSepc.type]

@filfreire filfreire merged commit c802256 into Kong:develop Sep 30, 2024
@jackkav
Copy link
Copy Markdown
Contributor Author

jackkav commented Sep 30, 2024

Related: INS-4503

@yaoweiprc
Copy link
Copy Markdown
Contributor

If we want to get all requests in a workspace, It's unintuitive that we need to pass [requestGroup, request] as the queryTypes argument to withDescendants and then filter out the requestGroups from result. Maybe we can do it within withDescendants method.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants