Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[feature] Enable pagination for GUAC GraphQL APIs #1525

Open
lumjjb opened this issue Nov 21, 2023 · 4 comments
Open

[feature] Enable pagination for GUAC GraphQL APIs #1525

lumjjb opened this issue Nov 21, 2023 · 4 comments
Assignees
Labels
enhancement New feature or request

Comments

@lumjjb
Copy link
Contributor

lumjjb commented Nov 21, 2023

Is your feature request related to a problem? Please describe.
Pagination as a feature has been brought up multiple times. For example, for data display and also to enable faster queries (i.e. prevent UI stalling).

Describe the solution you'd like
Enable certain queries that may return large amounts of data to allow pagination.

Describe alternatives you've considered
NA

Additional context
Add any other context or screenshots about the feature request here.

@lumjjb lumjjb added the enhancement New feature or request label Nov 21, 2023
@pxp928 pxp928 self-assigned this Nov 25, 2023
@pxp928
Copy link
Collaborator

pxp928 commented Nov 27, 2023

The Relay-style pagination seems to be the most recommended and compatible with Apollo and can also be used by REST.

Note: ApolloClient can be used to enable caching of the results.

The graphQL schema needs to be updated with the following (per noun and verb):

type ArtifactConnection {
    totalCount: Int!
    pageInfo: PageInfo!
    edges: [ArtifactsEdge!]!
}

type ArtifactsEdge {
  cursor: ID!
  node: Artifact
}

extend type Query {
  "Returns all artifacts matching a filter."
  artifacts(artifactSpec: ArtifactSpec!): [Artifact!]!
 "Returns all artifacts matching a filter in chucks based on cursor"
  artifactsList(artifactSpec: ArtifactSpec!, after: ID, first: Int): ArtifactConnection
}

with PageInfo defied as below:

type PageInfo {
    hasNextPage: Boolean!
    startCursor: ID
    endCursor: ID
}

With the Edge, Connection, node and pageInfo defined as:

  • A connection is a paginated field on an object — for example, the friends field on a user or the comments field on a blog post.
  • An edge has metadata about one object in the paginated list, and includes a cursor to allow pagination starting from that object.
  • A node represents the actual object you were looking for.
  • pageInfo lets the client know if there are more pages of data to fetch. In the Relay specification, it doesn’t tell you the total number of items, because the client cache doesn’t need that info. It would be up to the developer to expose that information through another field.

Example test implementation for the key/value backend can be found here: https://github.com/guacsec/guac/compare/main...pxp928:artifact-ff:test-pagination-artifact?expand=1

Example implementation with ENT: https://betterprogramming.pub/clean-architecture-with-ent-and-gqlgen-a789933a3665

Other useful docs:
https://medium.com/@chris.czurylo/implementing-pagination-in-graphql-and-go-using-gqlgen-2ea3786a71dc
https://www.apollographql.com/blog/graphql/pagination/understanding-pagination-rest-graphql-and-relay/

@mihaimaruseac
Copy link
Collaborator

This is what I was planning to do for pagination too. Initially I thought we might want to define a custom paginationSpec argument and just add it to all queries, but, while this is simple (one single change, one single place in code to update), it does not follow the standard so it won't benefit from caching, etc.

Let's do the right thing and follow the process @pxp928 mentioned above. I think in the long term, artifacts will be the one that has pagination, replacing artifactsList, but for now let's use both to not break users while we migrate

@pxp928
Copy link
Collaborator

pxp928 commented Apr 21, 2024

Based on further discussion, removing before: ID, last: Int and hasPreviousPage: Boolean as it will not be utilized and add unnecessary complexity to the codebase. The other values remain as they are. The plan is to add artifactsList in one PR and remove artifacts in the following PR by replacing the functionality of artifactsList with artifacts (making any updates as necessary).

@pxp928
Copy link
Collaborator

pxp928 commented May 7, 2024

The only queries that have not been implemented with pagination yet are: FindSoftware and Neighbors

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

3 participants