add pagination feature for querys

[letmejustputthishere]

right now for APIs that need a mandatory limit field, there is no easy out of the box way to allow for pagination, filling up the store query by query.

the reponse from the query have to be stored outside of the KIT QL store and concatenated with each subsequent queries response.

from a conversation with the author of this library it seems like he is already working on this feature.

[jycouet]

Once #156 is fully operational it will be possible.

---

We have to document things. My notes for now:

Pagination Types

Type 1 (I don't know what name to give?)

Item form 0 to 9
button > next
Item form 10 to 19

Type 2 (Infinite Scroll / append)

Item form 0 to 9
button > next
Item form 0 to 19

Pagination Values

• Should be easy to use
• Should not overfetch
• Should work with
  • offset, limit on a List
  • first, last on connection / edge

[letmejustputthishere]

that's amazing news, very excited!

would this also work without manual interaction like clicking a button? e.g. incrementally fetch as much data as there is for a given query and notify me once thats done?

[jycouet]

Once building blocks are in place, I guess that it's not that complicated to do yes 👍
But if you have 1 000 000 lines, you probably don't want to have everything loading in multiple queries... you will wait for the user to see the bottom of the page to load the next x items, no?

Something like on:line 15 visible => loadNextPage (next 20 items)

[letmejustputthishere]

that depends on the use case i guess 😄 i want to query the github graphql api for date ranges and then visualize the data. the visualization only makes sense with all the data available. i could of course emualte the behaviour of a user clicking a button until all data has loaded using totalCount.

[jycouet]

You will definitely not need to create a fake button to click on it 🤣

[jycouet]

Could you share your operation query? I'm curious!

[letmejustputthishere]

You will definitely not need to create a fake button to click on it 🤣

haha no, i meant programatically 😄

Could you share your operation query? I'm curious!

query BurstinessPRs($querystring: String!, $endCursor: String = null) {
	rateLimit {
		cost
		limit
	}
	search(query: $querystring, type: ISSUE, first: 100, after: $endCursor) {
		pageInfo {
			endCursor # points to the last item
		}
		edges {
			node {
				__typename
				... on PullRequest {
					url
					createdAt
				}
			}
		}
	}
}

^ this for example returns the first 100 items for this search, but there are potentially more than 100 items. i want to visualise the entirety of the data, not only the first 100 items.

is there an ETA for this feature to arrive?

depending on that i would just code the hacky solution where i copy the data out of the store into my own object where i aggregate the queries.

[jycouet]

You will have a function doing this directly called loadNextPage.
Here is the current doc https://www.houdinigraphql.com/guides/pagination
We will have a few updates before the release but essentially it will fill the store merging more and more data into it.

In your case, you want to "avoid" pagination and get "everything". Fine for 80 items, 102, items, 202 items...
But what if "everything" is 1 000 000 items?

[letmejustputthishere]

that's a good point. i was thinking of capping the queries at maybe 1000 elements (would be cool if we could set this cap optionally). if i don't cap it, in my special case the API has a rate limit that will be triggered anyways.

[jycouet]

So you will be able to loadNextPage until 1000.

first: 100 => loadNextPage => 100
first: 100 => loadNextPage => 200
first: 100 => loadNextPage => 300
first: 100 => loadNextPage => 400
first: 100 => loadNextPage => 500
first: 100 => loadNextPage => 600
first: 100 => loadNextPage => 700
first: 100 => loadNextPage => 800
first: 100 => loadNextPage => 900
first: 100 => loadNextPage => 1000

[jycouet]

You have it Out-Of-The-Box in Houdini now.
You can check the KitQL Doc Pagination and to go more in depth, the Houdini Doc Pagination