chore(*) introduce PaginationStore

[nickolaev]

Summary

Introduce the PaginationStore and delegate all pagination to be done there instead of implementation specific.

[bloqhead]

I'm trying this out and this is the query:

http://localhost:5681/dataplanes+insights?gateway=true&size=12

I still receive this error though:

{
  "title": "Could not retrieve dataplane overviews",
  "details": "Pagination and filtering at the same time is not supported"
}

Please let me know if I'm doing something wrong 😅.

Should this change work for Dataplane Insights also? My scenario is that I want to fetch only Gateway Dataplanes but I also want to reduce the size for pagination to 12 items per page. I would then do the same for Ingress and Standard (neither Ingress nor Gateway) Dataplanes.

I'm trying this in Universal on macOS. Thanks for the work you're putting in on this! It will make the GUI much more useful to the end user and give us a more robust HTTP API.

---

Edit: This commit fixes the issue! Thanks!

[jakubdyszkiewicz]

which makes their implementation more simple

that's not the dealbreaker. Dealbreaker was that it's not even possible to implement this on K8S. We could have implemented in memory pagination only on Kubernetes and native on Postgres, but it's still difficult because our DB schema in Postgres.

[nickolaev]

OK, I don't see this to be an incorrect statement. We are not explaining what is and what is not a dealbreaker in the code IMO. The point is to leave a note here for our future selves what actually is going on here. When the next persistent storage gets introduced we'll not even consider paging, we'll just do the simple "get all" version and let this implementation deal with paging. Which is actually the point.

[jakubdyszkiewicz]

I like to write comments that answer why is this (and what was the alternative) not what is this.

The point is to leave a note here for our future selves what actually is going on here

Exactly. So imagine there is a new developer on the team and we have performance problems with pagination. He takes this task and finds this code and reads "which makes their implementation more simple" and "proved to be fast enough, although not that efficient". So I see there was a compromise, we thought it was fast enough but it isn't now. But what was the alternative? I'd like to see in the comment that

• There is no filtering + pagination on the native K8S database, so you have to do this in memory anyways.
• On Postgres, we keep the object in a column as a string. We would have to use JSON column type and convert it to native SQL queries.

the next persistent storage gets introduced we'll not even consider paging (...) Which is actually the point

IMHO the main point was to get Kube and Postgres to work

[jakubdyszkiewicz]

nit: delete empty line

[jakubdyszkiewicz]

filteres typo

[jakubdyszkiewicz]

I like to write comments that answer why is this (and what was the alternative) not what is this.

The point is to leave a note here for our future selves what actually is going on here

Exactly. So imagine there is a new developer on the team and we have performance problems with pagination. He takes this task and finds this code and reads "which makes their implementation more simple" and "proved to be fast enough, although not that efficient". So I see there was a compromise, we thought it was fast enough but it isn't now. But what was the alternative? I'd like to see in the comment that

• There is no filtering + pagination on the native K8S database, so you have to do this in memory anyways.
• On Postgres, we keep the object in a column as a string. We would have to use JSON column type and convert it to native SQL queries.

the next persistent storage gets introduced we'll not even consider paging (...) Which is actually the point

IMHO the main point was to get Kube and Postgres to work