Support server-side pagination - alpha

[richard-cox]

Summary

Fixes #9478

----------------

This is an alpha and not meant for production or review (see list of updates required)

----------------

Occurred changes and/or fixed issues

• only applies to
  • specifically configmaps, secrets and pods. other resources can be incrementally added
  • cluster store. management store should in theory be simple to support
  • resources that don't have their own custom list. this should be easy to resolve though
• sorting / filtering are disabled for...
  • any column who's value was computed locally (e.g. model getters).
    • secret - subTypeDisplay - translates things like kubernetes.io/service-account-token to Svc Acct Token
    • configmaps - data - we convert data from two properties into something more readable
• sorting is enabled but not working for (backend issue)
  • columns that were originally from schemas
    • pod - Ready, Restarts and IP all come from schema --> metadata.fields[index] (attributes.columns[0].field: "$.metadata.fields[1])
• special note - state sort & search is done via actual value, rather than the label we apply locally

Updates post alpha

• loading indicator when pages are being requested (take care not to blip)
  • should be subtle, as per incremental loading indicator
• performance setting (global enable / disable)
• improve - the list group by feature adds a namespace option if resource is namespaced. we replace this with config for paths via listGroupsWillOverride. this isn't so neat
• improve - there's a lot of computed properties in sortable table that fire when things happen in store. need to avoid these
• we re-fetch the list when we return to the page. this means the list is updated, but user has to wait for http request again. could leave as is, candidate for refresh button
  • Refetch list on every visit
  • Add refresh button
• resolve two TODO: RC FIX
• list changed notification - when we receive a socket message for resource that is outside of current page
• when leave a resource list where pagination is enabled make sure to unsub

Updates that really should be in alpha

• add/update comments... everywhere
• remove debug consoles
• there are two subscribe messages sent for the list's type
• there's no label on the namespace groups. this is due to the field used to search instead of the value
• workload lists fail to load

Technical notes summary

General flow

• new resource list mixin determines if a list supports pagination, bubbles down to sortable table
• sortable table bubbles up pagination changes
• resource list pagination mixin makes request for new page given pagination settings
• find action takes pagination settings
• pagination settings applied in urlFor
• find action persists page, pagination settings and pagination results (total results)
• resource list bubbles down pagination results (total results) to sortable table

Design / Patterns

• updates to resources that are not in the current page are ignored
• We can only sort and filter by values known to the backend. Therefore a lot of the existing table headers, which reference model properties
  are invalid. To get around this we define new headers which refer directly to properties on the raw resource. see pagination-table-headers
  this means we avoid things like name & namespace property helpers, searchFields that are functions to values, etc
• the revision of the list is ignored. when user goes to page two it will be page 2 at that new time (rather than from the time page 1 was requested)
• Question. the only way for a user to update their page is to change pagination in some way. we should consider a refresh button

Backend / API Issues

• configmaps
  • different count provided when adding/removing sort=-metadata.namespace
    • Update: This might be FE... looks like the ns filter is also being removed alongside sort
    • Update2: It is a FE issue, toggling group by is for some reason removing the filter on ns
• secrets
  • the resource has a _type field, yet sort=_type does not work
  • the resource has a _type field, yet filter_type does not work
  • 
• pod
  • question. spec.containers is an array of container objects. we'd like to sort/search using container.image.
    • these can be filtered filter=spec.containers.image=alpine as per docs. sorting doesn't make sense
  • some columns come from schemas. the field is a specific value in an array. can we sort by it?
    • pod 'restarts' field is determined from a reference in it's schema. attributes.columns[3] field points to roughly the path in the pod object where the value is $.metadata.fields[3]".
    • we can filter by this (filter=metadata.fields.3=xyz)
    • sorting though doesn't seem to work (sort: metadata.fields.3)

Areas or cases that should be tested

• no regressions for
  • non-paginated lists
  • namespace/project filtering
  • manual refresh (list)
  • incremental loading (list)
  • garbage collection (?)
  • advanced filtering
  • fleet workspace selection
• group by namespace / no namespace
• group by namespace sticks on refresh
• the hideSystemResources product config (determines namespaces used in filters)
• the pref ALL_NAMESPACES (determines namespaces used in filters)
• user navigates to last page in a list with a single result on the last page --> in another browser delete the resource
• user navigates to page before last in a list with a single result on the last page --> in another browser delete the resource --> nav to last page