Skip to content

Improve subscription handling for more efficient data transmition #55

@nexus49

Description

@nexus49

Current Behavior

Today a subscription on a list (e.g. accounts) returns a number of events before it has transmitted all of the accounts. It starts with an empty array and then sends an event for each received account.

This is highly inefficient as many events are send before all requested items are received.

Improved Behavior

Instead of sending a huge number of events the proposal is the following:

  • Subscriptions only return changed items:
    When subscribing to a list like accounts the client can provide a resource version and events would continue starting from that provided resourceVersion or newest if none was proided. The gateway would then send an even event for each resource that has changed - as received by the watch client from k8s.
  • Need for envelope: In order to allow clients to properly process these events it is necessary to indicate if this is an UPDATE, CREATE or DELETE event so that clients can update their internal cache. The envelope should be closely aligned to the envelope that the api server returns. https://pkg.go.dev/k8s.io/apimachinery/pkg/watch#Event
  • In order for clients to display updated lists, clients would first issue a query for the resource and the have a follow on subscription using the resourceVersion that was received in the query.

Sub-issues

Metadata

Metadata

Assignees

Labels

enhancementNew feature or request

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions