New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement IPVS "Dynamic Feedback Load Balancing Scheduling" #10

Open
murali-reddy opened this Issue May 22, 2017 · 3 comments

Comments

3 participants
@murali-reddy
Member

murali-reddy commented May 22, 2017

IPVS has this nice LB method http://kb.linuxvirtualserver.org/wiki/Dynamic_Feedback_Load_Balancing_Scheduling

Which is all the more relevent in case of distrubuted load balancing requirements of 'ClusterIP' and NodePort service types. Each node doing load balancing in round robin fashion has below limitations.

  • if connections are mixture of short lived and long lived then we will end up in imbalance
  • each node is acting on the local connection information it has. Due to distrubuted nature each node ends up doing load balancing without taking account of other nodes or global state

Here is the proposal:

  • Implement monitor daemon that collects connection per endpoint from the node using contrack
  • use in-memory distrubuted key-value store to persist the data.
  • use the global view of connections to dynamically adjust the weights on each node
@bzub

This comment has been minimized.

Show comment
Hide comment
@bzub

bzub Jul 7, 2017

Collaborator

Initial Implementation

The monitor daemon could weight real-servers with:

  • Standard pod metrics provided by Heapster
  • Readiness probe response times (if I can find that information exposed within kubernetes)

These kubernetes specific sources should alleviate the need for kube-router to maintain its own distributed data store to share weight variables. For distributed configuration between kube-router instances a ConfigMap could be updated by a daemon/Pod responsible for calculating load-balancing weights, and kube-router could watch/consume that config.

Research and Justification

There are several ways to get at least pod system-level load metrics in Kubernetes. Application-level load metrics are also increasingly common in Kubernetes apps. Especially true since it's a requirement to expose these metrics to use Horizontal Pod Autoscaling

So here's some potential sources of load metrics that our monitor daemon could support:

  • Prometheus scrape endpoints
  • Pod and Container Readiness Probe events (response times)
  • cAdvisor builtin and custom metrics
  • Ingress Controller metrics (nginx, haproxy, traefik). In real use cases these could be independent or fed through Heapster/cAdvisor or Prometheus

Heapster is a relatively mature and popular cluster addon that collects from several of the above mentioned sources, so it could be a definitive source for system-level load metrics. There is also an effort to standardize this type of information in the apiserver with a metric endpoint. This API might be more apt for app-level load metrics. More info on that in this issue, and implementation work in this repo.

Collaborator

bzub commented Jul 7, 2017

Initial Implementation

The monitor daemon could weight real-servers with:

  • Standard pod metrics provided by Heapster
  • Readiness probe response times (if I can find that information exposed within kubernetes)

These kubernetes specific sources should alleviate the need for kube-router to maintain its own distributed data store to share weight variables. For distributed configuration between kube-router instances a ConfigMap could be updated by a daemon/Pod responsible for calculating load-balancing weights, and kube-router could watch/consume that config.

Research and Justification

There are several ways to get at least pod system-level load metrics in Kubernetes. Application-level load metrics are also increasingly common in Kubernetes apps. Especially true since it's a requirement to expose these metrics to use Horizontal Pod Autoscaling

So here's some potential sources of load metrics that our monitor daemon could support:

  • Prometheus scrape endpoints
  • Pod and Container Readiness Probe events (response times)
  • cAdvisor builtin and custom metrics
  • Ingress Controller metrics (nginx, haproxy, traefik). In real use cases these could be independent or fed through Heapster/cAdvisor or Prometheus

Heapster is a relatively mature and popular cluster addon that collects from several of the above mentioned sources, so it could be a definitive source for system-level load metrics. There is also an effort to standardize this type of information in the apiserver with a metric endpoint. This API might be more apt for app-level load metrics. More info on that in this issue, and implementation work in this repo.

@bzub bzub added this to the v0.2.0 milestone Jul 10, 2017

@SEJeff

This comment has been minimized.

Show comment
Hide comment
@SEJeff

SEJeff Sep 18, 2017

Contributor

There is an ingress doing something vaguely similar, but they are storing traffic stats in etcd.

Contributor

SEJeff commented Sep 18, 2017

There is an ingress doing something vaguely similar, but they are storing traffic stats in etcd.

@SEJeff

This comment has been minimized.

Show comment
Hide comment
@SEJeff

SEJeff Jun 8, 2018

Contributor

Also, heapster has sort of been replaced by the metrics api. Additionally, groupcache might fit into your ideas of an in-memory k/v store with the benefits of scaling out

Contributor

SEJeff commented Jun 8, 2018

Also, heapster has sort of been replaced by the metrics api. Additionally, groupcache might fit into your ideas of an in-memory k/v store with the benefits of scaling out

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment