Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(kuma-cp) scaling-up issue #1282

Merged
merged 3 commits into from
Dec 9, 2020
Merged

fix(kuma-cp) scaling-up issue #1282

merged 3 commits into from
Dec 9, 2020

Conversation

lobkovilya
Copy link
Contributor

@lobkovilya lobkovilya commented Dec 7, 2020

Summary

After #1036 Kuma supports application probes on Kubernetes. But we didn't take advantage of it and didn't take probes into account when generating EDS. That means during scaling up some requests might return 503. This PR extends Dataplane model with the new field health:

inbound:
- tags:
      kuma.io/service: backend
  health:
      ready: true

And then during generating EDS we don't include endpoints that are not ready yet.

Caution

This PR doesn't solve the scaling-down issue. That means during scaling down you can observe some 503. The scaling-down issue will be solved when we implement a retry policy.

Issues resolved

Fix #1132

Documentation

NOT READY

…er status in Kubernetes

Signed-off-by: Ilya Lobkov <ilya.lobkov@konghq.com>
@lobkovilya lobkovilya requested a review from a team as a code owner December 7, 2020 16:42
Signed-off-by: Ilya Lobkov <ilya.lobkov@konghq.com>
@@ -67,6 +67,10 @@ message Dataplane {
// e.g. kuma.io/service=web, version=1.0.
// `kuma.io/service` tag is mandatory.
map<string, string> tags = 2 [ (validate.rules).map.min_pairs = 1 ];

message Health { bool ready = 1; }
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Docs, please indicate that it is not required and if null, then assumed that it is healthy

@@ -73,6 +73,9 @@ func fillDataplaneOutbounds(outbound core_xds.EndpointMap, dataplanes []*mesh_co
iface := dataplane.Spec.Networking.ToInboundInterface(inbound)
// TODO(yskopets): do we need to dedup?
// TODO(yskopets): sort ?
if inbound.Health != nil && !inbound.Health.Ready {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

drop a comment why inbound.Health == nil is assumed healthy

Signed-off-by: Ilya Lobkov <ilya.lobkov@konghq.com>
@lobkovilya lobkovilya merged commit 8417151 into master Dec 9, 2020
@lobkovilya lobkovilya deleted the feat/dp-inbound-health branch December 9, 2020 14:12
mergify bot pushed a commit that referenced this pull request Dec 9, 2020
(cherry picked from commit 8417151)
nickolaev pushed a commit that referenced this pull request Dec 10, 2020
(cherry picked from commit 8417151)

Co-authored-by: Ilya Lobkov <ilya.lobkov@konghq.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Kubernetes: Dataplanes join and leave mesh gracefully
2 participants