Node-local services

[therc]

Sometimes users require daemons to run on each node. Scheduling them is easy, with DaemonSets. Discovery is more complicated and currently requires e.g. iptables hacks that bypass Kubernetes altogether. Possible use cases:

• monitoring agents such as datadog
• logging agents such as fluentd
• authenticating proxies such as aws-es-proxy or loasd (LOAS Daemon #2209)

At the very least, this new kind of services should support 1:1 service->daemon mapping. For cases like loasd, it would be ideal to support N:1 (a number of services all getting forwarded to the same daemon, with the daemon able to tell which service each connection was meant for).

cc @thockin

• Write proposal
• Implementation
  • Extend Service API
  • Extend kube-proxy and/or kube-dns

[Random-Liu]

@thockin This is also very useful for node problem detector. Both node problem detector and kube-proxy are running as DaemonSet. If we want kube-proxy to report problem to node problem detector via network, we can rely on specific port in host network now.
However, if we have this, it could be cleaner. :)

[therc]

I created #28637. Feel free to poke holes in it.

[gtaylor]

To add some more examples to the list that we're trying to figure out at Reddit:

• Per-node pgbouncer
• synapse (already in use outside of k8s)
• tallier (statsd compatible)

[bgrant0607]

Thread with use cases:
https://groups.google.com/forum/?utm_medium=email&utm_source=footer#!msg/kubernetes-dev/cK0djP-0ajY/HGeQ_oz0BAAJ

[sandys]

Another example i would like to add is linkerd. https://linkerd.io

[ivanilves]

👍

[k8s-github-robot]

@therc There are no sig labels on this issue. Please add a sig label by:
(1) mentioning a sig: @kubernetes/sig-<team-name>-misc
(2) specifying the label manually: /sig <label>

Note: method (1) will trigger a notification to the team. You can find the team list here.

[cmluciano]

/sig network

[fabiand]

Ping?

[arussellsaw]

@thockin is this still the relevant issue? you mentioned this work was ongoing at the London Kubernetes meetup a few weeks ago

[igorpeshansky]

This issue is definitely still relevant for our team. Are there any obstacles to merging #28637?

[Vlaaaaaaad]

This is also relevant for us. Any updates?

[ColinChartier]

Kludge until this gets resolved:

1. Make a ClusterIP service with no selector whose endpoints are the same as the service's IP, e.g., with kubectl edit endpoints/kubernetes (my service ip is 10.96.0.1)
2. Make a DNAT iptables rule to redirect the kubernetes request to the node itself, e.g., if your WAN ip is 1.1.1.1, the command would be: sudo iptables -t nat -D OUTPUT --destination 10.96.0.1 -p tcp --dport 443 -j DNAT --to-destination 1.1.1.1:443 (this redirects traffic from 10.96.0.1:443 to 1.1.1.1:443)

[fejta-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

[fabiand]

Please keep this in mind.
For daemonsets this is somethign we want.

[jobrs]

👍

[fabiand]

/remove-lifecycle stale

[techdragon]

Can anyone familiar with the work, give an update on how far off we are from having this fixed? The arbitrary topology stuff seems cool, but its also giving me a growing feeling that a case of architecture astronautics is taking place. Has anyone built a node local proxy or something else to fix this? LinkerD looks interesting my primary use for this needs UDP support. (Metrics/StatsD, etc)

[johnbelamaric]

We cut out the complicated stuff and are trying to make it as simple as possible. See kubernetes/community#2846 - we hope to have this KEP accepted this cycle (meaning really soon). Then implementation would happen in 1.14 timeframe (for alpha).

…

On Wed, Nov 14, 2018 at 9:01 PM Samuel Bishop ***@***.***> wrote: Can anyone familiar with the work, give an update on how far off we are from having this fixed? The arbitrary topology stuff seems cool, but its also giving me a growing feeling that a case of *architecture astronautics* is taking place. Has anyone built a node local proxy or something else to fix this? LinkerD looks interesting my primary use for this needs UDP support. (Metrics/StatsD, etc) — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#28610 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AJB4s9mrKm2vqKESmI0E0wciuYjVUsQhks5uvPU0gaJpZM4JHPm7> .

[techdragon]

@johnbelamaric Thanks for referencing that KEP. I didn't see that one while I was reading and following the different chain of links to other issues/repos/etc to see how things were progressing.

[lentzi90]

Cross linking, since the KEPs moved. The KEP for this is now here.

Issue available here targeted for v1.15

[draveness]

KEP is now here https://github.com/kubernetes/enhancements/blob/master/keps/sig-network/20181024-service-topology.md

[johnbelamaric]

Available as alpha in 1.17: https://kubernetes.io/docs/concepts/services-networking/service-topology/

/close

[k8s-ci-robot]

@johnbelamaric: Closing this issue.

In response to this:

Available as alpha in 1.17: https://kubernetes.io/docs/concepts/services-networking/service-topology/

/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.