DNS-backed cluster #207

Closed
wants to merge 5 commits into
from

8 participants

@agleyzer

Per discussion on the finaglers mailing list: https://groups.google.com/d/msg/finaglers/HqfNWJF3qZk/mWD-By0MtOgJ

DnsCluster continuously resolves a host name into a set of SocketAddress instances, loosely based on ZK cluster implementation .

LingeringCluster is a generic Cluster filter that propagates Rem events after a predefined delay, this helps dealing with firewall that does its own DNS resolution.

@mosesn

I like the idea here, but Clusters are deprecated. It doesn't make sense to add a new one right now. Can you get these to work with the Name api?

@mariusae
Twitter, Inc. member

It should be pretty simple to adopt to the new APIs.

@mariusae
Twitter, Inc. member

(This is really cool btw.)

@agleyzer

I've looked at ZkResolver to see how to implement a similar DnsResolver, but it uses a Group internally and there's a comment about taking that out as well (I understand Groups are deprecated too). I wonder if my best strategy is to wait until the dust settles and then take another look? Or am I barking up the wrong tree?

@evnm

The deprecation storm has settled and the Name/Var[Addr] API is the outcome. I think it's safe to assume that it's stable enough to build upon. ZkResolver is still written in terms of Group because no one's had time to migrate StabilizingGroup to the Var[Addr] world.

A DnsResolver would provide a function def bind(arg: String): Var[Addr]. See twitter-server's FlagResolver class for a simple example.

@agleyzer

Thanks for your response, but I am still a bit confused where my DNS stuff would fit in this brave new world... On one hand, it could be used as a poor man's ZK, with multiple machines resolving the same hostname, so that's why I looked at the ZkResolver, but then we can also have a Name implementation, something like this: https://gist.github.com/agleyzer/7363353. What would you guys prefer?

@mariusae
Twitter, Inc. member

I think we just want this to be part of the inet! resolver.

agleyzer added some commits Dec 13, 2013
@agleyzer agleyzer Merge remote-tracking branch 'local/master' ef9e11e
@agleyzer agleyzer Merge remote-tracking branch 'upstream/master'
bd9c489
@mosesn

@agleyzer Not sure if you're still interested in this problem, but I just stumbled on this ticket again. I think @mariusaeriksen is right that this shouldn't be special behavior, this should just be the way that the InetResolver works all the time. Do you still have the bandwidth to make this happen?

@jdanbrown

@mosesn We keep getting bit by inet! resolved names failing to track DNS names that change over time (e.g. AWS ELBs), so I'd be interested in working this into the inet! resolver if no one is tackling it yet.

What's the best approach?

  1. Stick with Name.Bound(Var[Addr]) (Resolver.eval + InetResolver.bind) and periodically push DNS changes to the Var. This would require periodically re-resolving the hostname on a Timer, which seems like it might be overly complicated.

  2. Add some kind of "bind-on-demand" state in Addr so that DNS resolution happens on each request. Is this even possible?—looking at DefaultClient.newStack0 my guess is that it needs the push behavior in (1)...

@roanta roanta closed this May 14, 2014
@roanta roanta reopened this May 14, 2014
@agleyzer

@mosesn sorry I just noticed your question. I might have some bandwidth next week.

@jdanbrown I ended up writing a subclass for Name with periodic changes: https://gist.github.com/agleyzer/7363353, it's been in use for a while, no complains so far.

@mosesn

@agleyzer that solution looks great. Would you mind if @jdanbrown refashioned it to be the default for "inet!"?

@mariusae
Twitter, Inc. member

+1, this looks great.

We should consider doing a few things:

  • limit the total amount of permissible concurrency;
  • have resolution timeouts (will thread interruption work here?).

In the long term, we might consider implementing our own DNS resolver (I think that there was a Netty summer-of-code project for this -- @trustin?) so that we can respect the upstream TTLs, etc. But, I know that anything involving "your own DNS resolver" is tantamount to famous last words.

@agleyzer

@mosesn @jdanbrown you are welcome to use that code, I am glad to be of any help

@mosesn

Awesome. @jdanbrown let me know if you need any help.

@trustin

Yeah, Netty has an asynchronous DNS resolution facility which came from last year's GSoC. It has not been merged yet because it needs some clean-up. Will let you know once it's ready to go.

@mosesn

@agleyzer @jdanbrown looks like this fell through the cracks, do either of you have the bandwidth to take this on?

@jixu

@mosesn , we are using the DNS Resolver written by @agleyzer in our product, and it works well. Can I know what is the progress to support automatically DNS resolving in finagle? I can help on issue if @agleyzer doesn't have time.

@mosesn

Sure, if you want to take it on, we'd be glad to take your PR. None of us at twitter has the bandwidth to do it, so it will have to be community driven.

@agleyzer
@jdanbrown

@mosesn @agleyzer @jixu Yeah, sorry, I dropped the ball on this. I still intend to do it eventually, but @jixu feel free to jump on it sooner if you want. Either way, I'll follow up here before I spend much time on it to make sure I'm not racing with someone else.

@jixu

@mosesn I have created a pull request for this issue:
#282
Please help to review when you have time.

@mosesn

#282 implemented this, so I'm going to close this PR. Thanks to @jixu and @agleyzer for this super cool feature!

@mosesn mosesn closed this Aug 6, 2014
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment